More than 15 years ago I had my first encounter in India through a three-month-long Fulbright lectureship. During the first month of my stay I read everything I could get my hands onnewspapers, magazines, journals-and met with a wide variety of people in Delhi. At the end of that month, I had the sense that I had gained an understanding of Indian politics, policy and administrative practices. But by the second month, I realized that I had no idea of what was really occurring. The society was so complex that it was hard to fathom what was taking place. Finally at the end of the third month, I wasn't even sure about the questions that should be asked to understand that society.
I have the sense that performance analysis and evidence-based policy making follow a similar pathway. The early encounters in the field make many feel some level of confidence that the collection of information about performance provides both scholars and decisionmakers with data that can be used to determine what works and what does not. But as Carolyn Heinrich suggests in this important and convincing piece, that level of optimism may be inappropriate. The four questions that she presents to frame her discussion begin to paint a picture of the complexity of the evidence-based policy and performance management movements and the dilemmas that are posed for its practitioners.
She asks us how tradeoffs can be made between knowledge creation and useable information. She notes how difficult it is to determine how quality and reliability of information will be judged. She asks whether users of information should have a role in the production of information. And she describes the conflict between processes that produce rigorous information and evidence and processes that allow public input and stakeholder representation.
Like some others before her in the evaluation field (e.g. Carol Weiss, Robert Stake, David Cohen and Charles Lindblom), Heinrich focuses on the limitations of the traditions of scientific method when applied to the policy world. Many of the issues that she discusses in this piece on performance analysis and evidence-based policy making have been confronted by those in the past who have sought to apply scientific methods and approaches to policy making. For some, the