As well as quantity, the quality of this research has greatly improved, with scientific principles being applied to isolate an intervention’s effect from other possible influences, says Howard White, executive director of the International Initiative for Impact Evaluation (3ie).
Speaking ahead of 3ie’s series of events in London, United Kingdom, this week, that will discuss impact evaluations for development, White identifies a 2006 report — When will we ever learn? — as the turning point in the field.
“If development practitioners wanted to assess their impact, there was no rigorous evidence based on causal analysis to actually know if programmes were making a difference.”
Howard White, 3ie
“It pointed out that, if development practitioners wanted to assess their impact, there was no rigorous evidence based on causal analysis to actually know if programmes were making a difference,” he tells SciDev.Net.
This shortcoming triggered the rise of impact assessments — aided by 3ie’s launch in 2008 — based on the scientific principle that comparison with a ‘control group’ given no assistance “allows you to say with certainty if a programme was having the desired effect”, he says.
By taking two populations — matched carefully to ensure they are broadly comparable — and only delivering a development programme to one, any changes observed in this group can be directly attributed to the intervention, White says.
3ie has already spent around US$80 million to fund around 150 impact studies in more than 50 countries, he says.
A 3ie database now contains more than 2,500 such reports, including studies conducted by other development agencies such as the World Bank, and across a wide range of development areas, such as governance, climate change and agriculture, he adds.
This growing evidence base is filtering into policy, with many governments, including those of Brazil, China, Colombia and Mexico, setting up systems to allow impact assessment to guide development activities, says White.
3ie is also working with the Ugandan government to systematically apply impact evaluations to a broad range of development programmes, which will go on to inform future policy decisions, he adds.
All those receiving 3ie funds to carry out impact assessments must outline how their research will be relevant to decision makers to ensure their work is orientated towards policy spheres, says White.
But in the wider impact assessment community, many researchers still do not recognise the importance of engagement with policymakers — an issue 3ie hopes to raise during this week’s series of talks, workshops and seminars, he adds.
Scott Rozelle, a fellow at the Freeman Spogli Institute for International Studies at Stanford University, United States, who runs a universities group focused on impact evaluation, agrees that the field has become much more visible.
“In China ten years ago, no one even knew what impact evaluation meant,” he tells SciDev.Net. “Now they have just passed a law to require impact assessment for all new development policies and programmes.”
But he warns that impact assessment is only a tool for guiding development and cannot be a “cure all” for bad policies.
> Link to When will we ever learn?