"Major gaps" in the evidence base relating to the impact of development aid on efforts to meet the UN Millennium Development Goal (MDG) on maternal and reproductive health outcomes (MDG 5) need to be addressed to ensure that aid's true impact can be measured, according to a study.
In the first ever such exercise, researchers systematically reviewed studies across 27 countries on aid-funded interventions designed to improve maternal and reproductive health, in an effort to summarise the evidence of the impact of official development aid on MDG 5 outcomes.
Even with a "generous application" of the MDG definitions, which led to the inclusion of studies that used proxy measures for some of the individual indicators within the goal, the review found "discernible" gaps in the evidence base on aid interventions targeting MDG 5.
The most notable related to indicators MDG 5.4 (adolescent birth rate) and 5.6 (unmet need for family planning). Of the 30 studies reviewed, only three provided any evidence relating to these indicators.
In the light of such gaps, the authors call for rigorous, independent and peer-reviewed studies that attempt to make a "plausible association" between the mode of aid delivery and MDG 5-related health outcomes, and that provide greater detail about aid interventions and outcome data.
"I suppose it's the first such evaluation [of its type] because the development sector is only now turning to what might be termed more rigorous evidence-based methodologies, drawing heavily from public health policymaking," principal investigator James Smith, professor of African and development studies at the University of Edinburgh, United Kingdom, tells SciDev.Net.
Rachel Hayman, head of research at the International NGO Training and Research Centre, Oxford, United Kingdom, and a co-author of the study, says: "In the last two years, there has been a huge debate in the sector about evidence, leading to a definite trend among donors and the larger NGOs, for example, to tighten up their systems for gathering data on their projects and activities."
The research team are hopeful that their conclusions will inform the process of setting up and evaluating the Sustainable Development Goals that are set to replace the MDGs in 2015.
Hayman says she hopes the review shows how the management of information on development interventions can be improved.
"We need to promote not just more rigorous [monitoring and evaluation] but also 'appropriate' M&E and evidence-based research," she says.
Timothy Powell-Jackson, a lecturer in health economics at the London School of Hygiene & Tropical Medicine, United Kingdom, says the study shows that the development aid research community needs to invest more in data systems and rigorous evaluation.
"The research agenda should focus on understanding what works and why. The impact of aid or the contribution of a particular donor is less relevant in my view. The best research addresses questions of policy relevance, draws on sound theory and uses rigorous research designs," he says.
The study was published in PLOS ONE last month (22 February).
Link to the study
PLOS ONE doi:10.1371/journal.pone.0056271(2013)