UNICEF tackling 'disconnect' with evidence tool
- Evidence mapping tool shows where research is needed
- Many approaches ‘unsupported by rigorous evidence’ at the moment
- Tool represents small contribution to larger evidence puzzle
“Across large parts of our organisation, [the] instructional incentives, time and space to slow down, take stock, learn lessons and draw upon the available global evidence base for improved decision-making are still evolving,” she told SciDev.Net.
Albright made the remarks as part of a consultation on an evidence mapping tool that she said is part of a broader push to build a stronger evidence culture at the agency. She cautioned that her comments were her personal views and did not represent the views of UNICEF.
The tool, called Evidence Gap Map (EGM), is a matrix designed to help policymakers and practitioners quickly see where high-quality studies are plentiful, or where more research is needed. It indicates the availability of evidence — not what it says — from various sources, not just peer-reviewed studies. “The global evidence base of effective interventions to meet the … challenges facing children is mostly weak, scattered and too poorly translated to be usable,” Albright told SciDev.Net. “Many widely-used approaches are unsupported by rigorous evidence.”
“Many widely-used approaches are unsupported by rigorous evidence.”
At UNICEF, this has partly to do with the agency’s original service delivery mandate that goes back to the aftermath of the World War II, according to Albright.
Working with the Campbell Collaboration, a social sciences organisation, to produce the mapping tools, is a small contribution to the larger evidence puzzle, she explained.
The overall goal is to generate the practical knowledge needed to better understand what works, and to understand barriers to implementation of interventions. This ultimately avoids the costs of duplication and enables the agency to reach more children, said Albright.
But to get there, there are hurdles to overcome. Agency staff currently have trouble even getting their hands on evidence they need – it’s only available through a research access account managed “on a shoestring”, Albright explained.
Laurence Chandy, UNICEF’s director of data, research and policy, added that staff are so burdened with operational work that they simply do not have the time to read about the latest studies. This means the evidence used in programmes can be outdated.
The UN agency is one of the organisations assessing how useful the mapping tool can be in practice. They also include the International Rescue Committee (IRC), Sightsavers, the Beam Exchange and USAID.
At a consultation this month (10 November) — which was part of London Evidence Week organised by 3ie, the International Initiative for Impact Evaluation, which also develops EGMs — Annalisa Brusatti, Senior Technical Advisor for Child Protection at IRC, echoed the need for evidence to improve the effectiveness of interventions on the ground. But in practice, there are limitations, she said.
Brusatti, who shared early experiences from the IRC’s use of the tool, said that although the maps make finding evidence easier, they often fail to capture the context that makes evidence meaningful for staff who need to use it on the ground.
Yet Howard White, CEO of the Campbell Collaboration, says that although context matters and can affect the impact of an intervention, its role can be overstated or misunderstood.
“[The] IRC transplanted a developed-country model of parenting programmes to developing countries,” he noted. But although parents in some cultures may not believe in stimulating infants with things like storytelling and play, the evidence shows that psychosocial development interventions do help cognitive development, he explained.
“So it is important to understand what elements of a programme are transferable and in what ways they need to be adapted,” said White.
Both the IRC and UNICEF continue to promote use of the tool internally. “There are still relatively few evidence gaps maps related to international development,” says Albright.
UNICEF is developing a mega-map of child welfare interventions that is planned for completion next year. This is a type of map that shows gaps in evidence synthesis by factoring in systematic reviews and gap maps, but no primary studies.
Development of the mapping tools is a work in process. According to White, feedback from users suggests they prefer maps that are easy to understand and enable them to “drill down to the underlying study”.