Bringing science and development together through news and analysis

Justin Forsyth, the Deputy Executive Director of the United Nations Children's Fund meets with students and teachers at a camp school. More than 120,000 Rohingya live at the camp.
  • UNICEF tackling 'disconnect' with evidence tool

Justin Forsyth, the Deputy Executive Director of the United Nations Children's Fund meets with students and teachers at a camp school. More than 120,000 Rohingya live at the camp.
Copyright: Panos

Speed read

  • Evidence mapping tool shows where research is needed

  • Many approaches ‘unsupported by rigorous evidence’ at the moment

  • Tool represents small contribution to larger evidence puzzle

Shares
UNICEF has been testing an evidence-mapping tool to bridge a “big disconnect” between its activities and the evidence behind them, according to Kerry Albright, chief of Research Facilitation and Knowledge Management at the UN agency’s Office of Research-Innocenti.

“Across large parts of our organisation, [the] instructional incentives, time and space to slow down, take stock, learn lessons and draw upon the available global evidence base for improved decision-making are still evolving,” she told SciDev.Net.  

Albright made the remarks as part of a consultation on an evidence mapping tool that she said is part of a broader push to build a stronger evidence culture at the agency. She cautioned that her comments were her personal views and did not represent the views of UNICEF.

The tool, called Evidence Gap Map (EGM), is a matrix designed to help policymakers and practitioners quickly see where high-quality studies are plentiful, or where more research is needed. It indicates the availability of evidence — not what it says — from various sources, not just peer-reviewed studies.
“The global evidence base of effective interventions to meet the … challenges facing children is mostly weak, scattered and too poorly translated to be usable,” Albright told SciDev.Net. “Many widely-used approaches are unsupported by rigorous evidence.”

“Many widely-used approaches are unsupported by rigorous evidence.”

Kerry Albright


At UNICEF, this has partly to do with the agency’s original service delivery mandate that goes back to the aftermath of the World War II, according to Albright.

Working with the Campbell Collaboration, a social sciences organisation, to produce the mapping tools, is a small contribution to the larger evidence puzzle, she explained.

Barriers, duplication
The overall goal is to generate the practical knowledge needed to better understand what works, and to understand barriers to implementation of interventions. This ultimately avoids the costs of duplication and enables the agency to reach more children, said Albright.

But to get there, there are hurdles to overcome. Agency staff currently have trouble even getting their hands on evidence they need – it’s only available through a research access account managed “on a shoestring”, Albright explained.

Laurence Chandy, UNICEF’s director of data, research and policy, added that staff are so burdened with operational work that they simply do not have the time to read about the latest studies. This means the evidence used in programmes can be outdated.

The UN agency is one of the organisations assessing how useful the mapping tool can be in practice. They also include the International Rescue Committee (IRC), Sightsavers, the Beam Exchange and USAID.

At a consultation this month (10 November) — which was part of London Evidence Week organised by 3ie, the International Initiative for Impact Evaluation, which also develops EGMs — Annalisa Brusatti, Senior Technical Advisor for Child Protection at IRC, echoed the need for evidence to improve the effectiveness of interventions on the ground. But in practice, there are limitations, she said.

Brusatti, who shared early experiences from the IRC’s use of the tool, said that although the maps make finding evidence easier, they often fail to capture the context that makes evidence meaningful for staff who need to use it on the ground.

Yet Howard White, CEO of the Campbell Collaboration, says that although context matters and can affect the impact of an intervention, its role can be overstated or misunderstood.

“[The] IRC transplanted a developed-country model of parenting programmes to developing countries,” he noted. But although parents in some cultures may not believe in stimulating infants with things like storytelling and play, the evidence shows that psychosocial development interventions do help cognitive development, he explained.

“So it is important to understand what elements of a programme are transferable and in what ways they need to be adapted,” said White.

Both the IRC and UNICEF continue to promote use of the tool internally. “There are still relatively few evidence gaps maps related to international development,” says Albright.
 
UNICEF is developing a mega-map of child welfare interventions that is planned for completion next year. This is a type of map that shows gaps in evidence synthesis by factoring in systematic reviews and gap maps, but no primary studies.

Development of the mapping tools is a work in process. According to White, feedback from users suggests they prefer maps that are easy to understand and enable them to “drill down to the underlying study”.
Republish
We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit SciDev.Net — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on SciDev.Net” containing a link back to the original article.
  4. If you want to also take images published in this story you will need to confirm with the original source if you're licensed to use them.
  5. The easiest way to get the article on your site is to embed the code below.
For more information view our media page and republishing guidelines.