Q&A: The best ways to track the SDGs

The indicators to measure the success of the eight Millennium Development Goals (MDGs) were an afterthought devised by UN officials “in a dark room in New York and thrust down countries’ throats” and, as such, were a lost opportunity, says Guido Schmidt-Traub, executive director of the UN Sustainable Development Solutions Network (SDSN).

To ensure the same does not happen with their replacements, the Sustainable Development Goals (SDGs) that will run from 2015 to 2030, the SDSN has been working to collate the various specific indicators that could be included. For nations to track progress on the 17 SDGs currently proposed a significant expansion of the public sector would be required, but tracking may also mean making better use of private-sector firms’ own data, he says.

But indicators are not an exact science, and some developing countries may struggle to gather the massive amount of new data needed. Here, Schmidt-Traub talks about this challenge and the “data revolution” that could help to monitor the SDGs.
 
What did the MDG indicators teach us about how to monitor the SDGs?

The SDSN started working on indicators almost a year ago. We realised that to better understand the operational implications and feasibility of the Sustainable Development Goals and targets we needed to have a sense of what indicators out there could be used.

The other motivation was that, with the MDGs, data really was not, in my view, a great success. It was an afterthought. The MDGs and targets were adopted and then, at some point, a group of UN officials met in a dark room and ticked off the list of available indicators, which somehow mapped to the MDGs. The result was that some MDG indicators don’t make much sense, and there are huge gaps. It was a bit of a lost opportunity. There was never any discussion or serious thinking about what we needed to measure.

The other problem, which we highlighted in a 2013 report we did on MDG indicators, is that the data is extremely patchy and comes with tremendous time lags. Poverty data is often years out of date when it comes out so it cannot be used in annual monitoring. It could not be used as a management tool for the MDGs and, by extension, the SDGs.
 
There will be many more indicators with the SDGs than there were with the MDGs, but how many are feasible?

For the SDGs, everyone was proposing goals and targets. UN-Water came up with around 42 indicators for the water and sanitation goals alone. There were something like 900 indicators, which, frankly, is not helpful. There was an urgent need for prioritisation. So SDSN started this work on indicators and we were the only ones to look at the whole range of indicators very, very early on.

We said we need to cap the number of indicators at 100. Many national statistics offices said even that number would push them beyond their comfort zone. They preferred significantly fewer. Then, everyone else who commented on the draft version of our report [Indicators for sustainable development, released for public consultation in February 2014] said: ‘It’s great, but please add the following five indicators’.

If it could be fewer, I’d love it, and anything more I struggle to see how it could be implemented, which is an indication of the feasibility of the 169 targets proposed by the UN’s Open Working Group on Sustainable Development Goals (OWG). There really has to be some consolidation.
 
You talk about the need to limit the number of indicators, but in some of your reports you also talk about additional complementary metrics. Isn’t that a contradiction?

In the report we talk about three sets of data. The first is core indicators, which we want to cut back to 100. These are the indicators akin to the MDG indicators that we propose every country should report on.

Many governments, of course, have vastly more data than 100 indicators and we identified what we call ‘tier 2’ indicators that respond to specific national needs, for example neglected tropical diseases are something that, by definition, only apply to tropical countries.

There are many different views, almost philosophical or ideological views, on inequality and how to measure it. We don’t think the global agenda can cover all those metrics. So if countries want additional metrics, they should include them in their own annual national reports.

These are the two main types of metric we are suggesting, but, of course, they don’t cover everything.

Then there are ‘unofficial’ metrics. That’s where you get all the excitement about the ‘data revolution’. We think there could be some interesting, innovative ways to get proxy data. For example on annual food loss. The way the official system goes about it is with expensive surveys that can only be done once every five years and take a long time to evaluate. That may be valuable, but there are also global food companies active in almost every market and they have internal business data. We’ve not done this work yet, but I’m pretty sure these could be used to construct proxies that would be available on a monthly or even quarterly basis.
 
You propose annual reporting, but won’t this require quite a change when some countries couldn’t even get to grips with four-year reporting in relation to the MDGs?

The boldest recommendation in our report is for annual reporting. It gets the backs of many statisticians up because it is far away from what they are currently doing. Looking at this in more detail, there is tremendous scope for moving towards annual reporting cycles. In some variables it doesn’t make sense because you don’t have much change — life expectancy would be one example, maternal mortality another — but those are a minority.

Now, imagine a situation where every year a country puts out this data in some form, and every year the people in the sector concerned get together on a regional or perhaps global basis to discuss and exchange data. It would be an effective way to highlight discrepancies and inconsistencies in data on that topic from different sources — and have a discussion away from the media limelight on how to address them. It won’t address every issue — the SDGs cannot be the single tool to address corruption [one of the major sources of discrepancies in data], but it can play an important role.

Annual reporting is important for turning the SDGs into what the MDGs were not, namely a management tool. In the health sector, the MDGs really were a tool, but that’s because that sector has monthly and quarterly, rather than annual, reporting. In all the global health campaigns, this incredible depth of data reporting was one of the things that drove change.

So our view is that this is possible, even if it will require change and significant additional investment in capacity to analyse and collect data. We had an experts’ workshop in June with many national statistics offices, and the statistical leaders from the multilateral system, including the OECD (Organisation for Economic Co-operation and Development) and many others. After two days of discussions, there was a growing sense that, actually, this is doable.

If countries are to report their indicators annually it will require additional capacity and investment. How will that be paid for?

We are looking into this. It is the topic for the UN Financing for Development conference [in July 2015 in Addis Ababa, Ethiopia]. The Intergovernmental Committee of Experts on Sustainable Development Financing will be issuing a report before that meeting, and it has come to the same conclusion [as the SDSN]: meeting the SDGs will require a significant increase in global investment. We estimate US$2-3 trillion will have to be invested additionally in developing countries over the SDGs’ lifetime.

Of course, [this money should come primarily from] domestic resources and private resources, and only any residual should be financed through overseas development assistance (ODA). But there is no doubt about it, there is clearly a need for additional ODA. I believe that money would be very well invested though.
 
What is the role of technological innovation in trying to ensure the goals can be met? Can these goals push innovation in certain directions?

The goals must push innovation. That is the conclusion of our work in every area, in particular the economic and environmental interface. Trade-offs still exist, but those can only be addressed through vastly more technological progress.

We’ve looked at this in great detail. We recently put out a ‘deep decarbonisation’ report that looks at this in the area of climate change. One of the key conclusions was that we need vastly faster technological progress than we have seen in the past, and vastly faster innovation than nations believe would be possible through a go-it-alone strategy. But if, for example, you get all the world’s major car manufacturers together — there’s only seven of them globally — then the challenge of meeting the goal of zero particle emissions by 2030, [which has been proposed during UN climate negotiations], can be met.

Technology is vital and, unfortunately, underappreciated in much of the debate. There’s also an under appreciation of how technologies develop. The transformations needed for sustainable development — the energy transformation, the agriculture transformation and so forth — will not be driven by the private sector alone. We need much greater international coordination and organisation of technological progress. We need technology road maps. There is an important role to be played by roundtables where you bring science and business — and others who can speak about specific, important areas of technologies — around the table and get them to address these issues.
 
The UN secretary-general’s high-level panel has called for a ‘data revolution’. That term is all over the SDSN’s latest report as well, but is it more than a slogan?

It means different things to different people. The high-level panel has done a great service to the world by raising this issue. But there is a pitfall we mustn’t fall into. Yes, a lot still needs to be done to improve data, particularly in low-income countries. But some of this work is pretty basic. It’s about doing more of the same, but can we do it better?

So, do we need household surveys? Absolutely. Do countries need to run censuses? Absolutely. Do we need household budget surveys, labour market surveys, business surveys? Absolutely.

I have not seen any credible proposal for new data tools — social data or Google Analytics, you name it — that could replace a core child mortality indicator. And that’s where it’s really important to make the data revolution discussion very specific.

The UN secretary-general recently announced a high-level advisory group on the data revolution that will submit a report by the end of the year, so that is a very short-term assignment. [Ed: This report, A world that counts, has now been released.]

But let’s not forget the basics — the capacity development, the capacity strengthening of national statistical offices in Africa — that are also needed. The science community can play an important role here.

The can force specificity into this debate. If they don’t there is a danger that ‘data revolution’ does become just a slogan. Actually, it’s not a slogan, the intentions are the right ones. If the data can be collected through Google Analytics or in other ways, that’s fantastic. Let’s look at them and see if they actually meet the policy. If there are cheaper, more revolutionary ways of collecting information that offer a better way than some of the more traditional indicators, I will be delighted.

But it should not be some abstract discussion around the fact that household surveys may someday become obsolete, because they are not obsolete right now.

I am a big believer in the ‘revolution’ side of the data revolution, but there is a lot of evolution as well that needs to happen. National statistical offices will not disappear, and we need to strengthen them. So if we don’t manage to track the SDGs well, and I would argue annually, then we have missed out on the data revolution.
 
What about regional goals and targets. How would these work?

There are different proposals out there. They include the high-level panel’s proposed goals, for example that child mortality is reduced by x, and then every country — if I understand the position correctly — then defines what x should be for them.

I understand the argument for such an approach because every country may be unique, but there are also pitfalls. It would make everything more complex. Once the goals are adopted there would be a three- to four-year process for governments to agree their national numbers, so we would lose time.

But these goals about the “getting to Zero” agenda; to make sure that, by 2030, extreme deprivation and human wellbeing across health, education and basic infrastructure, hunger, income poverty are addressed. For this agenda, it is possible to have global benchmarks.

I am aware of the arguments many academic colleagues will put forward: that some targets may be impossible to achieve for some countries and therefore they may be demotivating. That issue does exist, but overall it is very small and what is vastly more important is that the global community supports these vulnerable countries so that failure to meet a goal is not just a failure of the country but also a failure of the global community.
 
Q&As are edited for length and clarity.
 
> Link to the SDSN’s November report on indicators for the SDGs

> Link to the SDSN’s Deep decarbonisation report

> Link to the UN A world that counts report