Now the hype is about ‘big data’ — and it is good that SciDev.Net is presenting the claims and counterclaims to help us assess its importance.
Having tried hard to get my head round it (or part of it), I have to admit that I need more time. I can see the point, but I remain unconvinced that it really is the tool to help the poorest. Like randomised controlled trials, which work well for assessing drugs, I think it would have some development-related applications, but I doubt it is the magic bullet claimed by its most enthusiastic advocates.
“I will be surprised if big data as a technologically induced salvation myth comes true.”
I am not even sure that what most people think of as big data is really the “data revolution” that the UN Secretary-General’s High Level Panel on post-2015 development are looking for.
I recently asked esteemed economists Finn Tarp and Tony Addison of UNU-WIDER (the UN University World Institute for Development Economics Research) what sort of data they think are needed for a data revolution in development. Their answer was more prosaic than the ‘magic’ of big data.
Properly collected data, for example through regular household surveys, provided the material needed to check what happened to poor households in Vietnam during the switch to a market economy. As Tarp points out, despite the country growing rapidly, many households initially became poorer. I cannot see how that would be picked up from mobile phone data, for instance.
Big data may help with some analyses of migration or the spread of infection, but I have yet to see anything that would enable you to design policies to deal with the impacts of inequality. I suspect there is no short cut: if you want good development data, strengthening national statistical offices (NSOs) so they can deliver consistent, long-term data collection and analysis will remain the front-line. This will need human resources, good management, finance, computing facilities, transport, communication and office equipment, good statistical practices and a sound regulatory framework, as British economist Jeffery Round argues. 
It could well be that NSOs can benefit from big data in the health and education areas — as statisticians Michail Skaliotis and Ceri Thompson recently argued.
The big data debate is definitely one to follow — but my money (whether transferred by mobile phone or not) is on poverty eradication requiring careful collection and analysis of household data, political action to redistribute income and provide social safety nets, and carefully designed policies.
I recall the phrase of a very political French theologian, Georges Casalis, who said “les idées juste ne tombent pas du ciel”. To paraphrase: correct ideas will not fall out of the sky.
So, I will read and learn more. But I will be surprised if big data as a technologically induced salvation myth comes true. I predict that it will produce some interesting applications. But much of the hard work on development statistics will need to be done at ground level.
Roger Williamson is an independent consultant and visiting fellow at the Institute of Development Studies at the University of Sussex, United Kingdom. Previous positions include organising nearly 80 international policy conferences for the UK Foreign Office and being head of policy and campaigns at Christian Aid.