Data bias fuelling flawed AI climate predictions -study

[NAIROBI] Developing countries risk missing out on important global climate action due to unreliable weather predictions resulting from biased data fed into artificial intelligence (AI) tools, researchers warn.

Biases in the collection of climate data, which AI-driven climate modelling tools rely on, can limit the usefulness of such emerging technologies for climate scientists trying to predict future scenarios and guide global action, says the paper published (17 August) in the Nature journal npj Climate Action.

AI computer programmes used in climate science are tailored to trawl through complex datasets in search of patterns.

“Our recommendation is that whoever is building a climate action AI, must be aware of the data gaps and embedded biases and what efforts are needed to reduce such gaps.”

Ramit Debnath, assistant professor of computational social science, University of Cambridge

But lack of information from certain locations, time periods or societal groups create “holes” in the data that can lead to inaccurate climate predictions and misleading conclusions, the researchers say.

Lead author Ramit Debnath, assistant professor of computational social science at the University of Cambridge, UK, says these data gaps are more pronounced in the global South due to challenges accessing datasets for all types of modelling and analytical purposes.

“The data gap is hard to quantify accurately, but the general trend is most AI-led climate companies are based in the global North,” Debnath told SciDev.Net.

It means there are “high chances the models are well-calibrated and built for global North scenarios as they already have appropriate data and weather monitoring infrastructure”, according to Debnath.

Biases can be broadly classified into biased programing, biased datasets and biased algorithms. The study says programing and algorithms are easier to correct as they are mathematical.

However, biased datasets are the toughest because the data does not exist and it takes significant capital and human investment to build it from the ground up, researchers say.

For example, many instances of generative AI – technology that can produce content such as text and images – shows it struggles with topics around the global South, especially when asked about local contexts in Sub-Saharan Africa, India, or any other non-western nations, Debnath explained.

“Similarly, many reports suggest present-generation AI technologies often show biases in job application results based on names, skin colour, etc,” he added.

“Such biases, although not directly related to climate action, show that lack of representative datasets is a challenge in using AI tools for any form of decision-making purposes.”

The consequence of biased climate data is that it will provide inaccurate predictions of extreme weather events, say the researchers.

Poorer global South nations are already more vulnerable to climate change impacts. Inaccurate predictions due to biased or lack of datasets will further slow mitigation and adaptation efforts—leading to more climate-induced damages to property, human and social capital, they argue.

“The most important finding of our study is data justice can be key to climate justice,” said Debnath, who is also a fellow of the university’s Cambridge Zero climate change initiative.

“Our recommendation is that whoever is building a climate action AI must be aware of the data gaps and embedded biases and what efforts are needed to reduce such gaps.”

‘Human-in-the-loop’ AI solutions

The authors suggest the “data holes” can be filled by human knowledge using a so-called “human-in-the-loop” design to offer AI climate change programs a hand in checking the reliability and context of data used to improve the accuracy of predictions.

They cite as an example the AI chatbot model ChatGPT, which can ask human users follow-up questions, admit mistakes, challenge incorrect premises and reject inappropriate requests.

The researchers also recommend strengthening data infrastructure in the global South and investing in home-grown data start-ups like COCO-Africa, a platform designed to bridge the data gap by collecting and curating local and context-driven datasets from Sub-Saharan African countries.

According to Joab O. Odhiambo, senior lecturer in the department of mathematics at Meru University of Science and Technology in Kenya, AI is critical for accurate weather monitoring in Africa, especially to avert climate disasters.

“AI can revolutionise weather monitoring and disaster management by enhancing prediction accuracy and enabling timely response to events such as droughts, floods and heatwaves,” he told SciDev.Net.

He stresses, however, that practicality of accurate weather monitoring and subsequent interventions depends on various factors including technology, infrastructure, education, funding and governance.

Obed Ogega, climate scientist and programs manager at the African Academy of Sciences (AAS), says the use of AI tools in climate change projections and adaptation or mitigation strategies is still in its development stage.

“It will take some time before we can have a full understanding of the accuracy [or] effectiveness of AI tools in climate science and climate action,” Ogega told SciDev.Net.

He noted that, due to more limited observational data in regions such as Sub-Saharan Africa that can be used to “train” AI tools , the “holes” referred to in the article are likely to be bigger there and the risk of misinformation greater.

“As noted in the article, there is need for a ‘human-in-the-loop’ approach in the use of AI tools—and this need is even more significant in the developing countries where significant data gaps exist,” he said.

This piece was produced by SciDev.Net’s Global desk.