The curse of policy-based evidence
13 August 2009 |
Journalists and others must question the science used to justify policy decisions
Journalists and scientists must guard against policymakers selecting science to legitimise pre-chosen and politically-motivated policies.
In the final session of last month's World Conference of Science Journalists, Britain's chief scientific advisor, Sir John Beddington, made a revealing confession.
He told how, when interviewed for his post, he had been questioned about what he would do if a prominent science journalist asked him to comment on a government decision that ran counter to his advice. His response was that he would avoid the question and thus, by implication, any embarrassment his answer might create.
Beddington's reaction emphasises the political pressures facing even supposedly-neutral science advisors. A previous occupant of his post, for example, has admitted being pressed to withdraw an unguarded public comment that cuts in public funding for agricultural research were partly responsible for the spread of disease among UK cattle in the early 1990s.
But Beddington's admission also highlights a growing concern among scientists and others (including experienced civil servants) that praise-worthy efforts to promote evidence-based policy are sometimes undermined by politicians seeking 'policy-based evidence' — research that can be used to justify politically-motivated action.
Bad science brings bad choices
The value of evidence-based policy is clear. Robust scientific findings, such as those linking human activity to climate change, or how animals are kept to disease outbreaks, can point convincingly to the need for immediate action.
Conversely, bad science used to make social choices inevitably brings bad decisions. The eugenics movements of the 1930s demonstrated that conclusively. So too have more recent claims by some scientists that HIV does not cause AIDS — an idea embraced for example by the former president of South Africa.
One reason for improving science communication skills within the developing world is to help people who provide decision-makers with scientific information to ensure that information is robust (for example, that it has been properly peer reviewed). This is important whether the information is direct scientific advice, or comes indirectly through the media.
But science journalists and other communicators must also be alert to the reverse situation, where politicians selectively refer to, and may even sponsor, research designed to endorse pre-determined action.
The UK Parliament explicitly raised this issue in a 2006 report, warning that ministers "should certainly not seek selectively to pick pieces of evidence which support an already agreed policy, or even commission research in order to produce a justification for policy: so-called 'policy-based evidence making'" (see Scientific advice, risk and evidence based policy making[1.5MB]).
More scepticism please
Journalists have long guarded against the problems that can arise when financial interests are involved. Think, for example, of the automatic scepticism meeting industry-funded research that makes safety claims for controversial products, whether cigarettes or baby food.
We must apply the same scepticism to politicians who may be using science primarily to legitimise their actions — particularly when the quoted research has been paid for, either directly or indirectly, by a government whose actions are under fire.
And we should also be sceptical when a politician wheels out a friendly scientist — a fellow university student perhaps — to support action in a field outside the scientist's professional knowledge or expertise. Respect for the authority embedded in the title "professor" should not mean automatic deference, nor does it excuse journalists and others from questioning whether the scientist speaks with authority based on relevant experience.
Sadly, deference often seems stronger in many developing countries than in the developed world, where the need to scrutinise scientists' expertise is gaining acknowledgement.
Scientists have a responsibility to help ensure that only sound science is used to inform policy-making. One of the roles of scientific academies, for example, should be to monitor government statements and complain, both privately and publicly, when these make inappropriate references to scientific research.
Journalists share this responsibility, but they need scientists' help. Journalists can find separating appropriate from inappropriate science a challenge, given their frequent lack of technical knowledge and unfamiliarity with scientists' procedures for making such judgements.
But what journalists can do is bring cases to public attention so democratic procedures can take corrective action.
All science journalists should be on watch against the danger of policy-based evidence. Indeed, all professional science communicators should be aware of the danger, and stand ready to expose it.
See Letter to the editor.
Shahzad Ali Khan
17 August 2009
I think this problem of "policy-based evidence" quite true even in the developing countries, where there is limited use of evidence in policy-making, whatever evidence is used is usually just for the sake of justifying pre-determined policies, benefiting few groups of the society. Examples include: making case for Liver transplant unit in Pakistan by using evidence that Hep-C is on the rise and is causing liver failures; constructing Trauma centers at various urban places by using the evidence that Traffic crashes are on the rise; and establishing Burn centers by providing evidence that burn is on the rise. All these problems need preventive approaches like prevention of Hep-C, Road Traffic crashes or burns, but policy makers use the evidence for costly projects. But all these civil works or construction works are pre-planned, need huge resources and thus huge kick-back and commissions.
United States of America
17 August 2009
Thank you. As consciousness increases of the value of "evidence-based policy" it will inevitably follow that some will cloak their biases with "scientific evidence" -- we will need more and more a scientifically literate world populace that can assess both empirical evidence *and* the coherence of scientific logic...AND we will need free and effective access to data...
18 August 2009
The trouble is that todays science of macroeconomy is not sufficiently good. The present theories are based on using only part of the full picture and making up for what is missing with statistics. This unfortunate approach is not acceptable in the light of recent findings (mine) which I would I would dearly love to share with you but to which I am still waiting for a positive response. I have proof that macroeconomics is an exact science and my theory is fully comprehensive.
18 August 2009
I completely agree with this! In particular, the HPA in Britain has the main responsibility for protecting our health. When people living experience unusual health problems immediately on activation of phone masts close to their homes - the HPA says, 'There is no proof that microwave emissions from masts and phones cause health problems.' I have sent several items of peer-reviewed proof that these pulse-modulated emissions DO damage our health by interfering with the electrical communications system of the individual cells in our bodies. (SEE: Barrie Trower's 'Report on Tetra for the Police Federation'; Prof. Andrew Goldsworthy's 'Facts about Cell Phone radiation'; Prof Magda Havas 'No Wi-Fi in schools' etc etc.)
The main evidence however is in the symptoms found near phone masts, which have not been investigated by the HPA in the 7 years during which I have communicated with them symptoms found near masts. The moment the Orange mast was activated in my village residents close by began to suffer insomnia; headaches; nausea and dizziness; earache and tinnitus; sore itching bloodshot eyes; nose bleeds, raised blood pressure and strokes; ulcerated mouths and throats with extreme thirst. In two homes several rooms became unusable. One man has slept on the floor at the back of his lounge for 7 years because he can't use any of his bedrooms - his house is opposite the mast only 14 metres away! His next door neighbour cannot use her front rooms. Two animals began to vomit, would not eat and eventually developed cancer - a dog got growths on his paws which he gnawed at continually and a cat got cancer of the throat. Every time I have appealed for help to government or the Ombudsman I have been referred back to the HPA - or the government, who then also refer me to the HPA! We need a group of scientific experts with knowledge of pulse-modulated microwaves and human physiology who are completely independent of the HPA, ICNIRP, WHO, and of course the phone industry!) scientific group who will investigate the symptoms using epidemiological tests in the areas worst affected. I was once told by the then Chairman of the MTHR that epidemiological tests would be too expensive! That group had just spent £7 million pounds on laboratory tests - and another payment of the same amount was made available very shortly afterwards! In Schwarzenburg in the 1990s epidemiological tests proved that emissions from the Transmitter there was causing the same type of symptoms as the mast in in my village! Emissions were inhibiting the hormone melatonin in animals and people. (Without this hormone we cannot go to sleep, and the main symptom suffered by all victims in our village was insomnia/disturbed sleep!) I wrote 23 letters and several emails over 7 years to the HPA, until I was told not to write any more - if I did my letters would be ignored! If you are out there and having similar problems - write to your MP (get him/her to put you in touch with the Ombudsman- my MP, Andrew Selous has been a tower of strength for us and has put me in touch with the Ombudsman twice - but of course, they referred to the HPA!); the HPA and the Government. Do something about it! The more noise we make, the more likely someone will listen. Look up the people I mentioned on the internet - they are trying to help! Very best regards, Gill Lyden
C. Valencia Ramirez
United States of America
19 August 2009
Minister Chacon’s response to and critique of the political nature of the scientific community should not be taken lightly (‘Minister defends Venezuelan science’ August 6, 2009 (http://www.scidev.net/es/opinions/-ciencia-venezolana-dando-grandes-pasos-.html)). Rather it demands self-reflection, not claims of nuetral objectivity. Bifano’s statements (Venezuelan science in jeopardy June 23, 2009 (http://www.scidev.net/en/opinions/venezuelan-science-in-jeopardy.html)) reflect a fundamental problem with how some scientists operate as gatekeepers of an exclusive club. His fear mongering and arrogance are typical of the self-labeled “prominent” scientific community in Venezuela and beyond (see also Controversy over sacked Venezuelan scientist May 20, 2009 (http://www.scidev.net/en/news/controversy-over-sacked-venezuelan-scientist.html) and Government-scientist tensions escalate in Venezuela June 1, 2009 (http://www.scidev.net/en/news/government-scientist-tensions-escalate-in-venezuel.html)). Science, in Bifano’s opinion, is not for the majority of Venezuelan citizens whom he understands as “inexperienced” and with “little knowledge” yet, he opposes training through the expansion of the university system. This attitude exposes the real fear: losing control over institutions and a privileged social status afforded by previous governments and the international capitalist development of science. Simultaneously, it attempts to shroud the real objection: the presidency of Hugo Chávez and popular participation. How valuable is a scientific community that is opposed to science with social aims, widespread education and participation? The rest of the scientific community can no longer accept arguments falsely centered on a supposed “objectivity” that poorly veil self-preserving and exclusionary political strategies and practices. Why does SciDev.net that proclaims to be dedicated to exploring how “science and technology can reduce poverty, improve health and raise standards of living around the world” bother (re)publishing Venezuelan opposition political strategies rather than the abundant innovative public and private science collaboration that is working toward inclusion and public service? SciDev director Dickson calls for increased deference to policy-based evidence used by governments in the “developed” world (The curse of policy-based evidence Aug 13, 2009). This same critical eye must be turned on the scientific establishment – often an economic, social, and political elite class – in “developing” countries.