By: Yao-Hua Law
Send to a friend
The details you provide on this page will not be used to send unsolicited email, and will not be sold to a 3rd party. See privacy policy.
[SEOUL] Journalists have much room to improve on in healthcare reporting.
To help journalists address health issues, the 9th World Conference of Science Journalists 2015 in Seoul held a session on how to read and report on medical and health studies.
Ivan Oransky, who teaches medical journalism at the New York University’s Arthur L. Carter Journalism Institute, and Hans van Maanen, who teaches journalism and statistics at several Dutch universities, gave out these pieces of advice.
1. Ask dumb questions
Medical studies are filled with jargon. To unravel this jargon and reveal the important details, one has to ask “dumb” questions. Maanen says he prepares interviewees by telling them that he's not well-versed in their fields and that he will ask dumb questions.
2. Ask smart questions
Was the study peer reviewed and where was it published? There are journals now that skip peer review. Alternatively, the study might also have been announced in a press meeting or science conference without being examined by peer review.
How many samples were tested, and what was the significance? Size does matter in experiments and surveys. Everything else being equal, the larger the sample size of the study, the more robust its results. A study of only a dozen mice can draw a much weaker conclusion than one with three dozen.
We often miss another crucial effect of sample size in a study — its power or effect. A study with more samples also has higher power which makes it more likely to detect a significant effect if there really is one.
3. Ask awkward questions
It's statistically significant, but is it clinically significant? Theoretically, with large enough sample sizes, one can squeeze a “p < 0.05” out of any comparison. But would this statistical difference translate into a noticeable effect in an individual or an entire population?
How effective is the treatment and what are the side effects? Give numbers. Almost every drug has side effects but the complications can be difficult to pinpoint in the study. Also, never settle for a qualitative answer like “people improved”; always get a quantitative answer.
4. Check statistics
Journalists cannot be expected to dissect studies and hunt for defects in the design of the experiment. Clinical trials are particularly challenging. Instead, keep a biostatistician friend in your back pocket, says Oransky. Find biostatisticians in your local universities and build rapport with them.
5. Dig deeper
Ask the scientists to point you to people who might disagree with them. If they claim that nobody would disagree, then it is a sign that you need to dig deeper and get alternative views.
Whose interest is vested in the findings? Consider how the potential conflict of interest among the parties involved — including alternative sources — may affect what they say. Obvious conflicts of interest are reporting results that relate to the funding sources. Oransky suggests journalists check the industry funding of doctors and researchers at Dollars for Docs and PharmedOut.
This article has been produced by SciDev.Net's South-East Asia & Pacific desk.
More on Journalism
