“Some scientists say…” “Several studies have shown…” These are weasel phrases indicative of poor science journalism, especially if the studies are not linked to, or referenced in, the story. No journalistic body is immune to this fallacy, including NPR. In a recent story about acupuncture and allergies on their blog, NPR committed a few of the classic logical fallacies of science journalism. Let’s have a look.
The story, “Acupuncture May Help With Nasal Allergies, Some Doctors Say” appeared on the NPR Health News Blog, “Shots,” on Feb. 2, 2015 [1]. The headline already commits one of the key fallacies:
- Appeal to Authority: “Some Doctors Say” is a vague weasel phrase that has no real meaning and implies that the authority of a few medically licensed individuals is enough to make a broad, sweeping recommendation for a large body of people. But smart and well-credentialed people make mistakes all the time. Two-time Nobel Prize winner Linus Pauling made the evidence-free assertion that mega-doses of Vitamin C are essential for general human health (apart from curing Vitamin C deficiencies). No reliable evidence ever backed up that claim. Smart people in positions of authority make mistakes all the time. What matters is their evidence. What is the evidence, and how good is it? Who are these doctors? What is their expertise? Are they Ph.D.s, capable of assessing published research, or are they merely M.D.s who are experts at treatment and diagnosis but not necessarily at scientific research? Are they not even M.D.s? Journalists have to ask that question and answer it in their story.
The article commits another fallacy a short time later.
“Acupuncture may help relieve symptoms and improve quality of life for people with perennial allergic rhinitis, and may help with seasonal allergies. too, according to several studies.” [1]
What studies, NPR? They never say. This weasel-phrase, “Some studies say,” without reference to the actual studies, is poor scholarship at best, and deceptive propaganda at worst. Were the studies published in reputable journals that use anonymous peer review, and were the reviewers independent from the journal’s own board of editors? Were the studies double-blind and placebo-controlled? Were they conducted by Ph.D. researchers with experience in designing medical studies? Were they published by acupuncturists, with a self-interest in propagating their own claims? Did the studies use a minimum sufficient sample size and control for dropouts during the experiment? What statistical methods did they use? Were those methods appropriate to their sample size?
So I went looking for these studies, because NPR couldn’t give me links to them in their electronic blog. I queried the SMU Central University Library system with keywords “acupuncture allergic rhinitis” and limited the search results to those from scholarly and peer-reviewed journals. Many search results appeared from disreputable journals like “The Journal of Chinese Medicine,” which is a clearinghouse for lots of nonsense alternative medicine claims with a poor history of catching bad scholarship. One recent study in a reputable journal, “The Annals of Internal Medicine,” caught my eye:
- Brinkhaus, Benno et al. “Acupuncture in patients with seasonal allergic rhinitis: a randomized trial.”Annals of internal medicine, 02/2013, Volume 158, Issue 4
The authors gave a subjective quality of life questionnaire to subjects to let them assess their symptoms. The subjects were given sham acupuncture, a standard allergy medication, real acupuncture, or combinations of these things. The authors found that there were small, positive score improvements in the subjective questionnaire over sham acupuncture or the antihistamine, but concluded that “…the improvements may not be clinically significant.” [4] Not exactly a stunning conclusion befitting inclusion in official medical guidelines.
What about this article, just a little bit older than the Brinkhaus et al. study?
- Lee, Myeong Soo. Acupuncture for allergic rhinitis: a systematic review. Annals of allergy, asthma & immunology : official publication of the American College of Allergy, Asthma, & Immunology, 04/2009, Volume 102, Issue 4
The authors did a meta-analysis of existing, published work. Before assessing the outcomes of studies, they defined selection criteria for including previous work as “best quality studies”: “… [we] retrieved data from 17 electronic databases, nonelectronic searches of conference proceedings, our own files of articles, and bibliographies of located articles. All randomized clinical trials (RCTs) of acupuncture for [allergic rhinitis (AR)] were considered for inclusion if they included placebo controls or were controlled against a comparator intervention.” In other words, studies were only considered if they used gold-standard medical testing practices – randomization, placebo-control, and comparison to the best known medical treatment.
What did the authors find? Of the 12 RCTs that met their criteria, only 7 actually crossed the threshold of methodological rigor. From studying those, they concluded that “The evidence for the effectiveness of acupuncture for the symptomatic treatment or prevention of AR is mixed. The results for seasonal AR failed to show specific effects of acupuncture. For perennial AR, results provide suggestive evidence of the effectiveness of acupuncture.”
“Suggestive evidence of effectiveness” was based on just a couple of studies . . . again, not exactly a firm foundation for medical guidelines. So the best evidence is mixed, and not exactly strongly in favor of having yourself needled instead of just taking an anti-histamine or other related class of allergic rhinitis treatment. And since acupuncture is not regulated in the same way that actual medicine is, quality of treatment varies widely and the risks of acupuncture have not been assessed by its practitioners. Patients have reported infections from needling, punctured lungs, and one patient was even abandoned in the practice with needles still inserted when the acupuncturist forgot they were there and left for the weekend. [6]
And what about their source for the story? The story appeared because of new guidelines on treating allergies. One of the members of the panel that wrote the guidelines was Dr. Sandra Lin. The NPR story describes her as, “an associate professor of otolaryngology at the Johns Hopkins School of Medicine and a guideline author.” “Otolaryngology” is “the study of ear, nose, and throat conditions…” [2]. Her own webpage [3] lists her credentials in that area, as a Medical Doctor (M.D.) and not as a researcher (Ph.D.). Becoming an M.D. is a very different process than that of becoming a Ph.D. – a Ph.D. is earned by learning to conduct independent research and developing an appreciation for the quality and limitations of measurement and data interpretation; earning an M.D. is a process of internalizing a huge body of information about human physiology and the biochemistry of drugs and treatment, but there is very little emphasis on assessing medical claims using the scientific method.
No doubt, Dr. Lin is an expert in her field; but what about in acupuncture, which is a non-medical intervention rooted in a misconception of “ancient Chinese medicine” propagated by a Frenchman in the 1800s? Dr. Lin certainly publishes her own research in reputable journals [4] and focuses on her area of expertise – all good signs for assessing the reliability of a source. But she doesn’t publish on assessments of acupuncture, suggesting she has no expertise in that area of claims. She is quoted in the NPR story, referring to the recommendation about using acupuncture to treat allergies, as saying of it that “I’m telling you there is some evidence base for it.” Again, NPR is merely appealing to authority with this quote. However, the medical literature doesn’t back her statement. She seems to have been either misled by poor-quality research touted as good research, or is ignorant of the body of research in the first place. A little homework on PubMed was all that was required to see that the evidence is weak or mixed, at best.
Journalists: you are the first, last, and often only line of defense against dangerous nonsense via the key immunization called “critical thinking.” When you do your job poorly, people with less knowledge who rely on you can do real harm to their bodies or their wallets by following your advice. Do a better job.
[2] https://en.wikipedia.org/wiki/Otorhinolaryngology
[3] http://www.hopkinsmedicine.org/profiles/results/directory/profile/0018013/sandra-lin
[4] http://www.ncbi.nlm.nih.gov/pubmed/?term=sandra+lin
[5] Brinkhaus, Benno et al. “Acupuncture in patients with seasonal allergic rhinitis: a randomized trial.”Annals of internal medicine, 02/2013, Volume 158, Issue 4
[6] http://on.aol.com/video/acupuncturist-forgets-pin-riddled-patient-and-leaves-office-517765446
