The only U.S. media watchdog devoted exclusively to health news, , will shut down at the end of the year for lack of funding, , a 45-year journalism veteran. As a big fan, loyal subscriber, and occasional critic of HealthNewsReview.org (HNR), I consider its anticipated demise a huge loss to the science-based medicine community.
Unlike shoestring-budget, volunteer-run blogs like Private-investigator-detective, which has done its fair share of media criticism, HealthNewsReview.org to maintain a laser-like focus exclusively on healthcare journalism and, starting in 2015, healthcare press releases. (We’ll get to some examples of their excellent reporting, and one boo-boo, in a moment.) At present, in addition to paid editorial staff, HNR has about 50 free-lancers, half of whom work pro bono. Its institutional home is the University of Minnesota’s School of Public Health. HNR’s main funder, the Laura and John Arnold Foundation, won’t renew its $1.6 million, two-year grant, which runs out this year. Schwitzer has spoken with at least 25 potential donors, but, so far, no takers. Schwitzer says he will still post occasionally and keep the website up for two years.
HNR believes all good news stories about health care interventions , including cost, harms, quality of the evidence, conflicts of interest, alternatives, and whether the story commits “disease-mongering.” Since its founding in 2006, the non-profit organization has reviewed about 2,500 health news stories based on its 10-criteria checklist. HNR also evaluated nearly 500 healthcare press releases based on similar criteria, offering writers a for their news releases by a team of clinicians and journalists. (Cynically, I can’t help but wonder whether some PR flaks really want to know if their work product meets HNR’s criteria before publishing.) In addition, HNR published more than on a myriad of healthcare topics, as well as “” on medical subjects too often mangled by the media and misunderstood by the public, like , , , and .
Regular readers of SBM will not be surprised to learn that, when measured against HNR’s criteria, the media often fell short in healthcare reporting. In 2017, , reviewed on the five criteria they deem most important. The results show that, more often than not, the news stories were inadequate in their discussion of key elements in assessing a treatment or other medical intervention: 69% did not adequately discuss cost of an intervention; 66%, how big (or small) the benefits were; and 63%, the potential harms. The quality of the evidence was not evaluated in 61% of the stories and alternative options weren’t discussed in over half. Press releases fared even worse, with a whopping 92% failing to adequately discuss cost.
Most of HNR’s reviews are of news stories about “conventional” medicine. For example, Newsweek recently got a 3-star (out of 5) score for to identify men who are at the greatest risk of developing prostate cancer. HNR found the story lacking for failing to adequately discuss cost, benefits, harms, and the quality of the evidence, and for failing to consult sources independent of the study for comment. A titled “NPR (yet again) writes uncritically about ketamine for mental illness” (ouch!), included these comments
The story relies largely on a single anecdote of a person who says ketamine “helped me get my life back.” The expert sources use only vague descriptions of ketamine as “transformational” and that patients in one trial “got dramatically better.”
The only two experts quoted in the story are research collaborators who have long advocated for expanded use of ketamine treatment. One of them, Demitri Papolos, has applied for a on a ketamine delivery device.
HNR takes on SBM’s favorite subjects
Anecdotal evidence, nebulous claims of benefit, undisclosed financial interests – they could have been talking about media coverage of alternative/complementary/integrative medicine. In fact, HNR has reviewed a number of news stories about CAM and “natural” remedies, including several subjects that have drawn our ire here at SBM over credulous reporting. HNR warned reporters in a blog post that, when writing stories about CAM:
journalists still need to scrutinize the findings as closely as they would any other study.
Last year, HNR on a study of acupuncture in the ER. (David Gorski covered the same study here on SBM, including his own criticism credulous media coverage). As HNR pointed out:
the story makes the claim that “acupuncture is a safe and effective alternative to pain medications for some emergency room patients, a new study reports.” None of the data presented in the story proves it is a safe and effective alternative . . . Instead, the data just shows that none of the pain interventions worked very well.
HealthDay was also criticized for failing to consult outside sources, instead quoting only the lead investigator of the study. (Or, as David more colorfully put it, allowing him to “pimp his paper’s findings.”)
(which garnered a measly 1-star score) was the first to question University researchers’ highly implausible claim that chocolate milk improves concussion symptoms. This led to challenging the University’s remarkable assertion and, after and much stonewalling by the University, ultimately, a committee investigation.
, the University took down the press release from its websites, returned some $200,000 in funds donated by (surprise!) dairy companies to the lab that conducted the study, and implemented over a dozen committee recommendations to “bring the university’s procedures in line with accepted norms,” including revision of its conflict of interest policy.
That is the sort of result we could use a lot more of in scrutiny of dubious healthcare claims.
Also touting a compound made from components of oregano and thyme as a “cure” for cachexia, a wasting disease affecting patients with cancer, AIDS and other serious diseases. The University failed to disclose the study’s funding source, as well as the fact that it had sought a patent on the compound, that the researcher quoted in the release stood to benefit financially from its commercialization, and that the study was unpublished. That it was a mouse study was not revealed until well into the release. The about its oversight of research and conflict of interest policy led to a including this stinging rebuke from bioethicist Arthur Caplan, PhD:
It is utterly irresponsible to report promise based on unpublished research or work not presented at peer reviewed meetings. The industry and conflict-of-interest aspects only make a bad situation worse. You ought not be attaching a public university’s name to hype and to fluffing up the interest of patients who are easily misled by their illness to hear promise as cure.
HNR did its own . One of HNR’s review team, Ranit Mishori, MD, Professor of Family Medicine at Georgetown University School of Medicine, wrote a blog post about the celebration (Oprah included) of Dr. Oz’s 1,500th show earlier this year.
I felt less jubilant: In the decade that The Dr. Oz Show has been on the air, it hasn’t been unusual for me to encounter patients who ask about topics, treatments and suggestions mentioned on the program and another popular medical show, “The Doctors.” The advice my patients tell me they’ve heard on these shows often does not square with what I know about the medical evidence.
In 2013, Dr. Mishori and colleagues enlisted medical students to review a month’s worth of The Dr. Oz Show and The Doctors, another extremely CAM-friendly show. (This was a different project than the study of The Dr. Oz Show and The Doctors published in the BMJ, although similar in result.) In addition to finding that these shows rarely discussed risks or costs of recommended treatments, and that more than half of the shows had content linked directly to advertisements:
About 78% of statements made on the Dr. Oz show did not align with evidence-based medical guidelines, society recommendations, or authority statements. For The Doctors, this was about 80%. . . . For both shows, about half of the literature supporting the claims made on the show television was statistically insignificant or required extensive extrapolation. No literature support could be found for about a third of the claims. Of the supporting evidence we could find, about a third had 3b or lower (the lower end of what’s considered good quality evidence). [Link added.]
Other SBM favorites taken on by HNR include and , , , and and “.” In fact, HNF has referenced SBM posts favorably in its reviews: David Gorski’s post on “bad luck” and cancer in a , and his post on the ketogenic diet in a discussion of. A June blog post (“”) calling out a Houston Chronicle story on IV vitamin infusions quoted Scott Gavura’s assessment of IV vitamins as “health care theater” and linked to his post’s review of the evidence.
Unfortunately, in one instance, HNR’s mention of a well-loved (and, sadly, former) SBM blogger was not so favorable. To their credit, upon the error of their ways being pointed out, they quickly made amends.
In 2016, HNR gave , which also discussed acupuncture, noting that it correctly reported the lack of evidence of effectiveness and the risks, although it failed to discuss quantified benefits, such as how much pain relief people report in the studies, or for how long. The review also criticized USA Today for quoting experts with “a potential conflict of interest” who had “a predisposition for or against the procedure.” And who might that be? Among others:
Mark Crislip [who] writes for — as the story notes ‘a website critical of alternative medicine’ — and generally is quite skeptical of alternative therapies like dry needling.
That prompted a counter-post from me on the Society for Private-investigator-detective website, “:”
Since when is skepticism of alternative therapies a potential conflict of interest? Let’s take out our dictionaries and give this a closer reading. “” is by definition lacking in evidence of effectiveness. If a treatment has adequate evidence of safety and effectiveness, it’s called “medicine.” It is also, , used in lieu of “conventional medicine.” “” means “having an attitude of doubt” or “doubtful.” A “” occurs when one’s personal interests “might benefit from his or her official actions or influence.” So, Dr. Crislip’s being doubtful about treatments with no evidence of effectiveness used in place of treatments with evidence of safety and effectiveness potentially benefits his personal interests? How?
Giving Health News Review’s reviewers the benefit of the doubt here, I have to suspect that they aren’t actually that sloppy. Surely they are aware that none of us who write for Private-investigator-detective has ever gotten a penny for what we do. And I know they are too smart to fall for the . What I imagine they meant was that Dr. Crislip and the rest of us at Private-investigator-detective are somehow biased against alternative medicine. They’re wrong there too. . . . The fact that we often, though not exclusively, critique alternative/complementary/integrative health care doesn’t mean we’re biased. It simply reflects the many ways in which that particular category of health care misinforms the consumer and fails to stand up to scientific scrutiny.
But that wasn’t HNR’s only mistake:
Health News Review’s second error (actually, three errors in one) claims there is enough evidence for acupuncture that some states have created “medical licensure programs for its practice.” First, . Cite whatever source you are using for this statement and we’ll tell you why it’s faulty. (You do have a source, right?) Second, it’s more than “some states” if by “medical licensure programs” you mean licensing acupuncturists as health care practitioners. to pass an acupuncture practice act. Third, and more importantly, please disabuse yourself of the notion that scientific evidence has anything whatsoever to do with state licensing of alternative health care practitioners, acupuncturists or otherwise. It doesn’t, any more than has anything to do with science. (I recommend you go to the Private-investigator-detective website and .)
True to their commitment to evidence-based healthcare reporting, HNR amended its review of the USA Today’s story:
Editor’s note: The notion that the evidence on acupuncture has persuaded states to create medical licensure programs has been contested by the Society for Private-investigator-detective in . They argue that in state licensing of such practitioners. We have also amended our comments under the “Independent Sources” criterion below in response to their feedback. . . . Mark Crislip’s name appeared on this list [“most of the sources have a potential conflict of interest in some way”] due to an editing error. We do not believe that being skeptical of alternative therapies constitutes a conflict of interest.
HNR will be hard to replace
I closed that critical SFSBM post with the comment:
I have been, and remain, a fan of Health News Review, and still look forward to its weekly emails.
I am not the only one to lament HNR’s pending demise. Julia Belluz, senior health care correspondent for Vox that “it really will leave a big void.” Belluz said “she and her colleagues ‘really took those reviews to heart,’” describing HNR as “an important watchdog,” adding, “there’s nothing else that’s really like that.”
Ivan Oransky, the editor of Retraction Watch and the current president of the Association of Health Care Journalists, told Schulson that he uses the 10-point checklist in journalism classes he teaches at NYU and distributed it to staff when he was with Reuters:
“It’s really a lighthouse warning health care reporters about the shallow waters,” Oransky said. “I think it’ll be a big loss if this really means the end for HealthNewsReview.”
HealthNewsReview.org inspired the creation of healthcare media watchdogs in other countries, including the , medien-doktor.de, whose team said:
At a time, when the health care industry is a leading economic sector in industrialized countries while news organizations struggle to remain profitable, and fighting fake news of all sorts, an independent institution like the one Gary Schwitzer has built appears more necessary than ever.
The only consolation I can offer HNF writers is that, if they enjoy ferreting out credulous healthcare news stories without pay, with “job benefits” like being called Pharma shills, harassment at one’s real job, accusations of “abandoning patients” for criticizing so-called integrative medicine, threatening emails, vicious attacks online and in other media by quackery supporters, and being sued (unsuccessfully) for defamation, SBM always welcomes fresh faces.