As crazy as our headline sounds, it is merely taking CNN’s headline, below, and spreading it out over a year.
CNN quoted one of the researchers:
The main message for people to take away from this is that coffee can be enjoyed as part of a healthy lifestyle, and coffee lovers can be reassured by this result in terms of blood vessel stiffness outcomes. (emphasis added)
Note that they didn’t report fewer heart attacks or fewer cases of heart disease. “In terms of blood vessel stiffness” was the endpoint. A surrogate endpoint, which, as our helpful primer points out, often doesn’t tell the whole story.
Seeking independent perspectives to analyze studies helps get at the bigger picture.
The Huffington Post reported:
“This is rather silly and irresponsible reporting in my view,” Dr. David Katz, a director at the Yale-Griffin Prevention Research center, told HuffPost. …
“This study does not prove that 25 cups of coffee a day are reliably safe,” Katz said. “It simply shows lack of one very specific harm in a population where some few self-selected to drink that much, quite possibly suggesting an extreme tolerance the rest of us do not enjoy.”
Fox News was among those reporting a quote from an official of the British Heart Foundation (which helped fund the study), claiming that the study “will hopefully put some of the media reports in perspective, as it rules out one of the potential detrimental effects of coffee on our arteries.” Earlier in the Fox story, one of the researchers admitted, “We can’t prove a causal link in this study.” So, you see, it didn’t rule out anything because it didn’t prove anything – except that the coffee craziness continues.
Meantime, blood vessel stiffness was also at the center of hyperbolic heart health claims at the New York Times this week. Another all-too-brief Times story of 200 words and some change.
At the end, the story acknowledged that “The study received support from the U.S. Highbush Blueberry Council.” But it still allowed the lead researcher, whose work was supported by this blueberry industry money, to project the benefits that would come “If everyone ate more fruits and berries…” There was no independent expert perspective in the story, only that promotional quote from a conflicted researcher.
But let’s take a closer look at the evidence. The story reported:
The authors estimate that eating a cup of blueberries a day reduced the risk of any cardiovascular event by 13 percent and the risk of coronary heart disease by 11.4 to 14.5 percent.
Really? How did they estimate reduced risk of cardiovascular events or coronary heart disease? The story continues:
But compared with the half-cup consumers and the placebo group, the full-cup group had reduced arterial stiffness (emphasis added) and increased levels of HDL (the “good” cholesterol). Those eating lots of blueberries also had increased levels of anthocyanin, a type of antioxidant found in plant pigments, and increased levels of cyclic guanosine monophosphate, a substance that relaxes smooth muscle cells and improves blood flow.
There’s that arterial stiffness again, along with other surrogate endpoints such as levels of HDL, anthocyanin and cyclic guanosine monophosphate. Should we really devote “all the news that’s fit to print” attention to these surrogates, suggesting that readers become fixated on more and deeper concerns about what silent killers may be lurking inside them? Or should we educate them about the outcomes that really matter when they hear about research? These surrogates are legitimate research pursuits, but news stories that elevate them to the level of heightened consumer concern – without explaining the limitations of such surrogates – are useless.
Can’t you just imagine the subway conversations as people read the Times? “Yo, how’s your arterial stiffness today?”
As we’ve noted before regarding unhelpful briefs from the Times, smart reader comments demonstrate that this kind of news is turning off a lot of people. Some examples of comments left by readers on the Times website:
But what else were the study participants eating? Judging from the fact that they were overweight or obese, and at high risk for cardiovascular disease, I would guess it’s a Standard American Diet (SAD), high in processed and ultraprocessed foods, as well as animal products (meat, dairy, eggs).
So the people who grow and sell blueberries paid for a study that concluded blueberries are good for us. Yeah, Yeah.
Perhaps daily stories like this produce more harm than good when recommending impossible to sustain daily eating habits.
When Jeopardy! host Alex Trebek was diagnosed with stage 4 pancreatic cancer three months ago, fans were shocked. But now the beloved Trebek has good news to share: He’s in “near remission,” according to doctors.
“It’s kind of mind-boggling,” the 78-year-old says, in PEOPLE’s new cover story. Although the cancer has a 9 percent survival rate, Trebek has been responding very well to chemotherapy. “The doctors said they hadn’t seen this kind of positive result in their memory…some of the tumors have already shrunk by more than 50 percent.”
What’s really mind-boggling is how news organizations across the country ran with this “good news” story without holding it up to scrutiny.
Boston’s NPR station, WBUR, reported that Trebek’s story reflects “significant progress” in treating pancreatic cancer, while the Boston Herald quoted a doctor who said it “provides hope to others” who have pancreatic cancer.
Few news organizations asked what “near remission” might actually mean, or whether tumor shrinkage significantly improves a patient’s chances of survival. Largely missing in the coverage were cautions that even complete remission does not equal cure, and tumor shrinkage does not always correlate with longer life.
“In my opinion it is far too early to get all gaga optimistic,” cautioned John James, a retired toxicologist and founder of Patient Safety America, who said tumor shrinkage is “a long way from a cure in pancreatic cancer.”
Celebrities can use their powerful platforms to educate and inspire, and Trebek has drawn praise for talking openly about the side effects of chemotherapy and his “surges of deep, deep sadness.”
A few news organizations provided much-needed context by interviewing oncologists. Fox 10 in Phoenix reported that a full remission is “not very likely” for Trebek (despite its news anchors’ description of Trebek’s chemo response as “truly incredible” and “amazing”). Prevention magazine quoted a specialist who said “advanced stage pancreatic cancer is basically incurable.”
In a Twitter thread, surgical oncologist David Gorski, MD, managing editor of the website Science-Based Medicine, called Trebek’s results “a good initial response, nothing more.”
Gorski said he hopes Trebek is among the “rare” pancreatic cancer patients who survive for a long time.
Painting an ‘upbeat’ picture
It’s a mistake to view emotional anecdotes as evidence of a benefit.
“If an individual improved after receiving a treatment, it does not necessarily mean that the treatment caused the improvement, or that other people receiving the same treatment will also improve,” said Mary O’Keeffe, a postdoctoral researcher at the University of Sydney who is part of Wiser Healthcare research collaboration, via email.
That may be especially true when it comes to celebrities. Like anyone, celebrities can find it hard to face their own mortality, said Barron Lerner, MD, an internist and faculty member at the NYU School of Medicine who’s written extensively about celebrities and illness. But in their quest to please the public, they also can “tend to paint a very upbeat picture in general and use language like ‘fighting’ and ‘beating.'”
With Trebek’s account, Lerner said: “I think you have to be careful. He’s not giving you scientific data. He giving you a story of how it’s going with him at the moment.”
Journalist Sue Rochman, who covers cancer and has served on our review team, said via email:
There is nothing wrong with optimism and hope following a cancer diagnosis. And who doesn’t want to be optimistic and hope that Alex Trebek’s pancreatic cancer will respond to treatment? Unfortunately, how he is describing his response — or at least how that response was relayed by People magazine — doesn’t add up to journalists who cover cancer. Having tumors shrink in response to treatment makes sense; calling this a “near remission” does not.
For people with cancer, rah-rah news coverage might create unrealistic treatment expectations.
Andrae Vandross, MD, a hematology-oncology specialist at UCLA Medical Center who has been a member of the HealthNewsReview.org review team, said via email:
I always worry about this type of reporting leading to patients thinking that the famous are getting “better” treatment. If my patient is not having the same response as Mr. Trebek, she may think that I’m not offering or am not aware of the “better” therapy.
In reality it may be that Mr Trebek has a tumor that happens to have a favorable response to standard treatment based on its inherent biology.
Without more detailed information about Trebek’s treatment, Vandross said, “we can’t know if the standard therapies are just working really well for Mr. Trebek, or if he is getting an ‘off label’ therapy,” a treatment that hasn’t been FDA-approved to treat pancreatic cancer. He said news stories could have provided “the basic landscape of what the typical responses to chemotherapy for advanced pancreatic cancer are.”
O’Keeffe also worries about people who are “hopping on” to Trebek’s positive story to praise new treatments for pancreatic cancer.
They seem to be using a testimonial as evidence of therapy effectiveness for a condition which has a poor prognosis. This is misleading. The journalists may improve their reporting by admitting that this is a story from one person and this does not provide evidence that late-stage cancer is curable.
Remembering Farrah Fawcett
Some news organizations have been observing the 10-year anniversary of the death of Farrah Fawcett, who shared publicly what she called her “private war” against aggressive cancer. She allowed the filming of her two years of excruciating treatments — chemotherapy, radiation, drugs, surgeries, and unproven “alternative” therapies.
There’s a parallel between Fawcett’s determined statements and Trebek’s on-air pledge to “beat the low survival rate statistics for this disease.”
Some believe celebrities’ use of such battlefield language may encourage other patients to engage in painful and prolonged treatment regimes that distract them from making the most of the time they have left.
It’s also risky to portray remission as a victory. In 2007 Fawcett’s medical team declared her “cancer free,” only to see tumors return in less than four months. In 2014 AARP’s website included soul singer Sharon Jones in a listicle of “12 Celebrities Who Beat Cancer.” Two years later, Jones died of pancreatic cancer.
Discussing the end of life
“I wonder why celebrities put out statements about their health, and who these statements serve,” said Sunita Puri, MD, medical director of palliative care at Keck Hospital and Norris Comprehensive Cancer Center of USC. She is the author of That Good Night: Life and Medicine in the Eleventh Hour.
The myth that stopping treatment means giving up blocks rational discussion of what the end of life should look like, Puri said. She’d like journalists to ask Trebek — and other celebrities with fatal illnesses — “about what quality of life would mean the most to him, and what his lines in the sand would be” with continuing treatment.
“If he said, ‘I am having discussions with my family about my wishes when my disease worsens,’ that could potentially normalize these sorts of conversations and inspire others to have them as well,” she said. “I have not seen a celebrity doing that to date.”
In CNN’s reality-based series, “Chasing Life,” correspondent Dr. Sanjay Gupta travels to far-flung reaches of the planet to explore health care practices rooted in cultural traditions.
The show — which is available on the streaming platform CNNgo — was produced by the same division that created Anthony Bourdain Parts Unknown, and it was clearly designed to capitalize on that profitable model of personality-driven TV.
While Bourdain promoted the idea that Americans can learn from culinary traditions of other countries, Gupta’s premise is that we can learn from their health care lore.
At the start of each episode, Gupta tells viewers: “I want the same thing you do: the secret to living longer, healthier and happier.”
As entertainment, it works. Following Bourdain’s formula, the series is filled with charming people, thrilling scenery, and eye-popping cultural rituals.
We see Gupta immersing himself in a scorching Japanese bath, getting a foot-pressure massage in India, milking goats in Italy, taking a high-altitude run in Bolivia, and on and on.
Gupta — once named among People magazine’s “Sexiest Men Alive” — also gets plenty of chances to take his shirt off.
But does this travel documentary do a good job of conveying health care information?
Where’s the evidence?
Despite the resources that went into this programming, viewers learn little about what evidence exists that the practices depicted are effective or safe.
In Norway we see Gupta slug down cod liver oil, which he calls that country’s “secret weapon” against seasonal affective disorder — depression that hits in the months when it’s totally dark. No mention of whether there’s evidence this stuff actually improves symptoms. (A 2015 review concluded there’s no high-quality evidence that fish oils help with depression.)
In Turkey, people with mental disorders such as Down syndrome and autism are shown engaging in an Islamic whirling dervish meditation dance. Gupta says we don’t know why this ritual “works” for these people. But does it really? What’s the evidence? We aren’t told.
After watching some of the show’s clips, Canadian cardiologist Christopher Labos, MD, who has a degree in epidemiology and biostatistics and was a member of HeathNewsReview.org’s review team, provided this reaction:
A lot of these segments are very interesting, but they aren’t science-based. … I would have liked to see him actually explore this stuff, hold up the stuff that does work and disprove that stuff that doesn’t. … The truth is, there is no secret to “living longer, healthier and happier.” We already know what we have to do: quit smoking, eat more fruits and vegetables, eat less junk food, exercise more, maintain a healthy body weight. … But re-inforcing these fundamentals probably wouldn’t make for a very entertaining TV show.
Potential harms go unreported
What producers apparently thought did make for good TV was playing up purported benefits without telling viewers about potential harms.
Savoring goat cheese in Italy, Gupta asserts that “eating foods that are more alive — like raw cheese — is not only going to help your digestion but also help the rest of your health.” What he doesn’t say is that the CDC calls unpasteurized milk products “one of the riskiest” things you can consume, due to the potential for foodborne illness.
In the same episode Gupta says studies show wine is “good for our health,” and drinking one or two glasses a day is “a pretty enjoyable way to fight disease.”
Labos called this “really glossing over the evidence.”
The idea that red wine is good for the heart is really not supported by good evidence, and in fact, the sugar found in alcoholic beverages will almost certainly make you gain weight, alcohol itself increases your risk of cancer, and the idea that there are healthy antioxidants in red wine has been studied in studies like InCHIANTI, which turned out negative. Drinking alcohol is probably a net negative for your health, but I didn’t get that sense of nuance from the clips.
In the episode on Bolivia, the show suggests that parasitic infections contribute to very low rates of heart disease in the remote Tsimane tribe. CNN tweeted:
Labos said he had “a few issues” with promoting idea that parasitic infections might actually be good for us:
1. Most of the low rates of heart disease claimed for the tribe can likely be explained by their healthy diet and high levels of physical activity. In fact that would probably explain lots of it. If they started eating junk food and sitting around all day, their rates of heart disease would almost certainly go up. I don’t think the parasites would do much good then.
2. There’s a popular logical inconsistency in this segment. The tribe actually has a lower life expectancy than the U.S. So they die earlier. Given that the most important risk factor for heart disease is age, an older population (like the U.S. population) will have higher rates of heart disease simply by definition. The basic point is that if you die young, you won’t live long enough to have a heart attack.
3. As for parasites being good for your heart, I think Chagas’ disease [a common parasitic infection of the heart] proves that that is not true. Parasites can actually be very dangerous for your heart.
Lack of skepticism for yoga, ayurveda
At times the lack of skepticism is glaring. In India Gupta nods agreeably as Prime Minister Narendra Modi offers a defense of his government’s promotion of the Hindu practices of yoga and ayurveda, a health system based on herbs and minerals.
Gupta doesn’t appear to raise an eyebrow when Modi asserts that yoga can bring health care spending to zero and rejects the need for objective evidence of a benefit. Viewers aren’t told how Modi’s embrace of yoga and ayurveda is intertwined with the oppression of India’s minority Muslim population and that a key Modi political backer runs a massive ayurveda business.
Buried in a story that ran online, Gupta reported there’s “hardly any” scientific evidence to back up ayurveda, and “much of the impact may just come from the thought that goes into creating your meals” along with fewer calories being consumed and a lack of processed foods and added sugars.
Such nuances barely come across on the TV show.
Appeals to tradition and nature
Appealing to tradition — the reasoning that something must work because it’s been done for generations — pops up a lot. With the foot-pressure massage: “They’ve been around so long, there’s got to be something to it.” With the traditional Japanese diet: “There’s are reason why foods like bitter melon have been preserved and passed down for generations in Japan. They intuitively knew these foods were good for you.”
Likewise an appeal to nature — that something must be healthy because it is natural — is a common refrain.
Timothy Caulfield, a professor of health law and policy at the University of Alberta, produced a six-part medical myth-busting documentary series on Netflix, “A User’s Guide to Cheating Death.”
Caulfield said the show skirts the complexity of evidence.
While the show doesn’t come out with strong endorsements of the many alternative practices they explore, there is this vague support for the embrace of science-free perspectives. We see Gupta asking if “organic” goats’ milk is “chemical free” (What, Dr. Gupta, do you think milk is made of?) and supporting “hope rooted in ancient tradition.”
Caulfield said pop culture is already inundated with what he calls “wellness woo.”
In this era of misinformation, Gupta should have used this space to embrace and encourage critical thinking. Instead we got a show the kind of does the opposite, inviting the viewer to ignore the science and embrace the experience. That kind of thinking — which props up the value of testimonials and anecdote – is exactly what has allowed anti-vaxx sentiments to thrive, bogus diet fads to take root and the science-free concept of “wellness” to grow into a multi-trillion dollar industry.
‘Facts First’ for health care?
You’ve probably seen CNN’s “Facts First” ad campaign, which uses the metaphor of an apple to underscore the network’s commitment to bringing viewers fact-filled political reporting. “There is no alternative to a fact,” its copy says.
Caulfield wonders why CNN doesn’t apply the same ethos to health care as it does to political news.
Why not explore what the science actually says and present a full and frank analysis of that evidence — explain how and why the evidence gets twisted and what viewers can do to navigate through the noise? Instead, the show seems to be asking viewers to do the opposite. Ignore the science. Follow your gut and intuitions. In the political space CNN has argued this approach is highly problematic. But it is OK when it comes to science and health?
The hitch is, gathering facts and exploring evidence requires journalists who are skilled in covering health care. Unfortunately, CNN now has far fewer of them after announcing a “restructure” last week that “ultimately impacts 6-7 employees” in its health unit. Gupta still has his job.
Why CNN didn’t skip the nutty globe hopping and save a job or two is anyone’s guess.
The more you screen, the more you find. And the more you find, the more you find that is not really something you needed to — or wanted to — find. That’s true with just about any health screening test.
Remember: screening means looking for problems in otherwise healthy people. So in this case it is looking for hernias in people who don’t have any symptoms or signs of problems now. If people did have a problem, then you’d call it a diagnostic test – looking for the right diagnosis for the cause of the symptoms.
But hospitals and surgeons are calling this screening. For one reaction to this trend in marketing, I turned to Kenny Lin, MD, MPH, deputy editor of American Family Physician and a professor of family medicine at Georgetown University School of Medicine. His reaction was blunt:
Generally, hernias don’t need to be treated unless they are causing symptoms (e.g., pain, vomiting).
Which is why screening the symptom-free general population for hernias seems really strange. I asked one of our former reviewers, Doug Campos-Outcalt, MD, for his opinion:
You are correct. It is very hard to improve on the asymptomatic healthy patient. There are many more things that can go wrong with screening and treating than can go right.
Underneath the ad pictured above was this noteworthy line:
The da Vinci® Xi Surgical System will be on-site for hands-on demonstration.
That helps to put the free screening campaign in a new light. It’s one way to find more patients to help pay off that expensive robotic surgery system. Dr. Kenny Lin again:
This is a naked attempt to drum up business for their surgeons and the hospital’s operating rooms. It’s even worse that they are advertising their da Vinci surgical system, when robotic-assisted surgery increases patient costs but has not been shown to improve surgical outcomes.
Outcomes after robotic-assisted inguinal hernia repair were similar to outcomes after laparoscopic or open repair. Longer operative duration during robotic repair may contribute to higher rates of skin and soft tissue infection.
Just three months ago, a Cleveland Clinic surgeon told the Minimally Invasive Surgery Symposium that research on robotic inguinal hernia surgery is difficult to find. Her comments were reported by MDedge.com.
“There’s not a lot out there,” Ajita Prabhu, MD, said. “If I stood here and went through every one of those studies detail by detail, I think I could do it in 15 minutes. …We’re finding additional evidence that it adds OR [operating room] time, and it’s expensive.” … Moving forward, she said, robotic inguinal hernia surgery should be tested so “we can make sure it’s actually better, not just cool. We need to be able to justify our utilization.”
The ad pictured above was for a Chicago-area hospital. But such marketing is popping up all over the U.S. It’s happening not just in paid-for ads and marketing, but also in supposedly independently vetted journalism. Here are just a few of many examples you can find online.
In Amarillo, Texas, a TV news anchor presented a fawning promotion for a medical center’s free hernia screening which was also tied directly to a promotion of its robotic surgery system. The anchorwoman could barely contain her admiration for the effort, gushing: “It is free. How nice is that? … You’re making it easy and patient-friendly.”
If left untreated, hernias can get bigger over time, become more visible and can cause serious problems, including blocking the intestine, pinched blood vessels and the hernia can break through the skin.
There is almost robotic repetition in the themes you hear consistently in these robotic surgery promotions. The emphasis is: It’s free. If you must have surgery, the outcomes from robotic surgery are fantastic. And if you don’t have surgery for the hernias we might find, you’re asking for big trouble.
All of these promotions occurred in recent months. Hmmm. It sure feels like a promotional push from the “mother ship” — Intuitive Surgical Systems, makers of the da Vinci robotic surgery system. Indeed, hernia repair via robotic surgery is one of the key promotions now on Intuitive’s website.
Harold DeMonaco is a visiting scientist of behavioral and policy sciences at the MIT Sloan School of Management. For years he was director of the Innovation Support Center at the Massachusetts General Hospital. He’s also reviewed countless media messages about health care for us. He emailed me his opinion of this hospital marketing trend:
I agree that this type of “screening” seems more like trolling for surgical cases. I suspect that this campaign is designed to increase hospital utilization of the very expensive robot platform (presumed advantage to the surgeon and hospital) and sale of disposables needed (a clear advantage to Intuitive Surgery).
But is there really an advantage for patients? Maybe all of that television happy talk time could be spent looking into that issue for the benefit of viewers.
Excerpts (with my comments added in bold italics):
“a smartphone app — one that works on iPhone and Android — that can identify the presence of fluid in a child’s ear.” But the presence of fluid does not always mean the presence of infection.
“physicians — and parents — can use it with relative ease” Emphasis on relative, perhaps.
“The study did test a small set of patients although plans for larger studies are underway.” Hmm. And why shouldn’t the news coverage wait until those larger studies are done?
STAT interviewed the researchers about their plans “for getting the app into millions of hands.” And I’ll bet that they’re very happy to get STAT’s help in furthering those plans.
A researcher explained: “We approached parents just before their kids were about to have surgery to have ear tubes put in. …We had 25 parents do the test, and 24 of the cases, the parents had the same result as the physicians who used the app.” And what is the statistical power of this small a sample?
The researcher also said: “It’s not currently available in app stores, but we’re hoping to have it in there in a year’s time. We’re also hoping to get FDA approval by the end of year. We created a company — Edus Health.” The cart is clearly in front of the horse. First comes the research, then maybe FDA approval – not a fait accompli, then we can talk about the company, and then we can talk about the app being in stores.
Alan Schroeder, MD, the associate chief for research in the division of pediatric hospital medicine at Lucile Packard Children’s Hospital Stanford, reacted to my comments via email.
“You’ve hit the nail on the head – the headline was “A smartphone app could help diagnose ear infections more accurately — and at home” but the patients studied did not have ear infections. They had fluid behind their ears. It does not seem that the app will be able to differentiate acute otitis media (an acute bacterial infection of the middle ear) from otitis media with effusion (fluid in the middle ear without signs or symptoms of inflammation – also called serious otitis media). Many patients with fluid behind their ears will not have otitis media with effusion. This brings up the potential for a lot of false alarms and unnecessary office visits/antibiotics.”
STAT wasn’t alone in reporting on the journal article in question. The Associated Press, Gizmodo, WebMD & HealthDay were among many others who reported on it. WebMD, picking up a HealthDay story, went beyond what the study actually showed and inaccurately reported:
Can a smartphone app spot an ear infection? It did so with high accuracy in new research.
It found fluid, not necessarily infection.
But NPR’s story, for example, took a markedly different tone. It placed caveats high in the story and had independent expert quotes that provided important context.
In the first sentence, NPR noted: “might help parents detect fluid buildup in a child’s ear — one symptom of an ear infection.” (emphasis added) The second sentence emphasized: “The app is still experimental and would require clearance by the Food and Drug Administration before it could hit the market.”
Independent experts were quoted:
“One big question is, just how useful will this be for parents and doctors?
Fluid behind the eardrum is a symptom of ear infection, but “not all fluid is an infection,” says Pamela Mudd, an ear, nose and throat specialist at Children’s National Health System in Washington, D.C.
Assuming the app is shown to be effective, Mudd says, she would want to talk to parents about how to interpret the results before recommending they purchase it.
“They may not have the knowledge that they need to understand what the device is telling them,” she says. The developers suggest that the app can help parents avoid a trip to the doctor’s office, but Mudd says the opposite may be the case.
“That may increase our use of the health care system” if parents take their kids to the doctor for what may be a temporary bit of fluid behind the eardrum. There may be instances where that’s appropriate, she says.
Kenny Chan, (an) otolaryngologist in Colorado, is also concerned about that. “To speculate that this may replace the need for a physician’s visit, I think that’s a little far-fetched,” he says.
I’ve had a child and a grandchild with awful recurrent ear infections. I know what it’s like. On this story, I’ll take the even-handed approach of NPR over the promotional tone of the STAT piece and some of the other stories any day.
Let me count the ways in which I am moved to write about another unwell piece in the New York Times Well section – “Statins May Cut Glaucoma Risk.” Let’s begin with the first line: “More good news about cholesterol-lowering statin drugs.” No, it’s not good news. There’s no “take it to the bank” definitive news-you-can-use here. In fact, the story ends with a statement from one of the researchers:
…glaucoma treatment or prevention is not by itself a reason to start statins. “Our study doesn’t address whether statins stop the progress of the disease,” she said. “That would require a clinical trial.”
As is often the case, the journal article had a substantial section on acknowledged limitations (below), none of which was mentioned in the Times story. But he/she who lives by the journal article du jour often cares little about limitations, but, rather, about meeting one’s story quota. I don’t expect consumer-readers to understand everything in the authors’ limitations section, but I present it here just so you can see how willfully some of this is ignored in mainstream news coverage of research:
Although our study was large, with a long follow-up, repeated assessment of statin use and hyperlipidemia, and the availability of key covariates, our study had several limitations. We likely had misclassification of statin use and cholesterol levels, as we did not measure participants’ recall errors. This misclassification would have biased associations toward the null. Our case ascertainment method had low sensitivity; however, methodologically, RR estimates are still valid if the case definition is highly specific and the case ascertainment is unassociated with exposure. Also, as participants were mostly white health care professionals all free of cancer, our results may not be generalizable to other community-dwelling older populations with different underlying risks for POAG (eg, in predominantly African American populations). Despite our adjusting for many potential confounders, there may still have been some residual confounding. Also, to comprehensively evaluate the association between hyperlipidemia, cholesterol-lowering treatment, and POAG, we performed many secondary analyses; many of these results showed weak associations. Any notable findings (eg, possible interaction with age for statin duration) may have been owing to chance; thus, they must be interpreted with caution and confirmed in other studies. Finally, our study was observational, but a definitive means of assessing the association between statin use and POAG would be to conduct a randomized clinical trial (although more observational studies are warranted to determine ideal dosage, statin type, and target population).
Wisdom of the crowds
It is noteworthy how many smart reader comments were left on the New York Times website. Clearly, readers are fed up with this repeated Well pablum. Examples of reader comments:
Many articles about statins, like this one, tend to accentuate the positive. The downsides of statins are well-documented and should be presented alongside any supposed benefits of those drugs.
“More good news about cholesterol-lowering statin drugs”?…Really? The second to last sentence says statins should not be started to treat glaucoma…wise.
Really tired of medical articles that “may” help a condition. Isn’t this laudatory article a little premature?
I can’t help wondering if this publication gets some sort of remuneration from articles like this – an ad for statins. It is ironic that in the same days publication the horrors of generic drugs are exposed rightfully while the risks of statins are glossed over in a facile puff piece. Perhaps I expect too much.
This is the second article by Nicholas Bakalar in two weeks promoting statins.
When will you cover the patients who take statins and develop muscle pain that does not always go away once stopping the drug? Polymyalgia is hard to treat, especially in seniors, and being unable to lift one’s arms or bend one’s knees is quite a valid reason for discontinuing. This keeps getting glossed over in the breathless coverage of these drugs. Stating that these effects are “rare” does nothing for the people who experience it. And if that side effect is so rare, I must have won a strange lottery because I know 4 people who went through this – some never got fully recovery.
I honestly believe it is better for the New York Times to stop reporting 226-word stories in the Well column. Use that paltry allotment of words in the Style section or the crossword puzzle instead, where no harm will be done. Increasingly, today’s news and health care consumers demand more context and analysis in health care journalism. They’re not getting it from this kind of shovelware – shoveled direct from journal to news story – without much or any independent vetting.
Spot the word “first” in a headline and you might assume a major milestone has been reached. Think first person on the moon, first woman on the Supreme Court.
But in health care news, things heralded for being first might not amount to a clear advance for patients. Take two recent FDA announcements that made a splash despite weak evidence that they really help people.
In March the FDA announced its approval of a “first treatment for post-partum depression.” Many stories parroted the “first” framing of an FDA news release:
Showering attention on a treatment because it’s a first can shift the focus from what matters: how well it actually works.
Uncertainty gets brushed over
That’s what happened in the coverage of the postpartum drug, brexanolone, which is marketed as Zulresso. The Times was the only mainstream news organization story that I saw that reported meaningful outcomes data, and that was near the end of a 38-paragraph story.
Many stories ran with sensational language like “breakthrough,” “game-changer,” and “historic.” The Washington Post referred to it as a “major advance.”
CNN anchor Erica Hill mentioned the FDA’s designation of the drug as a “breakthrough therapy” without explaining that the FDA’s use of the word “breakthrough” has an entirely different meaning than the dictionary definition. (We’ve written about how news stories can contribute to public confusion over the FDA definition of “breakthrough.”)
There were ubiquitous interviews with a Connecticut mother who raved that the drug quickly made her feel like “a cloud was lifted.” Despite all the reporting power of NBC, ABC, CBS, CNN, and the Times, all five featured this same woman, who was undoubtedly made available to reporters by the drug maker’s PR engine — a classic case of “tyranny of the anecdote.”
For the most part, coverage bemoaned barriers such as the drug’s high cost and the need for a multi-day intravenous infusion. It didn’t delve into the skimpy evidence of a benefit or concerns about whether the drug can be safely administered outside of a carefully controlled trial.
Some news organizations, including CNN and NBC, misleadingly reported that the drug was effective for a majority of women who took it.
The drug’s effectiveness was tested in just 130 women, a small number for a drug that will be marketed to millions.
In those manufacturer-sponsored trials, the drug performed only slightly better than a placebo. In one study, after 60 hours women who were given brexanolone improved by an average of 14.6 points on a survey called the Hamilton Depression Rating Scale (HAM-D), while women given a placebo improved an average of 12.1 points. Both groups progressed from being on the low end of very severely depressed to being considered mildly depressed.
The long-term effect is unknown. Mood improvements lasted 30 days in both the drug and placebo groups, although in one trial the placebo group actually showed additional improvement at 30 days and was doing better than those who had received the drug. Researchers said the drug’s effectiveness beyond 30 days hasn’t been studied.
Concern that women might pass out and hurt themselves or their babies was barely mentioned. Five patients who got the drug, or four percent, lost consciousness during the infusion. Adverse events were more common in the group that took the drug and included headaches, dizziness, nausea, infusion site pain, sleepiness, and fatigue.
‘Potential for unintended side effects’
It was drug safety advocate Kim Witczak — not journalists — who raised some of the toughest questions about brexanolone, such as: “Will it be a game changer for new mothers or just another marketing travesty?”
In a blog post for the website Mad in America, which bills itself as a “catalyst for rethinking psychiatric care in the United States,” Witczak said journalists didn’t sufficiently report safety concerns expressed by members of an FDA advisory panel on which she served.
Those concerns prompted the FDA to require safety procedures, called Risk Evaluation and Mitigation Strategies (REMS), including that the drug be administered only to patients in a medically supervising setting. However, REMS programs lack teeth.
REMS programs are neither enforceable nor a guarantee of safety. As a member of the FDA Advisory Committee, I believe it’s our moral obligation to make sure that all drugs with this risk potential are scrutinized and held to a higher standard in the approval process. More research is needed before we can be sure that this is not just the latest in a long line of drugs offered to women as a quick fix with minimal efficacy and the potential for unintended side effects. I fail to see why scientifically proper studies cannot be completed prior to rollout of this new drug treatment, to ensure efficacy and safety. A rushed study process reinforces my concerns, especially when the company performing the studies is the manufacturer and there are no true checks and balances in the system.
The FDA’s framing of brexanolone as a “first” also obscured the fact that there are already treatments for postpartum depression, including psychotherapy and standard antidepressant drugs. The need for social supports for new mothers, which has been found to mitigate postpartum depression, went unacknowledged in the FDA news release and the bulk of news coverage.
“It’s not like postpartum depression is a hitherto untreated condition,” said Adriane Fugh-Berman, MD, a family medicine doctor and director of Georgetown University’s PharmedOut project. She pointed out that while brexanolone is the first drug approved by the FDA specifically for postpartum depression, that fact is not what the agency should emphasize to the public.
‘Pathetic’ regulatory system
Similarly, stories about the ADHD device largely characterized it as a breakthrough while downplaying slight evidence of a benefit.
The device was studied in just 62 kids who were randomized to the device or a placebo device for four weeks. That means only 32 children were tested on the actual device. They were assessed on a clinician-administered rating scale that included factors such as difficulty paying attention or regularly interrupting others.
ABC News’ Jennifer Ashton, MD, called a newly approved postpartum depression treatment “historic.”
At the end of week four, the FDA said in its news release, the average score in the device group decreased from 34.1 points to 23.4 points, versus a decrease from 33.7 to 27.5 points in the placebo group.
Susan Molchan, MD, a geriatric psychiatrist and one of our longtime reviewers, suggested in an email that patients are not helped very much by being told that the device is first on the market.
She pointed out that ADHD drugs are typically tested for at least 12 weeks: “Four weeks is such a short period of time. If this was all it was approved on, it points out how pathetic our device regulation system is.”
“If we knew it would be helpful, which we don’t, it would be great to have another option for treatment, especially a non drug one,” she said. “it’s just unfortunate our device regulation system is so weak that consumers really can’t ascertain from ‘approval’ that the device is any good.”
A loaded marketing term
Some experts question whether the FDA should be calling out treatments as “first” in its news release headlines. Witczak described it to me as a “loaded marketing word” — like newest and latest — that might be used to sell a car.
Steven Woloshin, MD, co-director of the Center for Medicine and Media at The Dartmouth Institute for Health Policy and Clinical Practice, co-authored research showing how using terms such as “breakthrough” and “promising” can inflate people’s beliefs about a the effectiveness of a new drug. It’s an impact that even extends to doctors.
He wrote in an email:
I think “first” is another example of marketing language finding its way into FDA communications — perhaps not as extreme as “breakthrough” but certainly along the same lines. As a result, I expect the word “first” will magnify enthusiasm for drugs/devices and show up in media reports and, of course, ads. This is a problem because in general, “new” or “first” should be seen as a warning, not as a benefit: an approved new drug/device has gone through typically rigorous testing — but on small numbers of highly selected people for short periods of time. Our understanding of the balance of benefits and harms evolves as more typical patients [such as those taking other medications or having other medial diseases] take the drug.
European regulators recognize this uncertainty by requiring a black triangle symbol on communications about drugs that require additional safety monitoring, Woloshin noted. The National Academy of Medicine, formerly the Institute of Medicine, has endorsed the idea in the U.S.
The FDA news release on brexanolone described potential harms but did a poor job of helping people appreciate how well the drug works, he said. For example, it offered no data on how much depression scores changed for new moms in the trials, or the percent who were no longer classified as depressed after treatment.
Instead, the release stated that the drug “demonstrated superiority to placebo in improvement of depressive symptoms at the end of the first infusion. The improvement in depression was also observed at the end of the 30-day follow-up period.” Those vague descriptors popped up repeatedly in news stories.
‘First’ should heighten caution
While some medical experts say “first” doesn’t aid public comprehension, the FDA seems enamored with that framing. By my count, the agency used “first” in 14 headlines so far this year. That’s on pace to exceed last year’s total of 37, which was up from 30 in 2017 and 20 in 2016.
Woloshin suggested that’s a trend in the wrong direction: “I think the way forward is for FDA — and journalists — to avoid marketing terms, to clearly summarize AND quantify key benefits and harms, and remind everyone that when it comes to drugs and devices, first or new should generally be seen as a red flag to heighten caution rather than enthusiasm.”
We see this word a lot: “potential.” In medical science, it’s pretty meaningless. I can safely say that my beloved mutt has potential to star in a movie some day. (After all, she looks exactly like that poopy dog in “Roma.”) Unfortunately, it’s unlikely that she will.
Claims are especially tenuous when they come from animal studies. A subhead in this release says this “new mechanism could be a ‘huge leap’ for immunology,” with no mention until the 13th paragraph that this was only tested in mice.
This release doesn’t give any data to support the idea that compression socks will help you (or your kid) perform better on the soccer field. In fact, the study involved testing just 20 players during a single soccer match and measuring whether the 10 who wore compression socks performed better on agility, heel-rise, and endurance tests after the game.
A researcher is quoted saying the study “confirmed that there is a protective effect with compression stockings that may be crucial for performance in soccer matches.” But where are the numbers to show the scope of this purported benefit?
As we’ve written, question-mark headlines can be used to skirt accountability for reporting an unproven claim.
This study showed that Facebook was a “good mechanism” to convey messages about cancer screening. A researcher states: “Engagement was high with Facebook ads, and those who viewed ads clicked through to the sign-up page, an indication of intent to enroll.”
In the last paragraph we learn: “Technically, the answer to whether Facebook advertising can prevent cancer remains unanswered – it’s impossible to tell how many patients who otherwise would have developed cancer were caught early due to the group’s text-based information program.”
The word “suggest” is a handy way to obscure that a claim isn’t proven. As we learn in the second-to-last paragraph, the drug — metformin — appeared to have an effect on mice with heart failure symptoms, not humans.
Unmentioned is that previous research has similarly indicated metformin may help people with heart failure, but it’s an idea that has yet to be shown conclusively in a randomized clinical trial.
Similarly, “may” is a word that masks uncertainty.
We’ve often advised readers that whenever they see “may” in a claim about a study result, feel free to substitute “may not.” Chances are good that you’d end up being right.
I don’t want to be too hard on this news release, which can be credited with explaining how this randomized trial at Duke University was conducted and emphasizing that results need to be interpreted cautiously due to the small number of participants.
But in using the word “may” no fewer than five times, it obscures the point — made near the end of the release — that bigger studies involving multiple trial sites are needed to prove that exercise sustainably improves executive function in adults at risk for cognitive decline.
Exaggeration is common
You might be tempted to think that yesterday’s batch of incautious releases — four out of five of which were issued by a university — was an anomaly, and that academic institutions usually take utmost care in communicating medical research to the public.
But check out a 2009 paper by Drs. Steve Woloshin and Lisa Schwartz of the The Dartmouth Institute, which scrutinized 200 news releases issued by academic medical centers.
They found that news releases from academic institutions often exaggerated findings:
Forty-four percent promoted animal or laboratory research, of which 74% explicitly claimed relevance to human health.
Of releases about human research, 23% omitted study size and 34% didn’t quantify results.
Only 17% of releases on human research promoted studies with the strongest designs (randomized trials or meta-analyses).
Forty percent reported on the most limited human studies—those with uncontrolled interventions, small samples of less than 30 participants, surrogate outcomes, or unpublished data — with 58% lacking relevant cautions.
For a closer look on what consumers and journalists should watch out for when they read news releases, go to our recent series on problematic PR releases.
“The quickest strategy for improvement,” Woloshin and Schwartz wrote, “would be for centers to issue fewer releases about preliminary research, especially unpublished scientific meeting presentations, because findings often change substantially—or fail to hold up—as studies mature.”
Dec. 14th’s lower court ruling that the Affordable Care Act (ACA) is unconstitutional has again thrust health care back on the national agenda. It also gives journalists a chance to redeem themselves from what I have argued has been poor news coverage that helped fuel the public backlash against the law from the beginning.
A lot has changed since the law passed in 2010. Perhaps most important the public has come to believe that people should be able to get health insurance, even if they have preexisting medical conditions. A Kaiser tracking poll before the November election found 75% of Americans said it was “very important” that protections for preexisting conditions and ensuring guaranteed coverage remain. The fact that so many candidates, including Republicans, aired commercials citing their bona fides when it came to preexisting conditions reinforced the potency of that issue among voters.
While further reform will be challenging given the current political climate, serious consideration of the issue is taking root as a wave of new lawmakers sweep into power on the promise of guaranteed coverage for all. This is a debate journalists ought to be illuminating for readers.
A shift in public attitudes
Despite last Friday’s ruling, there’s no going back to the old world of health insurance, a point that many Democrats were beginning to make clear. The drive for a more inclusive health care system will continue–regardless of continued efforts by opponents to strike down the ACA.
Since the failure of Congress in 2017 to kill the ACA, which many members promised they would do, I’ve detected a shift in the public’s attitudes toward some kind of national health care system. T. R. Reid, former Washington Post reporter and advocate for universal health insurance, who worked on the failed 2016 campaign in Colorado, told me he has observed the same shift in the public’s thinking: “There is much acceptance. The striking change came after people saw Congress struggle with it and fail. Everybody can see what we have is unworkable.”
Are journalists up to the challenge?
Whether journalists are up to the challenge of chronicling a new and vicious debate remains to be seen.
Coverage of the now-suddenly crucial public debate over alternatives to the ACA has so far been relatively scarce with the public pretty much in the dark about the various plans being formulated by interest groups.
A few days before the midterm elections, a story with a clever headline appeared on the sites of California Healthline and Kaiser Health News: “Quick: What’s the Difference Between Medicare-For-All and Single-Payer?” A few voters offered an opinion. One thought the government should play a larger role. Another said he wanted private health insurance companies out of the picture. But most voters approached for the article did not want to be interviewed, saying they didn’t understand the issue. Paul Her of Sacramento summed up their predicament: “I just don’t know enough.”
If reporters in Colorado, Illinois, or Virginia asked the same question, no doubt they’d get the same answers. To most Americans, what a new kind of health care system would look like is as murky as swamp water.
‘It’s time to reframe the public debate’
Nevertheless, there may be a way through the swamp, argues former Medicare administrator Don Berwick and colleagues in the policy journal Health Affairs. “It is time to reframe the public debate,” they wrote, and noted the election was “evidence of renewed public interest in expanding access to health coverage and controlling costs,” the twin shortcomings of America’s health insurance system. If the country’s 106-year-old debate over what I’ll call a national health insurance system is rekindled in the coming year, as Berwick believes it can be, journalists have a huge role to play.
What kind of reporting is needed? Berwick et al outline some key policy questions: “knotty issues such as determining the covered benefits of the plan, exploring alternative revenue sources and cost controls, deciding how providers would be paid and at what prices, how to pave a practical transition pathway from current health plans, how to craft a soft landing for the employees of the current insurance industry, and more.”
The reticence of journalists at this point so far is hard to explain. Is it that reporters are afraid to tackle the subject, or are they still suffering a hangover from the last round of health reform? During the run-up to passage of the Affordable Care Act, I felt reporters were generally not interested in writing about alternatives to the brand of reform the health policy establishment had chosen to push through Congress. As I’ve written many times, journalists were reluctant to carefully examine the bill and critique its built-in shortcomings, which contributed to many of today’s problems and some of the initial public discontent with the law. And in a few cases supporters of the ACA tried to choke off coverage of what then were called single-payer advocates.
One prominent health care blogger wrote to my editor at the Columbia Journalism Review suggesting we should avoid giving “single-payer too much ink” because single-payer advocates “don’t understand (or don’t want to understand) the economics of health care. Single-payer won’t happen—not now. Their refusal to accept that fact is muddying the waters.” In a CJR post about this, I also share how John Rother, then AARP’s chief lobbyist, approached the late Saul Friedman, Newsday’s popular columnist on aging issues, also asking him to stop writing about single payer. Rother told Friedman:
“Folks who oppose the Obama approach on the grounds that it’s not ‘good enough’ are just playing into the opposition. We can’t afford that. I certainly hope you recognize the need to get something important done on health reform this year, and won’t join the opposition.”
Hopefully journalists will be more open-minded this time around in explaining what’s happening and what’s at stake.
Outlining the steps needed for change
One of the best pieces I’ve seen so far in this latest public discussion about changing the country’s health care arrangements appeared in The Nation over a year ago. Written by Nation Institute fellow Joshua Holland, it moved way past the what’s-it-going-to-be-called stage–which is what the Kaiser story was about–to the steps needed for significant change and the challenges involved.
For example, Holland pointed out that Medicare for All, which seems to be the current front-runner label, is a partly privatized arrangement.
More than one-third of beneficiaries are in Medicare Advantage plans run by profit-making health insurance companies that have been deliberately overpaid by the federal government with the goal of eventually privatizing the entire Medicare program.
Is that what supporters and the public mean by Medicare for All? Or do they mean the traditional Medicare program where the government, not private insurers, still provides the benefit? Are they talking about the guts of Medicare—a social insurance program that people paid into during their working years and earned the right to a guaranteed benefit beginning at age 65?
We have guaranteed lifetime coverage now, says Nancy Altman, who heads the advocacy group Social Security Works. “It ends at the grave but doesn’t start until 65. We have to start at birth.”
Medicare and Social Security are social insurance programs, but our vocabulary of social insurance has “become increasingly replaced with a vocabulary of welfare and redistribution,” says Yale professor emeritus Theodore Marmor. “By the end of the 20th century, the category of social insurance had seemingly lost its place in the vocabulary of American politics.”
Holland, who supports universal coverage, is a realist in his perspective and writes that the term single-payer can also be misleading: “Most countries with ‘single-payer’ systems rely on some combination of public insurance, various mixes of mandatory and voluntary private insurance (usually tightly regulated), and out-of-pocket expenditures (often with a cap).” Canada is often held up as a single-payer system when in fact, each of the provinces is a payer and contributes funds to finance the program just as the federal government does. Whenever I’ve tried to explain the German system of “sickness funds” and why the country does not have a single-payer system but everyone has insurance, journalists’ eyes have tended to glaze over. One nice explainer is this detailed piece from The Atlantic.
It doesn’t so much matter in the coming debate exactly how other countries achieve their universal systems, but what are the undergirding principles those countries have that America does not. The principle of social solidarity comes to mind. It’s the social solidarity in Germany, Canada, and other countries that binds the population together and makes it unthinkable that any citizen would go without medical care. In the U.S. it’s acceptable in many states for its residents, especially those who are poor, to go without care. The ongoing debate over Medicaid expansion and the imposition of work requirements as a condition for coverage and thus medical treatment is a prime example. Politico noted in its coverage of Medicaid expansion in Idaho that a spokesperson for the conservative Freedom Foundation admitted more people on Medicaid “puts us on a path to single payer system,” which his group opposed.
What are the country’s common principles?
Journalists also must address the common principles the population shares or purport to share that a revised system would reflect. This can be a good starting point for future coverage. The country needs to agree or come close to agreeing on common principles before reporters can explain the different ways of achieving them. Unless they are clear, there’s a good chance the public will become thoroughly confused before a real debate even gets started.
Last week Vox published a guide to eight plans that have been introduced in Congress. Some eliminate private insurance and cover everyone while others let people buy into Medicare or Medicaid or let people continue to choose private insurance. It’s a good start and more efforts like that are needed–perhaps with even more translation than Vox offered to help people understand all the jargon.
It’s California resident Paul Her–and his statement to Kaiser that “I just don’t know enough” –that we should keep in mind as we set out to cover the next and predictably contentious debate over the kind of health system we want.
In 2005, as I began building HealthNewsReview.org, I got permission from Dr. David Henry of the Media Doctor Australia project to adopt their news story review criteria. In the ensuing 13 years, no one has ever placed in front of me a solid, specific suggestion for a better set of criteria for reviewing stories about health care interventions. This was a patient-centered approach, laying out questions that patients needed answered when hearing news about treatments, tests, products and procedures.
Eventually, our U.S. project–and others in Canada, Hong Kong, Japan and Germany–adopted the same Australian criteria. Ours was the best-funded, with the biggest team, the greatest output, and the biggest reach in terms of audience, including internationally. Now we are ceasing our systematic reviews as we close down regular daily publishing next week.
So, one last time, we update our report card on the grades for more than 2,600 news stories and more than 600 PR news releases. As I’ve often said, I don’t think that all 10 of our review criteria have an equal weight or importance. The five that I list below, and the grades for them, are the five most important in my view. They are also the five categories that gave journalists and PR professionals the most trouble.
You can do your own comparison of news organization grades
If you go to the news story review index page or any of the individual news story review pages, scroll down the page and in the right-hand margin you’ll see a colored pie chart. That chart image links to a page where you can see the “national average” of all of the 2,600+ news stories our team of reviewers has reviewed. For ease of display, how a story grades on the 10 criteria we use is translated into a star-score of 0-5 stars. The “national average” of all stories we’ve reviewed is 3.1 stars.
Below that first pie chart is another one, and a pulldown menu to let you choose any one news source at a time to see how its grade compares with the national average. The sample of stories varies widely among all the news organizations. More than 330 stories by the Associated Press and 260 stories by the New York Times, for example. But only 30 by Bloomberg, 19 by Buzzfeed, 6 by FiveThirtyEight.com, for example. Some news organizations made it harder to find relevant stories that would be eligible for review. Some news organizations we began reviewing only recently, while others have been looked at every day for 12+ years.
We would have preferred to have larger sample sizes for all news organizations. But we stretched our available funding to even get this much accomplished. And even in cases where we had smaller sample sizes, it’s still a larger sample size of this type of systematic criteria-driven reviews than anyone else in the world has ever done.
National average 3.1 stars (2,610 reviews)
Vox 4.46 stars (26 reviews)
STAT 3.91 stars (44 reviews)
Philadelphia Inquirer 3.69 stars (75 reviews)
Associated Press 3.66 stars (336 reviews)
Wall Street Journal 3.54 stars (233 reviews)
Los Angeles Times 3.43 stars (217 reviews)
NPR 3.41 stars (155 reviews)
USA Today 3.41 stars (92 reviews)
Reuters Health 3.38 stars (265 reviews), but the business-oriented Reuters arm of the company was one of the worst – 2.6 stars (80 reviews)
Washington Post 3.12 stars (121 reviews)
The Guardian 2.29 stars (42 reviews)
The once-venerable weekly newsmagazines’ online efforts are all below-average:
Newsweek 2.98 stars (61 reviews)
US News & World Report 2.97 stars (32 reviews)
TIME 2.74 stars (93 reviews)
Each of the TV networks’ online news operations – except for CNN – was also below average across the board:
CNN.com 3.36 stars (75 reviews)
NBCNews.com 3.06 stars (48 reviews)
CBSNews.com 2.47 stars (53 reviews)
ABCNews.com 2.42 stars (19 reviews)
And FoxNews Health had the lowest grade of all news organizations with at least 10 reviews – 2.13 stars (40 reviews)
PR news release grades
The distribution of news releases to journalists is where the faucet of polluted information often gets turned on. Not always. But as you can see from the grades in the chart above, news releases graded worse than news stories in each of the key categories.
While we have nearly 13 years of news story grades, our PR news release review database began just four years ago. Because we look at many more sources of PR news releases (government agencies, industry, academic medical centers, physician groups, journals, etc.) than we do for journalism, we don’t have the same interactive comparison chart for the PR products. The sample for any one organization is limited.
Common flaws in both news stories and PR news releases
More important than those grades is why the content of those articles was graded that way. Common flaws, with links to our Toolkit section where we try to help people understand and address those flaws, are listed below.
One of the things that I’m most proud of about my team’s work is that we didn’t merely grade performance. We offered constructive criticism intended to improve performance. We did that in every review, with comments explaining why our reviewers judged something to be satisfactory or unsatisfactory. Beyond that, the primers and tips and resources in our Toolkit were one of the strongest collections of such aids to be found anywhere.
And all of that help will remain available online for some time to come even though we cease regular daily publication next week. The site – and these tools – will still be accessible. I hope to keep the site open for at least three years. So, please, take advantage of all the work that went into the development of these resources.
Other stats from our project
First, it is noteworthy that in this, our final year, we attracted 15% more visitors than in our previous highest year. How’s that for going out with a bang? Our weekly email digest was sent to 6,000 people every week. Our cumulative website traffic of millions of visits during our project’s existence is something I never anticipated when I launched this in 2006.
We also have published 2,870 blog posts. These often go beyond the boundaries of daily news reporting and PR news releases to touch on topics of health care advertising, marketing, journal policies, TV talk shows – any topic that influences the public dialogue about health care.
At our peak, we had 6.3 full-time-equivalent staffers and 50 occasional editorial contributors.
We were funded for 8.5 years by the Foundation for Informed Medical Decision Making (now defunct) and for four years by the Laura and John Arnold Foundation. In the middle of those years, we went 19 months without any funding, yet kept the site alive. I’ll try to do that again as long as I can with occasional future articles by me and freelance contributors.