Food and Drug Administration Commissioner Stephen Hahn drew a barrage of criticism from scientists this week for misinterpreting the results of a study in a way that highlighted the benefits of convalescent plasma. He later apologized, but critics say the damage was done.
Kevin Dietsch/AP
hide caption
toggle caption
Kevin Dietsch/AP
Food and Drug Administration Commissioner Stephen Hahn drew a barrage of criticism from scientists this week for misinterpreting the results of a study in a way that highlighted the benefits of convalescent plasma. He later apologized, but critics say the damage was done.
Kevin Dietsch/AP
Updated at 4:35 p.m. ET
The head of the Food and Drug Administration undermined the agency’s claim that it bases its decisions on science, not politics.
During a White House event on Sunday with President Trump, FDA Commissioner Stephen Hahn used a deeply misleading statistic to claim that a treatment the agency had just authorized to treat the coronavirus would save 35 lives out of 100 people who receive the treatment.
This false claim drew heavy criticism from scientists, in news articles and on Twitter, who say it was a gross exaggeration of the benefits.
“I don’t recall an FDA or commissioner error as serious as this,” Dr. Eric Topol of the Scripps Translational Research Institute told NPR.
On Monday night, Hahn tweeted an apology, saying “The criticism is entirely justified.”
Hahn attempted to explain his error by describing it as a poor choice of words. “What I should have said better is that the data show a relative reduction in risk and not an absolute reduction in risk.”
This explanation is opaque to anyone who hasn’t followed the digital twist saga behind Hahn’s claim. Here is the context.
The 35% figure is apparently derived from an as-yet-unpublished study that observed tens of thousands of people who were given convalescent plasma to treat their coronavirus infections.
Treatment involves taking blood plasma from people who have already been infected and transfusing it to sick people. The concept is sound and has been used to treat other illnesses with some success over the past century. These transfusions include antibodies capable of neutralizing the coronavirus – at least in the laboratory.
But this particular coronavirus study lacked a comparison group — everyone in the study received convalescent plasma. Some people get it earlier in their treatment than others. So, to look for signs of effectiveness, the study compared the effects in people who received it within the first three days of hospitalization with the results in people who received the treatment later. The researchers then compared the mortality rate of these two groups, both after seven days and after 30 days.
After seven days, the death rate for the group who received convalescent plasma early in their illness was around 9%, compared with a 12% death rate for people who got it later. At 30 days, the mortality rate was 22% for the early treatment group compared with a mortality rate of 27% for those who received treatment later.
Mortality therefore apparently improved by 3 or 5 percentage points. This is the “absolute” risk reduction that Hahn’s tweet refers to. Arriving at a “relative” 35% reduction in mortality involves comparing the before and after groups in a way that greatly exaggerates the treatment effect.
Some of Hahn’s critics aren’t happy with his apology. “Serious errors undermine your credibility,” Topol tweeted. “Sorry your [Twitter thread] it’s not enough,” Topol continued. “When will you have a press conference to completely fix this? And fix the FDA website? To restore public confidence. Please.”
Jeremy Berg, former editor of Science magazine, Hahn also chided. “Be more careful with your language next time,” he wrote on Twitter. “It really matters to the public’s understanding of COVID-19 and science.”
Adding to the political tinge of this incident, the FDA press release on Sunday departed from the usual explanation of the facts and included in a headline that the clearance is “Another achievement in the administration’s fight against the pandemic”.
On top of all this, other critics have questioned the accuracy of the unpublished paper the FDA relied on to authorize convalescent serum as a treatment. The work was done by respected researchers at Mayo Clinic and Johns Hopkins University, but it has not been peer-reviewed. As part of this process, a reviewer may ask for clarification, such as whether people who received prompt treatment were more likely to be treated in recent months, for example, when comprehensive patient management COVID-19 has improved. If so, this could explain the better survival rates of people who got the plasma quickly.
Based on this study and other smaller efforts, the FDA issued an emergency use authorization for convalescent plasma. This is short of formal approval. It is strongly based on the observation that convalescent plasma appears to be safe and has reasonable enough signs of efficacy to be included as a treatment for COVID-19. However, the FDA says the treatment has yet to undergo the randomized controlled trials that are typically required for approval of a new therapy. These studies are currently in progress.
The researchers conducting this research hope that they will still be able to attract patients – and have enough blood plasma available – to complete their studies.
Hahn has previously misinterpreted scientific findings to support the Trump administration’s talking points. In March, he touted a study from China that he told NPR morning edition showed that the coronavirus tests developed in this country were of poor quality. In fact, as NPR reported, he misrepresented the article’s conclusions, based on a cryptically written scientific abstract that summarized the article. The search itself had in fact already been retracted.
The FDA’s credibility is crucial right now as it evaluates potential coronavirus vaccines. Trump has frequently suggested a vaccine could be approved before the November election. But the public already appears to be wary of a coronavirus vaccine – with a third of those polled telling pollsters they would not take it. This position could harden, or even worsen, if the public perceives that the FDA approved a product based on politics and not science.
“When politics intertwines with science, as evidenced here,” Topol says, “it could be the precursor to an even worse situation with the vaccine in the impending weeks.”
You can contact NPR Science Correspondent Richard Harris at [email protected].
