On Sunday evening, President Donald Trump and two of his top health officials touted a promising therapy for COVID-19 patients: blood plasma from people who had recovered from the virus.

Trump, speaking at a press conference, said, “It has proven to reduce mortality by 35 per cent. That’s a tremendous number.” Health and Human Services Secretary Alex Azar and FDA Commissioner Stephen Hahn made similar remarks touting the 35 per cent benefit.

“What that means is -- and if the data continue to pan out -- 100 people who are sick with COVID-19, 35 would have been saved because of the administration of plasma,” Hahn said.

But the data don’t actually show that.

“That was not the way that I would have worded it,” said one of the doctors who led the study, Arturo Casadevall, chair of the department of molecular microbiology and immunology at the Johns Hopkins School of Public Health. “I hope they will issue a clarification.”

What the data do show is that a higher dose of blood plasma is better than a lower one. And while there are promising signals that it will lead to a real benefit when compared to a placebo, that’s not known yet.

“Until we have a randomized controlled trial, we don’t know definitively,” Casadevall said.

Spokesmen for Hahn, Azar and Trump didn’t immediately respond to requests for comment on Monday.

The administration’s misrepresentation of the data may raise fears about how Hahn and the rest of the administration will treat data on a vaccine for the virus. Trump has said he expects one to be ready in time for his potential re-election, and on Saturday accused unnamed members of the “deep state” at the FDA of slowing work to hurt him politically. The “deep state” is a term used by Trump to describe employees of government agencies that he believes are manipulating policy to work against his interests. There is no evidence this is happening at the FDA.

The 35 per cent statistic also has several fatal flaws. Since everyone in the program received blood plasma, it’s not known what would have happened compared with patients who didn’t get the therapy. And scores of variables, like how sick the patients were and when they were treated -- that could have skewed the results.

Robert Califf, the FDA commissioner under President Barack Obama, said that he thought Hahn had misspoken. “It would be good for Steve to publish a correction,” Califf said on Twitter.

Azar and Hahn both have extensive experience with drugs and therapies. Azar is a former pharmaceutical executive, and Hahn has spent several decades treating patients and researching cancer. Before joining the FDA, he was the chief medical executive of the University of Texas MD Anderson Cancer Center, a leading oncology treatment and research hospital.

Wide Use

Blood plasma from recovered patients is being used around the world, with the hope that its infection-fighting antibodies can help combat the virus. It doesn’t appear to pose a major safety risk, and on Sunday the Trump administration announced an emergency FDA measure to make it more widely available. Several studies have shown promising signs of efficacy.

“Based on the data we have today, it’s very likely that plasma is reducing mortality,” Casadevall said. “The one thing we are missing is a randomized controlled trial,” the gold-standard test that will tell researchers and regulators if blood plasma is a breakthrough, an incremental help or something in between.

To understand the confusion over the 35 per cent figure, it’s important to look at two concepts: relative risk and absolute risk.

Imagine a clinical trial to test an experimental drug, with 2,000 patients split into two groups. The first 1,000 patients don’t get the drug, and in that group 10 people die. The other group of 1,000 patients gets the drug, and five people in that group die.

Using relative risk, that’s a 50 per cent improvement -- a tremendous number. But using absolute risk, the imaginary drug only decreases the likelihood of death by 0.5 per cent. That means 5 more of those 1,000 people treated with the drug would live, not the 500 implied if you mistakenly use the 50 per cent relative risk number.

The claim of a 35 per cent mortality benefit made by Trump, Azar and Hahn uses the first measure -- relative risk. But because clinical trials of plasma therapy haven’t been completed, how many lives it actually saves -- the absolute risk improvement -- still isn’t known.

Tweets

Doctors and patients rely on the FDA to put out authoritative information about the safety and efficacy of drugs, vaccines, medical devices and other products, guiding their use not just in the U.S. but around the world. The agency has historically carefully guarded its reputation and scientific independence,

Hahn’s comments about 35 out of 100 people being saved were still posted to the FDA’s official twitter account as of Monday afternoon.

Hahn’s new FDA spokeswoman, Emily Miller, repeated the error in a tweet after the press conference, saying “convalescent plasma has shown to be beneficial for 35 per cent of patients.” While she clarified the error in a follow-up message about an hour later, the FDA’s main twitter account still carries Hahn’s misstatement.

Alyssa Farrah, a White House spokeswoman made a similarly misleading tweet, saying that the therapy cuts mortality by 30 per cent to 50 per cent. And Michael Caputo, Azar’s chief spokesman at HHS, made a similar claim: “If you’re one of the 35 people out of a hundred who survive severe COVID symptoms because of convalescent plasma, you’re damn right this is a BREAKTHROUGH.”

Farrah didn’t immediately respond to emails requesting comment. Caputo and Miller didn’t have an immediate comment.

Hahn’s predecessor Scott Gottlieb, who served as FDA commissioner under Trump from 2017 to 2019, said he thought the FDA was right to allow convalescent blood plasma for emergency use. But he suggested the press conference hadn’t given the correct picture of what health regulators actually know from the data.

“When we overstate findings it erodes confidence in science and undermines public trust in regulatory decisions. The right message was this ‘may’ provide a benefit, it could be meaningful for some patients, but we need more evidence to prove it, Gottlieb said in a tweet.

“The way the public part was handled will erode precious public confidence,” he said. “You earn public confidence in small drops and you loose it in buckets.”