How to interpret the risk ratio? - A controversy related to FDA's EUA announcement of convalescent plasma in treatment of Covid-19

This past Sunday, FDA Issues Emergency Use Authorization for Convalescent Plasma as Potential Promising COVID–19 Treatment, Another Achievement in Administration’s Fight Against Pandemic. The EUA (emergency use authorization) was mainly based on a study conducted by Mayo Clinic and the study results were described in the paper (not yet peer-reviewed yet) uploaded to Medrxiv.

Joyner 2020 “Effect of Convalescent Plasma on Mortality among Hospitalized Patients with COVID-19: Initial Three-Month Experience”.

 Issuance of EUA for convalescent plasma immediately draw some criticize about not having strong evidence to support the EUA approval and about the timing of the EUA announcement (right before the Republic National Convention) – We will leave this to others to debate.

One thing related to the statistics is how to present or interpret the results from Mayo Clinic’s study. In the paper, the author stated the hypotheses for analyzing the data from this study that has no concurrent control group.  

“We hypothesized, based on historical data that earlier administration of convalescent plasma with high antibody levels would be associated with reduced mortality. To address this hypothesis, we evaluated seven and 30-day mortality in 35,322 hospitalized adults transfused with COVID-19 convalescent plasma by asking two questions. First, was earlier treatment of patients with convalescent plasma after diagnosis of COVID-19 associated with reduced mortality compared to later treatment in the course of disease? Second, were higher antibody levels in the transfused convalescent plasma associated with reduced mortality?”

For the statistical analyses, the study cohort "was stratified into categories based on the days from COVID-19 diagnosis to plasma transfusion, including: 0, 1-3, 4-10, and 11 or more days and for some graphical presentations and analyses, dichotomized into 0-3 vs. 4 or more days". The relative risk was calculated for each sub-group for mortality for patients who received convalescent plasma with IgG S/co greater than 18.45 (high antibody level group) vs. patients that received less than 4.62 S/Co (low antibody level group). The pooled, or common, relative risk for 7-day and 30-day mortality were then calculated. The paper concluded, "the pooled relative risk of mortality among patients transfused with high antibody level plasma units was 0.65 [0.47-0.92] for 7 days and 0.77 [0.63-0.94] for 30 days compared to low antibody level plasma units."  For 7-day mortality, the forest plot was depicted below:

 

How to interpret the pooled relative risk of 0.65? FDA commissioner, Dr. Hahn, was criticized for overstating the efficacy results because of his interpretation of this relative risk. 

“In the optimal patients … treated with convalescent plasma at the highest titers, there was a 35% improvement in survival, which is a significant clinical benefit,” Hahn said during the press conference, noting that, “This clearly meets the criteria that we’ve established for emergency use authorization.”
 
Hahn went on to say that, “A 35% improvement in survival is a pretty substantial clinical benefit. What that means is—and if the data continue to pan out—100 people who are sick with COVID-19, 35 would have been saved because of the administration of plasma.”

Dr. Hahn's statement immediately drew a lot of criticizes and he had to come up to apologize for overstating the treatment effect of convalescent plasma in the treatment of COVID-19. 

• Lawmakers, experts raise questions after convalescent plasma EUA
• FDA chief apologizes for overstating plasma effect on virus
• FDA's Hahn Apologizes For Overselling Plasma's Benefits As A COVID-19 Treatment
• Covid-19 Live Updates: 2 F.D.A. Public Relations Experts Are Fired After Plasma Fiasco
• Is convalescent plasma safe and effective? We answer the major questions about the Covid-19 treatment

What is the right way to state the relative risk of 0.65? Well, relative risk is relative and can't be stated as an absolute benefit. CDC’s website “An Introduction to Applied Epidemiology and Biostatistics” has a chapter about the relative risk. It explains how the relative risk is calculated and how to interpret the results. 

In general, a risk ratio greater than 1.0 indicates an increased risk for the group in the numerator, usually the exposed group. A risk ratio of less than 1.0 indicates a decreased risk for the exposed group, indicating that perhaps exposure actually protects against disease occurrence.

With the convalescent plasma study by Mayo Clinic, the relative risk is calculated as the ratio of "the risk of 7-day mortality in patients receiving convalescent plasma with high antibody level" divided by "the risk of 7-day mortality in patients receiving convalescent plasma with low antibody level". A relative ratio of 0.65 (less than 1.0) indicates the benefit of receiving convalescent plasma with high antibody level - an indication of dose-response. 

It would be correct to state:

"The risk ratio of 0.65 indicates that COVID-19 patients who received convalescent plasma with high antibody level were only 65% as likely to die in 7 days as were patients who received convalescent plasma with low antibody level"

or

"The risk ratio of 0.65 indicates that the convalescent plasma with high antibody level reduced the risk of 7-day mortality by 35%"