Odds ratio and risk ratio in clinical trials #2

In my previous article, I discussed the odds ratio and risk ratio (or relative risk ratio). In clinical trials with binary outcome, both odds ratio and relative risk ratio are used. Since the clinical trials are similar to the cohort studies in epidemiology field, it seems to be more reasonable to use relative risk ratio in clinical trials. However, the odds ratio may be more commonly used in practice. This may be due to the fact that the odds ratio can be easily modeled using logistic regression. This could also be due to the fact that the odds ratio is typically larger than relative risk ratio that may be desired by the researcher.

For a non-inferiority or equivalence trials with binary outcome, one may desire to have a smaller standard error, therefore a narrower confidence interval – in this case, the relative risk ratio may be better than odds ratio.

Wikipedia gives an excellent comparison between relative risk ratio and odds ratio.

In an article “How can I estimate relative risk in SAS using proc genmod for common outcomes in cohort studies?”, the calculation of odds ratio, relative risk ratio, and their confidence intervals are illustrated.

Using SAS Proc Genmod, both odds ratio, relative risk ratio, and their confidence intervals can be easily calculated:

For odds ratio:

Proc genmod data = xxx descending;
    class treatment;
    model  outcomevariable = treatment
                                                  / dist = binomial link = logit;
    estimate 'Beta' treatment 1 -1/ exp;
run;

Here, the “link=logit” can be omitted since the logit link function is default when distribution is binomial.

For relative risk ratio, 
proc genmod data = xxx descending;
    class treatment;
    model outcome = treatment
                              / dist = binomial link = log;
    estimate 'Beta' treatment 1 -1/ exp;
run;

Here, the “link=log” can NOT be omitted since the log link function is NOT default when distribution is binomial.

Relative risk ratio can also be estimated using poisson regression especially when the event ratio is small.
 
Proc genmod data = eyestudy;
    class id;
    model outcome = treatment
                          / dist = poisson link = log;
    repeated subject = id/ type = unstr;
    estimate 'Beta' treatment 1 -1/ exp;
run;

Here, the “link=log” can be omitted since the log link function is default when distribution is poisson.

There are several advantages of using Proc Genmod to calculate the odds ratio and risk ratio. Adjusted odds ratio and adjusted relative risk ratio can be easily calculated when there are continuous or categorical covariates. The model can be easily modified to fit the longitudinal data.

Proc Logistic can be used for calculating the odds ratio (and the confidence interval) and can adjust for continuous or categorical covariates. However, Proc Logistic can not be used for calculating the relative risk ratio.

proc logistic;
     model outcome = treatment;
run;

Proc FREQ can be used for calculating the odds ratio and relative risk ratio (and asymptotic confidence interval) using /cmh option. For adjusted odds ratio or risk ratio, only the categorical covariate can be used. 

proc freq order=data;
    tables covariate*treatment*response / CMH;
run;

In the output, the odds ratio will be explicitly indicated while relative risk ratio will be labeled as “col1 risk” or “col2 risk”.

There is no regulatory guidance forcing the use of odds ratio or relative risk ratio. However, in clinical trials, if we compare the ratio of two proportions (eg the proportion of success in treated group vs. the proportion of success in control group), relative risk ratio seems to be better. Relative risk ratio resemble the hazard ratio in may aspects.

In FDA’s guidance “Diabetes Mellitus — Evaluating Cardiovascular Risk in New Antidiabetic Therapies to
Treat Type 2 Diabetes” The calculation of risk ratio is suggested. The guidance indicated
"Sponsors should compare the incidence of important cardiovascular events occurring with the investigational agent to the incidence of the same types of events occurring with the control group to show that the upper bound of the two-sided 95 percent confidence interval for the estimated risk ratio is less than 1.8. This can be accomplished in several ways. The integrated analysis (meta-analysis) of the phase 2 and phase 3 clinical trials described above can be used. Or, if the data from all the studies that are part of the meta-analysis will not by itself be able to show that the upper bound of the two-sided 95 percent confidence interval for the estimated risk ratio is less than 1.8, then an additional single, large safety trial should be conducted that alone, or added to other trials, would be able to satisfy this upper bound before NDA/BLA submission. Regardless of the method used, sponsors should consider the entire range of possible increased risk consistent with the confidence interval and the point estimate of the risk increase. For example, it would not be reassuring to find a point estimate of 1.5 (a nominally significant increase) even if the 95 percent upper bound was less than 1.8.”

In a presentation by Dr Bob O’Neill “Non-Inferiority Clinical Trials Some key statistical issues and concepts” he suggested that Log (Hazard ratio) or Log(relative risk) is preferred when determining the non-inferiority margin

In statistical review for Maxipime (cefepime hydrochloride) NDA, the risk ratio and 95% confidence interval are used. In an article “Relative risks of reported serious injury and death associated with hemostasis devices by gender”, the risk ratio were reported.

However, there are also many cases of using odds ratio instead of risk ratio in clinical trials. Some examples are:

• Antiepileptic drugs and suicidality NDA review
• Tenecteplase, recombinant (TNK-tPA) for Treatment of acute myocardial infarction (AMI) BLA
• In vitro polymerase chain reaction (PCR) based assay for HBV viral load detection PMA review

In summary, while both odds ratio and risk ratio can be used in clinical trials, risk ratio should be given the adequate emphasis in comparing the ratio of two proportions (between two treatment groups). In non-inferiority clinical trials, the risk ratio and its confidence interval are preferred.

FDA's new guidance on device approval process and device clinical trial

For a long time, device makers have been complaining about the FDA's device approval process. I personally heard a lot of talks that the device approval process is easier than the approval of the drug and the biological product. The requirements for clinical trials in device approval has lower standard comparing to  clinical trials in drugs and biological products. FDA device division (CDRH) has the loose criteria for the product approval. In response to the critics, FDA now releases two new draft guidance on August 15, 2011 and is seeking to educate industry on device approval.

The first guidance “Factors to Consider when Making Benefit-Risk Determinations in Medical Device Premarket Review” explains the agency's approval process for diagnostic and therapeutic devices, specifically:

• How the agency weighs the benefits and risks of a device
• How the agency assess the seriousness of a disease or condition
• How many people would use the device if approved
• The availability of other devices approved to treat the same condition

It is interesting enough that in examples used in this guidance, several clinical trials were mentioned as flawed or with unreliable data, however, FDA would approve the device anyway. This is just another reflection that CDRH indeed has lower data quality standard comparing to CDER for drugs and CBER for biological products.

In the second draft document “Design Considerations for Pivotal Clinical Investigations for Medical Devices”, the FDA laid out its expectations for clinical trials for medical devices. The agency stated that it looks for a study to provide reasonable assurance that the device is safe and effective.

Back in July 2011, FDA issued a guidance on “in Vitro Companion Diagnostic Devices” where the In Vitro Companion diagnostic device (IVD companion diagnostic device) is defined as an in vitro diagnostic device that provides information that is essential for the safe and effective use of a corresponding therapeutic product. The guidance intended to accomplish the following:

• Define in vitro companion diagnostic device
• Explain the need for FDA oversight of IVD companion diagnostic devices
• Clarify that, in most circumstances, if use of an IVD companion diagnostic device is essential for the safe and effective use of a therapeutic product, the IVD companion diagnostic device and therapeutic product should be approved or cleared contemporaneously by FDA for the use indicated in the therapeutic product labeling
• Provide guidance for industry and FDA staff on possible premarket regulatory pathways and FDA’s regulatory enforcement policy
• Describe certain statutory and regulatory approval requirements relevant to therapeutic product labeling that stipulates concomitant use of an IVD companion diagnostic device to ensure safety and effectiveness of the therapeutic product

No matter what the device is, if clinical trials are required, the statistical analyses can be based on the FDA guidance “Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests” that was issued in 2007.

In Device field, not all approval requires clincial trials. If clinical trials are required, they do not have to be interventional. If clinical trials are interventional, they do not have to be randomized, controlled. If clinical trials are randomized, controled, they do not have to be double-blinded. Some clinical trials in device field may be using the samples (for example blood samples) from patients with no intervention performed. The blood samples could be historical retains or prospectively collected. The use of the blood samples for device clinical trial still needs the informed consent from the patient.

Equipoise and Lack of Equipoise in Randomized Clinical Trials

According to Merriam-Webster dictionary, the word "Equipoise" means a state of equilibrium. In clinical trial, the concept of 'clinical equipoise' means that there is genuine uncertainty over whether a treatment will be beneficial. In other words, in randomized controlled clinical trials, there should be substantial uncertainty or there is no clear evidence that one treatment arm is particularlly better or worse. Clinical equipoise provides the ethical basis for medical research involving patients assigned to different treatment arms of a clinical trial - it is unethical to assign a subject to an inferior arm if the lack of equipoise exists and if there is substantial evidence that one treatment is better or worse than another treatment.

In the real word, when we plan a randomized, controlled clinical trial, 'lack of equipoise' may often exist. This is especially true in late stage clinical trials. In late stage clinical trials, there typically be some evidences about the benefit and treatment effect of the experimental drug from early phase I or phase II clinical trials. Our sample size calculation is based on the assumed treatment effect of the experimental drug.


We recently published a paper in the Journal of Neurology "Challenges of clinical trial design when there is lack of clinical equipoise: use of a response-conditional crossover design" where we discussed a situation of 'lack of equipoise' and the use of a response-conditional crossover design to ease the concern about the lack of equipoise in clinical trial design. Several small trials had suggested that IVIg is beneficial in treating the disease CIDP - lack of equipoise. However, in the absence of an approved treatment for this indication,  gaining regulatory approval for the use of IVIg in this indication required the conduct of large-scale, placebo-controlled confirmatory trials. Using the response-conditional crossover design, we eased the concern about subjects exposed to the 'perceived' inferior arm (Placebo in this case). The results indicated that we minimized subject's exposure to the inferior treatment arm.

Interpretation of 'clinical equipoise' may be different among clinicians, investigators, patients, clinical trial sponsors, and regulatory agencies. Evidences of treatment effects from small-scale clinical trials may be thought as real evidence for clinician and patients, but not for regulatory agencies.

When we bring the overall benefit/risk into the picture for assessing the 'clinical equipoise', it may be difficult to determine whether or not a clinical equipoise exist or not. A new treatment may have been demonstrated beneficial in efficacy, but with great uncertainty in safety.