Constructing stopping rule for safety monitoring

In many early phase clinical trials, if there is a known risk or theoretically potential risk for patients' safety, it is very common that a DSMB (data safety monitoring board) will be established to review the data on a on-going basis. Usually, there is probably a stopping rule for safety - the stopping rule based on the accumulated safety data that is different from the stopping rule for each individual patient.

The stopping rule can be based on the expert's opinion and more commonly can be derived from the statistical standpoint based on the known background rate. Here are some articles about the safety monitoring and the stopping rule for safety:

• Zhu et al (2015) Statistical Monitoring of Safety in Clinical Trials
• Chap Le (lecture notes) “AE” MONITORING IN PHASE II TRIALS
• Lecture notes: The Sequential Probability Ratio Test
• FDA’s Clinical Investigator Course: Safety Considerations in Phase 1 Trials

In fact, an easy way to construct a stopping rule is to utilize the hypothesis testing approach. The idea is to rule out that the observed event rate is higher than the background event rate and is to stop the trial if excessive event rate is observed.

Here is an example from an early phase clinical trial for ischemic stroke where a known risk for the experimental drug is to cause excessive symptomatic intracranial hemorrhage (SICH). The plan is to enroll and dose up to 20 subjects. The literature reviews reveal that the background rate for SICH in ischemic stroke patients is about 10%. We can then construct a stopping rule based on the background rate for SICH of 10%.

The DSMB is empowered to stop the trial whenever they deem fit to protect the safety of patients. DSMB will consider adopting a stopping rule that evaluates the rate of SICH after 5 enrollments, requiring suspension of trial entry if the observed rate of SICH be sufficient to reject the null hypothesis that the true SICH proportion (P) is 10% or less in favor of the (one-sided) alternative hypothesis that the true SCIH proportion is more than 10%. The table below depicts the stopping guidelines. Exact binomial methods are used to compute the p values and confidence limits. One-sided lower 90% confidence limit is calculated.

Number of Subjects Enrolled in the Study	Number of SICH Needed to Reject Null Hypothesis	Observed Rate of SICH (%)	One-sided p values	Lower 90% Confidence Limit
~5	2	40%	0.0815	11.22%
6-10	3	30%	0.0702	11.58%
11-15	4	27%	0.0556	12.18%
16-20	5	25%	0.0432	12.69%

If 2 SICH events are observed in less than and equal to 5 subjects enrolled, the lower confidence limit of 11.22% will be exceeding the 10% background rate. The stopping rule for safety will be triggered and the study should be stopped.

The SAS program for calculating the stopping rule will be something like below. Notice that we obtain two-sided 80% confidence interval in order to obtain one-sided 90% lower limit. Try-and-error method can be employed to find the lowest number of SICHs that will have lower 90% confidence limit exceed the background event rate (10%).

data stopping;
  input scenario SICH $ count;
  datalines;
  1 Have 2
  1 No   3
  2 Have 3
  2 No   7
  3 Have 4
  3 No   11
  4 Have 5
  4 No   15
run;

proc freq data=stopping;
  weight count;
  tables SICH /  binomial (p=0.10) alpha=0.20 cl;
                    **p=0.10 option indicates the background rate to compare with, here we assume the
                        SICH rate of 10%;
                    ** Alpha=0.20 to obtain two-sided 80% confidence interval;
  exact binomial;  *Obtain the exact p-value;
  by scenario;
run;

Can drug be approved without demonstrating efficacy?

Pharmaceutical industry is assessing the impact of the Trump administration on regulations in drug approval and drug price. It looks like there will be some government intervention on the drug price after all. It is also likely to have some changes in drug approval process, primarily through the reform in FDA.

• Trump is promising a big shake-up at the FDA – here’s what that could mean
• Pharma Execs Like Gottlieb for FDA Commissioner - Most firms don't want radical change

There have been a lot of buzz around the Trump’s choice for new FDA commissioner. The choice of the new FDA commissioner could be a signal for the future direction in drug regulation. The most discussed candidate is Jim O’Neill and the most discussed topic is his view about the drug approval without efficacy. 

• Jim O’Neill: The Best Trump Pick You’ve Never Heard Of
• Trump Team Said to Consider Thiel Associate O’Neill for FDA
• Trump is considering a radical to lead FDA. That’s dangerous for public health
• Jim O'Neill, Trump's potential FDA head, thinks drugs should be approved without efficacy
• Trump’s F.D.A. Pick Could Undo Decades of Drug Safeguards
• What Trump's Libertarian Pals Don't Understand About The FDA -- Or Reality

I went to the youtube to find the original speech by Jim O’Neill at a biotech conference. At around 18 minutes, he discussed the ways for speeding up the drug approval process

“...another great, probably better idea is progressive licensing. We should reform the FDA so that there is an approving drug after the sponsor has demonstrated the safety.  Let people start to use them at their own risk, but not much on the safety. Let demonstrate the efficacy after it has been legalized.” 

It is true that FDA may have some bureaucratic practices and has become the roadblock to the drug approval process. A lot of times, FDA is too conservative and too cautious in approving the drugs, this is obvious ever since the Vioxx incidence. Whenever there is any uncertainty (in both safety or efficacy), FDA will rather kill (not approve) a new product. In this way, nobody will blame them for approving an unsafe drug or a drug that is not efficacious. Exceptions are some cases where FDA reluctantly approved the drug because of the pressures from outside (for example, the female Viagra for HSDD in 2015 and eteplirsen for Duchenne Muscular Dystrophy in 2016).  

• FDA bureaucracy stifles access to life-saving drugs
• Heart of Bureaucratic Darkness -Why won’t the FDA make a decision on a muscular dystrophy drug?
• The FDA Empire Strikes Back - The bureaucracy is talking down a drug the agency approved
•  FDA's Bureaucracy Stalls Innovation

It is likely that FDA reform is needed to cut bureaucratic red tape that slows the progress of science, reduce the time and cost for bringing the next generation of drugs to the market. However, approving a drug without demonstrating the efficacy (with demonstrating only the safety) will not work. It is jumping from one extreme to another.

As a matter of fact, it is more difficult to demonstrate a drug’s safety than efficacy just because some of the safety signals require a very large sample size to detect. For diabetes drugs, FDA issued a guidance “Diabetes Mellitus — Evaluating Cardiovascular Risk in New Antidiabetic Therapies to Treat Type 2 Diabetes” with notion that the evaluation is basically based on the meta analysis, not individual study because sample size requirement for detecting the cardiovascular risk is too big for a single trial.

Secondly, if a drug can be marketed without demonstrating the efficacy, the market will be full of the products like the ‘snake oil’ – no harm, but no effect. Wonder who will pay for the cost of these drugs in an era that the health insurance cost has come almost unmanageable?

Thirdly, the safety assessment is usually full of the subjective component in it. The same safety signal could be viewed as critical by some and as not critical or trivial by others. Recently, a new antibiotics, Solithromycin,was not approved by FDA because the data from the pivotal clinical trials indicated more subjects in Solithromycin group had ALT/AST elevations than the control group. However, this imbalance in ATL/AST elevations could be viewed as not critical because the elevations were transient and disappeared after the use antibiotics was stopped. Considering that the use of solithromycin is usually short-term, the risk for solithromycin causing the liver damage can be viewed as a manageable risk. However, FDA declined the approval of solithromycin anyway even at the time that fighting the antibiotics resistant infections was so critical.

My wishful thinking is that FDA will be reformed to speed up the drug approval process by removing some of the regulations, but not go to another extreme to approve a drug without demonstrating the efficacy. To protect the safety of the American people, the clinical trial and drug approval process will remain highly regulated. Even with FDA reform that causes significant reduction in developing time and cost, it will still be a lot more than developing a software.