Another Way for Constructing Stopping Rule for Safety Monitoring of Clinical Trials

In my previous post "Constructing stopping rule for safety monitoring", I discussed the use of exact binomial confidence interval as a way to construct a stopping rule, but it was for a single arm study. 

For randomized, controlled study, the similar way can be used, but we have to calculate the exact confidence interval for the difference of two binomial proportions. We can then make a judgment if there is an excessive risk or elevated risk in the experimental arm for stopping the study for the safety reason. 

I recently read a oncology study protocol and noticed the following languages to describe the stopping criteria:  

An independent DMC will review accumulating safety data at scheduled intervals  with attention focused on the percentage of subjects with SAEs, AEs of particular concern, Grade 3 or 4 toxicities, and any Grade 5 toxicity considered at least possibly related to study treatment. Excess risk will be determined according to the lower 97.5% exact lower confidence bound on the difference between incidence rates for Group B minus Group A; a lower bound greater than 0% will be flagged as a possible reason to stop the trial. Incidence calculations will depend on the respective numerators and denominators at the time of each interim look. Wilson scores method will be used to calculate confidence limits.

To use this approach for stopping rule, we will need to calculate the exact confidence interval on a continuous basis. While Wilson scores method is mentioned for calculating the exact confidence interval, there are other methods for this calculation too.

In a paper by Will Garner (2007) Constructing Confidence Intervals for the Differences of Binomial Proportions in SAS, total 17 methods were discussed for calculating the confidence interval for two binomial proportions where a couple of methods could calculate the exact confidence interval. In SAS, proc freq can be used to calculate the exact confidence interval based on the method by Santner and Snell and the method by Chan and Zhang.

In the example below, I constructed a data set with two scenarios:
Scenario #1: 4 out of 10 patients in group A having an event and 0 out of 10 patients in group B having an event.
Scenario #2: 5 out of 10 patients in group A having an event and 0 out of 10 patients in group B having an event.

The lower bound of 95% confidence interval for scenario #1 and scenario #2 will be -0.0856 and 0.0179 respectively based on Santner-Snell exact method. Since the lower bound of 95% confidence interval for scenario #2 is greater than 0, the stopping rule for safety will be triggered.

data testdata;
 input trial treat $ x n alpha;
 datalines;
 1 A 4 10 0.05
 1 B 0 10 0.05
 2 A 5 10 0.05
 2 B 0 10 0.05
;
data testdat1;
 set testdata;
 by trial;
 if first.trial then treatn = 1;
 else treatn = 2;
 y = n - x; p = x/n; z = probit(1-alpha/2);
run;
data testdat2a(keep=trial x y z rename=(x=x1 y=y1));
 set testdat1;
 where treatn = 1;
run;
data testdat2b(keep=trial x y rename=(x=x2 y=y2));
 set testdat1;
 where treatn = 2;
run;
data testdat2;
 merge testdat2a testdat2b;
 by trial;
run;
proc transpose data = testdat1 out = x_data(rename=(_NAME_=outcome COL1=count));
 by trial treat;
 var x y;
run;
/* Methods 1, 6 (9.4 only), 10, 12, and 13 (9.4 only) */
ods output PdiffCLs=asymp1;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff (CL=(WALD MN WILSON AC HA));
 weight count;
run;
data asymp1;
 set asymp1;
 length method $25.;
 if Type = "Agresti-Caffo" then method = "13. Agresti-Caffo";
 else if Type = "Hauck-Anderson" then method = "12. Hauck-Anderson";
 else if Type = "Miettinen-Nurminen" then method = " 6. Miettinen-Nurminen";
 else if index(Type,"Newcombe") > 0 then method = "10. Score, no CC";
 else if Type = "Wald" then method = " 1. Wald, no CC";
 keep trial method LowerCL UpperCL;
run;

/* Method 5: MEE (9.4 only) */

ods output PdiffCLs=asymp2;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff(CL=(MN(CORRECT=NO)));
 weight count;
run;
data asymp2;
 set asymp2;
 length method $25.;
 method = " 5. Mee";
 keep trial method LowerCL UpperCL;
run;

/* Method 3: Haldane */
data asymp3;
 set testdat2;
 by trial;
 length method $25.;
 method = " 3. Haldane";
 p1 = x1/(x1+y1);
 p2 = x2/(x2+y2);
 psi = (x1/(x1+y1) + x2/(x2+y2))/2;
 u = (1/(x1+y1) + 1/(x2+y2))/4;
 v = (1/(x1+y1) - 1/(x2+y2))/4;
 w = z/(1+z*z*u)*sqrt(u*(4*psi*(1-psi)-(p1-p2)*(p1-p2)) + 2*v*(1-2*psi)*(p1-p2) +
4*z*z*u*u*(1-psi)*psi+z*z*v*v*(1-2*psi)*(1-2*psi));
 theta = ((p1-p2)+z*z*v*(1-2*psi))/(1+z*z*u);
 LowerCL = max(-1,theta - w);
 UpperCL = min(1,theta + w);
 keep trial method LowerCL UpperCL;
run;
/* Method 4: Jeffreys-Perks */
data asymp4;
 set testdat2;
 by trial;
 length method $25.;
 method = " 4. Jeffreys-Perks";
 p1 = x1/(x1+y1);
 p2 = x2/(x2+y2);
 psi = ((x1+0.5)/(x1+y1+1) + (x2+0.5)/(x2+y2+1))/2; /* Same as Haldane, but +1/2
success and failure */
 u = (1/(x1+y1) + 1/(x2+y2))/4;
 v = (1/(x1+y1) - 1/(x2+y2))/4;
 w = z/(1+z*z*u)*sqrt(u*(4*psi*(1-psi)-(p1-p2)*(p1-p2)) + 2*v*(1-2*psi)*(p1-p2) +
4*z*z*u*u*(1-psi)*psi+z*z*v*v*(1-2*psi)*(1-2*psi));
 theta = ((p1-p2)+z*z*v*(1-2*psi))/(1+z*z*u);
 LowerCL = max(-1,theta - w);
 UpperCL = min(1,theta + w);
 keep trial method LowerCL UpperCL;
run;
/* Method 16: Brown and Li's Jeffreys Method */
data asymp5;
 set testdat2;
 by trial;
 length method $25.;
 method = "16. Brown-Li";
 p1 = (x1+0.5)/(x1+y1+1);
  p2 = (x2+0.5)/(x2+y2+1);
 var = p1*(1-p1)/(x1+y1) + p2*(1-p2)/(x2+y2);
 LowerCL = max(-1,(p1-p2) - z*sqrt(var));
 UpperCL = min(1,(p1-p2) + z*sqrt(var));
 keep trial method LowerCL UpperCL;
run;
data asymp;
 set asymp1
 asymp2
 asymp3
 asymp4
 asymp5
 ;
run;
/* Methods 2 and 11 */
ods output PdiffCLs=asymp_cc;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff(correct CL=(wald wilson));
 weight count;
run;
data asymp_cc;
 set asymp_cc;
 length method $25.;
 if index(Type,"Newcombe") > 0 then method = "11. Score, CC";
 else if index(Type,"Wald") > 0 then method = " 2. Wald, CC";
 keep trial method LowerCL UpperCL;
run;
/* Exact methods: Methods 14 and 15 (Exact) */
ods output PdiffCLs=exact_ss;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff(cl=(exact));
 weight count;
 exact riskdiff;
run;
data exact_ss;
 set exact_ss;
 length method $25.;
 method = "14. Santner-Snell";
 keep trial method LowerCL UpperCL;
run;

data exact;
 set exact_ss;
run;

/* Combine all of the outputs together */
data final;
 set asymp asymp_cc exact;
run;
/* Sort all of the outputs by trial and method */
proc sort data = final out = final;
 by trial method;
run;

proc print data=final;
 title "Methods and 95% Confidence Interval for Difference between two rates";
run;

Steroid Tapering Design Clinical Trials

In the most recent issue of New England Journal of Medicine, Stone et al published the results from "Trial of Tocilizumab in Giant-Cell Arteritis". The study used a steroid tapering design with the primary efficacy endpoint of "the rate of sustained glucocorticoid-free remission at week 52 in each tocilizumab group as compared with the rate in the placebo group that underwent the 26-week prednisone taper."

There are some chronic diseases where the effective treatment is the high dose of steroid (corticosteroid, prednisone,...). To control the symptoms, the patients are usually put on the long-term use of the high dose steroid. While the steroid treatment may be effective, it can cause serious, irreversible side effects.

The list of side effects of long-term steroid use includes, but not limited to:

• mood changes 
• forgetfulness 
• hair loss 
• easy bruising 
• a tendency toward high blood pressure and diabetes 
• thinning of the bones (osteoporosis)
• suppression of the adrenal glands
• muscle weakness
• weight gain
• cataracts 
• glaucoma

It will be useful to develop an alternative treatment that can replace the long-term steroid use or at least minimize the steroid dose required. To investigate the effect of the alternative treatment, clinical trial can be designed to demonstrate if the alternative treatment can taper down the steroid dose to very low or zero level while maintaining the stabilized symptoms – we call this as steroid tapering or steroid sparing design.

In a steroid tapering design, the purpose of the study is not to pursue the further improvement in disease symptoms. The steroid tapering design will have a study endpoint based on the reduction in the steroid dose while maintaining the stabilized symptoms. The possible efficacy endpoints could be the following:

• Steroid dose reduction at Week xx from baseline
• Percent of subjects with zero steroid dose at Week xx
• Percent of subjects with steroid dose less than xx mg at Week xx
• Percent of subjects with steroid dose reduction greater than and equal to 50%
• AUC for steroid dose between week x to week y

In one of studies investigating the steroid tapering effect of IGIV in generalized myasthenia gravis, FDA confirmed during the pre-IND meeting that the treatment effect in reducing the steroid dose is meaningful.  This study is sponsored by Grifols and is currently ongoing. as indicated in clinicaltrials.gov, the sponsor chose "the percent of subjects with steroid dose reduction greater than and equal to 50%" as the primary efficacy endpoint. 

Efficacy and Safety of IGIV-C in Corticosteroid Dependent Patients With Generalized Myasthenia Gravis

When designing a steroid tapering trial, the following issues need to be addressed:

• Steroid tapering design has a wash-in, wash-out feature. With the effect of new treatment kicking in (if the active treatment is effective), the dose of the steroids will be reduced.
• The purpose of the study is not the improvement in disease symptoms. The purpose is to maintain the symptoms (no deterioration) while the steroid dose is reduced. 
• Considering the withdrawal effect of the steroid, steroid tapering design will therefore include a run-in period – the early period when the new treatment added, but steroid tapering has not started yet. To ensure the patient safety, the steroid dose tapering will only start at the end of the run-in period. During the run-in period,
• Changes / reductions in steroid dose could influence outcomes; The treatment effect of steroid reduction must be established on the maintenance of the disease symptoms. There should be a rule to define the worsening of the clinical symptoms when the tapering must be slowed or stopped. There must be a standardized steroid tapering regimen and standardized rescue measure when disease symptoms exacerbated due to the steroid tapering.
• Subjects who entered into the study and before the randomization should have a stable steroid dose. If the patients are not on stable steroid dose while entering the study, at the end of the study, it is not possible to tease out if the steroid dose reduction is due to the fluctuation of the steroid dose itself or due to the effect of the new treatment.
• The stratified randomized can be used to include the baseline steroid dose category as a stratification factor to ensure that within each steroid dose category, equal number of subjects are randomized into active treatment or placebo control. Patients on higher steroid dose at baseline are more likely to have steroid dose reduction. The stratified randomization can minimize the biases due to this.
• If the endpoint is “the mean change from baseline in steroid dose”, the magnitude of the steroid reduction between two treatment group needs to be clinically meaningful.
• In steroid tapering design, there must be a rescue plan in the case of symptom worsening / deterioration (or exacerbation) due to the decrease in steroid dose.  
• At the end of the study, there should be a safety follow-up period. 

There is a FDA Guidance for Industry Systemic Lupus Erythematosus — Developing Medical Products for Treatment where the steroid tapering design is proposed. 

d. Reduction in concomitant steroids Reducing corticosteroid use is an important goal in treatment of patients with SLE if it occurs in the context of a treatment that effectively controls disease activity. Therefore, for a medical product to be labeled as reducing corticosteroid usage, it should also demonstrate another clinical benefit, such as reduction in disease activity as the primary endpoint. In an add-on trial to test the steroid-sparing potential of a new medical product, patients should be enrolled during a flare and randomized to the addition of the new medical product or placebo to induction doses of corticosteroids. In both study arms, when patients achieve quiescent disease, the corticosteroid dose should be tapered to a maintenance dose that is not usually associated with major toxicities while still maintaining quiescence. The induction steroid dosage and duration of induction therapy and taper schedule should be based on the severity of disease activity in the dominant organ system involved.8 The evaluation of efficacy should be based on the proportion of patients in treatment and control groups that achieve a reduction in steroid dose to less than or equal to 10 mg per day of prednisone or equivalent, with quiescent disease and no flares (see definition above) for at least 3 consecutive months during a 1-year clinical trial. For a result to be clinically meaningful, the patient population should be on moderate to high doses of steroids at baseline. Trials should also assess the occurrence of clinically significant steroid toxicities.

See also Comments on Draft Guidance for Industry: Systemic Lupus Erythematosus – Developing Drugs for Treatment (Federal Register, 70 FR 15868) 

The steroid tapering design can be used in various disease areas, the following examples are the application of steroid tapering design in severe refractory asthma, myasthenia gravis, systemic lupus erythmatosus, giant cell arteritis (GCA), and severe asthma. 

• Mepolizumab Steroid-Sparing Study in Subjects With Severe Refractory Asthma
• Phase II trial of methotrexate in myasthenia gravis

The primary measure of efficacy in our study will be the nine-month prednisone AUC (months 3–12), which measures the total prednisone doses of each patient in nine months. A reduction of prednisone AUC demonstrates that patients improved on clinical grounds so that the prednisone dose could be decreased. If the patients receiving MTX have a smaller prednisone AUC compared to the placebo patients, this will have demonstrated the efficacy of MTX. 

• A single-blinded trial of methotrexate versus azathioprine as steroid-sparing agents in generalized myasthenia gravis
• An international, phase III, randomized trial of mycophenolate mofetil in myasthenia gravis
• GL701 (DHEA, prasterone) for the Treatment of Systemic Lupus Erythematosus (SLE) in Women

Based on pre-IND discussions with FDA and consultants, it was decided that the primary efficacy variable for the corticosteroid reduction study should be, for patients who were corticosteroid dependent, a reduction of the patients’ current prednisone dose to 7.5 mg/day (upper limit of physiologic levels) or less, without worsening of SLE.
The design of the steroid sparing study was a forced titration; i.e., the patient’s steroid dose at each monthly visit was to be reduced, by algorithm, if her disease activity was stable or improved. However, when a patient worsened or flared, the associated increase in corticosteroid dose, if any, required to treat the patient’s exacerbation was at the physician’s discretion and not by algorithm. The steroid reduction algorithm was based on the patient’s disease activity improving or being stable, which was defined as no change in or a decrease in SLEDAI score in comparison to her previous visit. As such, one of the issues discussed at the pre-study investigator meeting was whether patients with low SLEDAI scores, and especially those with SLEDAI scores of 0, should be enrolled into the study. There was concern that those patients with low SLEDAI scores had inactive disease, and therefore would not be affected by steroid reduction, i.e., might not be steroid dependent. However, some investigators and consultants felt that if patients were truly dependent on steroids, their low SLEDAI scores represented active disease suppressed by corticosteroids, which would worsen or flare as soon as their corticosteroids were reduced. Therefore, because there was no experience with such trials, it was decided not to exclude patients with low SLEDAI scores. The concern regarding enrollment of potentially inactive SLE patients was revisited prior to unblinding of the study. In addition it was recognized that because of the forced downward titration of steroid dose as the patients’ disease improved or remained stable, other evaluations of disease activity such as SLEDAI, etc., would not be expected to improve.

• NDA 21-239 Statistical Review and Evaluation

The pivotal study was designed as a double-blind, randomized, placebo-controlled, parallel group trial to evaluate GL701 100 and 200 mg/day versus placebo in female patients with mild to moderate prednisone-dependent systemic lupus erythematosus (SLE).
The study included two primary efficacy variables. The first one was responder rate. A responder was defined as a patient with the achievement of a decrease in prednisone dose to 7.5 mg/day or less sustained for no less than three consecutive scheduled visits, including the termination visit (i.e., two consecutive months), on or after Visit 7. The second primary efficacy variable was percent decrease in prednisone dose determined by comparing the prescribed prednisone (or steroid equivalent) dose at Baseline (Qualifying Visit) and the last visit prednisone dose using the physician prescribed prednisone dose recorded on the Medication Record Form.

• Elimumab May Have Potential As A Corticosteroid-Sparing Drug When Added To Standard-Of-Care Treatment For SLE, Research Suggests.

Research suggests “the monoclonal antibody belimumab (Benlysta) may have potential as a corticosteroid-sparing drug when added to standard-of-care treatment for systemic lupus erythematosus (SLE).” Investigators found, “in pooled data from two large randomized controlled trials,” that “this blocker of B-lymphocyte stimulator was moderately associated with a higher probability of corticosteroid dose reduction and a greater average dose reduction over” one year. The findings were published in Arthritis & Rheumatology.

• Cumulative Corticosteroid Dose Over Fifty-Two Weeks in Patients With Systemic Lupus Erythematosus: Pooled Analyses From the Phase III Belimumab Trials
• Development and Implementation of a Double-Blind Corticosteroid-Tapering Regimen for a Clinical Trial

This paper described the design and operationalization of a blinded corticosteroid-tapering regimen for a randomized trial of tocilizumab in giant cell arteritis (GCA). The study design is sketched in the diagram below. The primary efficacy endpoint is “Proportion of patients in sustained remission at week 52 following induction and adherence to the protocol-defined prednisone taper regimen”

• Zonda Trial: "Oral Glucocorticoid–Sparing Effect of Benralizumab in Severe Asthma"
• Corticosteroid tapering with benralizumab treatment for eosinophilic asthma: PONENTE Trial