Another Way for Constructing Stopping Rule for Safety Monitoring of Clinical Trials

In my previous post "Constructing stopping rule for safety monitoring", I discussed the use of exact binomial confidence interval as a way to construct a stopping rule, but it was for a single arm study. 

For randomized, controlled study, the similar way can be used, but we have to calculate the exact confidence interval for the difference of two binomial proportions. We can then make a judgment if there is an excessive risk or elevated risk in the experimental arm for stopping the study for the safety reason. 

I recently read a oncology study protocol and noticed the following languages to describe the stopping criteria:  

An independent DMC will review accumulating safety data at scheduled intervals  with attention focused on the percentage of subjects with SAEs, AEs of particular concern, Grade 3 or 4 toxicities, and any Grade 5 toxicity considered at least possibly related to study treatment. Excess risk will be determined according to the lower 97.5% exact lower confidence bound on the difference between incidence rates for Group B minus Group A; a lower bound greater than 0% will be flagged as a possible reason to stop the trial. Incidence calculations will depend on the respective numerators and denominators at the time of each interim look. Wilson scores method will be used to calculate confidence limits.

To use this approach for stopping rule, we will need to calculate the exact confidence interval on a continuous basis. While Wilson scores method is mentioned for calculating the exact confidence interval, there are other methods for this calculation too.

In a paper by Will Garner (2007) Constructing Confidence Intervals for the Differences of Binomial Proportions in SAS, total 17 methods were discussed for calculating the confidence interval for two binomial proportions where a couple of methods could calculate the exact confidence interval. In SAS, proc freq can be used to calculate the exact confidence interval based on the method by Santner and Snell and the method by Chan and Zhang.

In the example below, I constructed a data set with two scenarios:
Scenario #1: 4 out of 10 patients in group A having an event and 0 out of 10 patients in group B having an event.
Scenario #2: 5 out of 10 patients in group A having an event and 0 out of 10 patients in group B having an event.

The lower bound of 95% confidence interval for scenario #1 and scenario #2 will be -0.0856 and 0.0179 respectively based on Santner-Snell exact method. Since the lower bound of 95% confidence interval for scenario #2 is greater than 0, the stopping rule for safety will be triggered.

data testdata;
 input trial treat $ x n alpha;
 datalines;
 1 A 4 10 0.05
 1 B 0 10 0.05
 2 A 5 10 0.05
 2 B 0 10 0.05
;
data testdat1;
 set testdata;
 by trial;
 if first.trial then treatn = 1;
 else treatn = 2;
 y = n - x; p = x/n; z = probit(1-alpha/2);
run;
data testdat2a(keep=trial x y z rename=(x=x1 y=y1));
 set testdat1;
 where treatn = 1;
run;
data testdat2b(keep=trial x y rename=(x=x2 y=y2));
 set testdat1;
 where treatn = 2;
run;
data testdat2;
 merge testdat2a testdat2b;
 by trial;
run;
proc transpose data = testdat1 out = x_data(rename=(_NAME_=outcome COL1=count));
 by trial treat;
 var x y;
run;
/* Methods 1, 6 (9.4 only), 10, 12, and 13 (9.4 only) */
ods output PdiffCLs=asymp1;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff (CL=(WALD MN WILSON AC HA));
 weight count;
run;
data asymp1;
 set asymp1;
 length method $25.;
 if Type = "Agresti-Caffo" then method = "13. Agresti-Caffo";
 else if Type = "Hauck-Anderson" then method = "12. Hauck-Anderson";
 else if Type = "Miettinen-Nurminen" then method = " 6. Miettinen-Nurminen";
 else if index(Type,"Newcombe") > 0 then method = "10. Score, no CC";
 else if Type = "Wald" then method = " 1. Wald, no CC";
 keep trial method LowerCL UpperCL;
run;

/* Method 5: MEE (9.4 only) */

ods output PdiffCLs=asymp2;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff(CL=(MN(CORRECT=NO)));
 weight count;
run;
data asymp2;
 set asymp2;
 length method $25.;
 method = " 5. Mee";
 keep trial method LowerCL UpperCL;
run;

/* Method 3: Haldane */
data asymp3;
 set testdat2;
 by trial;
 length method $25.;
 method = " 3. Haldane";
 p1 = x1/(x1+y1);
 p2 = x2/(x2+y2);
 psi = (x1/(x1+y1) + x2/(x2+y2))/2;
 u = (1/(x1+y1) + 1/(x2+y2))/4;
 v = (1/(x1+y1) - 1/(x2+y2))/4;
 w = z/(1+z*z*u)*sqrt(u*(4*psi*(1-psi)-(p1-p2)*(p1-p2)) + 2*v*(1-2*psi)*(p1-p2) +
4*z*z*u*u*(1-psi)*psi+z*z*v*v*(1-2*psi)*(1-2*psi));
 theta = ((p1-p2)+z*z*v*(1-2*psi))/(1+z*z*u);
 LowerCL = max(-1,theta - w);
 UpperCL = min(1,theta + w);
 keep trial method LowerCL UpperCL;
run;
/* Method 4: Jeffreys-Perks */
data asymp4;
 set testdat2;
 by trial;
 length method $25.;
 method = " 4. Jeffreys-Perks";
 p1 = x1/(x1+y1);
 p2 = x2/(x2+y2);
 psi = ((x1+0.5)/(x1+y1+1) + (x2+0.5)/(x2+y2+1))/2; /* Same as Haldane, but +1/2
success and failure */
 u = (1/(x1+y1) + 1/(x2+y2))/4;
 v = (1/(x1+y1) - 1/(x2+y2))/4;
 w = z/(1+z*z*u)*sqrt(u*(4*psi*(1-psi)-(p1-p2)*(p1-p2)) + 2*v*(1-2*psi)*(p1-p2) +
4*z*z*u*u*(1-psi)*psi+z*z*v*v*(1-2*psi)*(1-2*psi));
 theta = ((p1-p2)+z*z*v*(1-2*psi))/(1+z*z*u);
 LowerCL = max(-1,theta - w);
 UpperCL = min(1,theta + w);
 keep trial method LowerCL UpperCL;
run;
/* Method 16: Brown and Li's Jeffreys Method */
data asymp5;
 set testdat2;
 by trial;
 length method $25.;
 method = "16. Brown-Li";
 p1 = (x1+0.5)/(x1+y1+1);
  p2 = (x2+0.5)/(x2+y2+1);
 var = p1*(1-p1)/(x1+y1) + p2*(1-p2)/(x2+y2);
 LowerCL = max(-1,(p1-p2) - z*sqrt(var));
 UpperCL = min(1,(p1-p2) + z*sqrt(var));
 keep trial method LowerCL UpperCL;
run;
data asymp;
 set asymp1
 asymp2
 asymp3
 asymp4
 asymp5
 ;
run;
/* Methods 2 and 11 */
ods output PdiffCLs=asymp_cc;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff(correct CL=(wald wilson));
 weight count;
run;
data asymp_cc;
 set asymp_cc;
 length method $25.;
 if index(Type,"Newcombe") > 0 then method = "11. Score, CC";
 else if index(Type,"Wald") > 0 then method = " 2. Wald, CC";
 keep trial method LowerCL UpperCL;
run;
/* Exact methods: Methods 14 and 15 (Exact) */
ods output PdiffCLs=exact_ss;
proc freq data=x_data;
 by trial;
 tables treat*outcome /riskdiff(cl=(exact));
 weight count;
 exact riskdiff;
run;
data exact_ss;
 set exact_ss;
 length method $25.;
 method = "14. Santner-Snell";
 keep trial method LowerCL UpperCL;
run;

data exact;
 set exact_ss;
run;

/* Combine all of the outputs together */
data final;
 set asymp asymp_cc exact;
run;
/* Sort all of the outputs by trial and method */
proc sort data = final out = final;
 by trial method;
run;

proc print data=final;
 title "Methods and 95% Confidence Interval for Difference between two rates";
run;