Subgroup Analyses for Clinical Trial Data

Subgroup analyses have been used in the clinical trials for many years and when the sample size is adequate, subgroup analyses can assess the qualitative consistency of treatment effect across different subgroups and can provide the information for identifying a sub-population that may have greater benefit from the treatment. Recently, the purpose of subgroup analyses has been expanded. As mentioned in the “Concept paper on the need for a Guideline on the use of Subgroup Analyses in Randomised Controlled Trials”, the subgroup analyses can be used to:

-          Assess internal consistency,

-          Try to rescue trials that ‘fail’ based on the full analysis set

-          try to identify patient groups with the most favourable benefit-risk profile

The subgroup analyses may:

-          be pre-specified in the trial protocol, based on demographic, genomic or disease characteristics (e.g. sub-entities of a disease that are widely recognised within the medical community)

-          materialise based on a need or desire to further explore study results.

Sub-group analyses may be especially useful in personalized medicine. Through sub-group analyses, certain biomarkers/subgroups may be identified so that we can develop the tailored therapeutics.

If the sub-group analyses are not pre-specified and are post-hoc after the data dredging / mining, the interpretation of the findings from the sub-group analyese needs to be cautioned. If we introduce the ‘learning/confirming’ concept, the post-hoc sub-group analyses is a learning process and the results of interest need to be confirmed in further prospectively designed trials.

There are many examples of clinical trials where the statistically significant treatment effect in a specific sub-group identified from a study can not be subsequently verified / confirmed in prospectively designed trials.

In PRASE (THE PROSPECTIVE  RANDOMIZED AMLODIPINE SURVIVAL EVALUATION) study, the study was powered to detect the treatment difference in death from any cause and hospitalization for major cardiovascular events between Amlodipine and Placebo in overall population. The study result was not statistically significant. Sub-group analyses were then performed to compare the treatment difference in the patients with ischemic heart disease and in the patients with nonischemic cardiomyopathy.The statistically significant difference between the Amlodipine and Placebo Groups was obtained among Patients with Nonischemic Dilated Cardiomyopathy. The author concluded “Amlodipine did not increase cardiovascular morbidity or mortality in patients with severe heart failure. The possibility that amlodipine prolongs survival in patients with nonischemic dilated cardiomyopathy requires further study”

A subsequent PRAISE-2 trial was conducted to confirm the finding from the sub-group analyses of the PRAISE study. Unfortunately, the results are negative. The study results were not published in peer-reviewed paper (since it is negative), but was presented in the scientific meeting by American Heart Association

“The Trial: PRAISE-2

Presenter: Milton Packer, Columbia University College of Physicians and Surgeons, New York, NY.
The study: A randomized, double-blind, placebo-controlled trial of amlodipine in patients with nonischemic cardiomyopathy on maximal medical therapy. A total of 1652 patients were randomized to receive either amlodipine (initially 5 mg/d, then increased to 10 mg/d after 2 weeks) or placebo. The primary end point of the study was all-cause mortality. The study was powered at 90% to detect a 25% difference in mortality between the treatment arms.
The results: No significant differences existed in all-cause mortality between the 2 arms (placebo, 31.7%; amlodipine, 33.7%; hazard ratio, 1.09; log-rank P=0.32). A pooled analysis of the PRAISE-1 and PRAISE-2 trials showed no significant affect of amlodipine on mortality (placebo, 34%; amlodipine, 33.4%; hazard ratio, 0.98; log-rank P=0.81).
Summary: Despite the fact that in PRAISE-1 a survival benefit was noted with amlodipine in patients with nonischemic cardiomyopathy, no such difference was noted in PRAISE-2 or when PRAISE-1 and PRAISE-2 were combined. Long-term treatment with amlodipine does not seem to be of benefit in patients with severe, chronic heart failure. “

 

In Sepsis indication, the Kybersept study compared the 28-day all cause mortality between High-Dose Antithrombin III and Placebo treatment groups. The results indicated “High-dose antithrombin III therapy had no effect on 28-day all-cause mortality in adult patients with severe sepsis and septic shock when administered within 6 hours after the onset. High-dose antithrombin III was associated with an increased risk of hemorrhage when administered with heparin. There was some evidence to suggest a treatment benefit of antithrombin III in the subgroup of patients not receiving concomitant heparin.”

Subsequently, a paper based on the post-hoc sub-group analyses were published and concluded “High-dose AT without concomitant heparin in septic patients with DIC may result in a significant mortality reduction. The adapted ISTHDIC score may identify patients with severe sepsis who potentially benefit from high dose AT treatment.”

Unfortunately, there was no formal randomized clinical trial to study this specific sub-group. Had a prospectively designed study been conducted to study the subjects without concomitant heparin, the results might not be significant as anticipated.

If the results from the sub-group analyses are used for supporting the drug approval or claim in the product label, it is obvious that the multiplicity issue will arise. A good paper about this is “A flexible strategy for testing subgroups and overall population” by Alosh and Huque.

For statistical issues arising from the clinical trial practice, European Medicines Agency (EMA) seems to be always ahead of the US. For the sub-group analysis, EMA organized an expert workshop on subgroup analysis in November, 2011. Various topics related to sub-group analyses were discussed during the workshop. The presentation materials and the workshop summary can all be found at EMA’s website.

Programme - Expert workshop on subgroup analysis

Workshop report - Expert workshop on subgroup analysis

Presentation - Subgroup analyses – scene setting from the EU regulators perspective (Hemmings)

Presentation - Subgroup analysis: A view from an industry statistician (Keene)

Presentation - Reliably basing conclusions on subgroups of clinical trials (Koch)

Presentation - A Subgroup or a Subpopulation (Wang)

Presentation - Confirmatory subgroup analyses: Case studies (Bretz)

Presentation - Efficacy Claims and Subset Analyses in Phase III (Carroll)

Presentation - Adaptive clinical trials with subgroup selection (Stallard)

Presentation - Multiple testing methodology in the context of subgroup analysis (Dmitrienko)

Presentation - Use of Subgroups to “Rescue” a Trial or Improve Benefit-Risk (King)

Presentation - Subgroup analyses: from nasty business to stratified medicine – an HTA perspective (Lange)

Presentation - Panitumumab: The KRAS Story (Fletcher)

Presentation - Perspectives on analysing subgroup effects of clinical trials and their meta‐analyses (Roes)

Adverse Event Collections for Screening Failures

In the last issue, I stated that a more accurate definition of Screening Failures may be as following “Potential subjects who were screened for the study participation, but were not enrolled (randomized or dosed) for the study”

Once we know the definition of the screening failures, the next question is about the data collection for screening subjects in clinical trials. How much data should we collect for screening failures? Should AE be collected and entered into the clinical database? for screening failures, should SAE be reported to regulatory agency and should SAE narratives be written?

Let’s take a look at a web posting of question and answers about the AE collection for screening failures below:

Question:

I have a question related to the collecting and recording of screening failure adverse events during clinical trials. I work for a data management group of a medium sized pharma company. Currently we collect and database all AEs that occur to screening failures in our clinical trials.
On further research I have not been able to find any regulation or guidance document that requires this. Can you tell me what the FDA position is on this?
Should we
1) Record and database all AEs for screening failures?
2) Not record and database AEs for screening failures?
Clinical Site Quality Control
3) Not record or database and put systems in place that can detect an unusually large number of AEs in a specific site for screening failures.

Answer:

I'm not sure what you mean by "screening failure adverse events." Are you referring to an intercurrent illness or condition that occurs between the time that the subject was enrolled but before randomization that leads to the subject being considered a screening failure? If so, we do not consider these to be adverse events; because the subject has not yet received any study drug, it would not be "associated" with the use of the test article. Nevertheless, because the intercurrent illness or condition affects the subject's eligibility for the study, it should still be recorded by the study site and reported to the sponsor. The clinical investigator should also ensure that the subject receives appropriate medical care, either by providing it directly or by referring the subject back to the subject's primary care physician.

 

The answer above was not all accurate. During the screening period, subjects were exposed to the screening procedures which could cause adverse events; subjects could have emotional changes / nervousness / anxiety just because of the study participation;  subjects could be asked to change their regular treatment/medication to meet the inclusion /  exclusion criteria that in turn could cause side effects (such as withdrawal effect). Therefore, it is very possible for subjects to develop adverse events during the screening period. In other words, adverse events can be reported for screening failure subjects even though the subject has not yet received any study drug.

Different situations for adverse event collections can be listed in the table blow:

	All Subjects Screened
	Eligible for study participation	Screening failures
Adverse events during the screening period starting from ICF signing	Non treatment-emergent adverse events	Non treatment emergent adverse events
Adverse events reported after the first dose of the study drug	Treatment-emergent adverse events	-
Statistical analysis	Non-treatment emergent and treatment emergent adverse events are summarized separately	Not included in statistical analysis since screening failures are not included in safety population

From the table above, we can see the followings:

1. For subjects who are eventually randomized and receive the study medication, all adverse events need to be captured and recorded in the clinical database starting from the informed consent signing. By comparing the AE onset date/time with the first dose date/time, the AEs can be separated as non-treatment emergent AEs and treatment emergent AEs.

For this group of subjects, it is accurate to say that any AE occurred after the informed consent signing should be recorded.

2. For screening failures, whether or not the AEs reported during the screening period should be recorded in the clinical database is up for debating.

• Some companies do not record any data into the clinical database for screening failures. All information about the screening failures are maintained in a screening log.
• Some companies record only the key information into the clinical database for screening failures. The key information may be demographics, reason for screening failures
• Some companies choose extreme conservative way and record all available information for screening failures in the clinical database.

Unfortunately, there is no clear regulatory guidance on what information (especially adverse events) should be recorded into the clinical database for screening failures. The languages from In ICH E3 (STRUCTURE AND CONTENT OF CLINICAL STUDY REPORTS), seems to suggest that for screening failures, only information needed may be the reason for screening failures (and adverse event could be one of the reasons for screening failure).

The most extreme (or conservative) situation could be that in a study, the SAE narratives would be written for all subjects including the screening failures. In order to have sufficient information for SAE narrative writing for screening failures, all details about SAE and the ancillary information (physical example, medical history, vital signs, laboratory, …) would need to be collected. A lot of time and efforts would be spent on the data collection, but the collected data would not be very useful or at least not relevant to the purpose of the study since in the end, the screening failures would be excluded from the safety population for the safety analysis. This practice of collecting almost every detail about the screening failures is not wrong, but is not an efficient way for conducting the clinical trials.

Nowadays, the industry trend is moving toward to being compliant with CDISC standards (SDTM, aDaM). There seems to be a lot of confusions about whether or not data for screening failures should be included in the database and if so, where to include. The following weblinks from CDISC Public Discussion Forum show the confusions.

-          http://bbs.cdisc.org/bbs/forums/thread-view.asp?tid=3179&posts=1

-          http://bbs.cdisc.org/bbs/forums/thread-view.asp?tid=2287&posts=2

-          http://bbs.cdisc.org/bbs/forums/thread-view.asp?tid=2721&posts=2

-          http://bbs.cdisc.org/bbs/forums/thread-view.asp?tid=2257&posts=5

-          http://bbs.cdisc.org/bbs/forums/thread-view.asp?tid=3208&posts=2

 To summarize, for screening failures, the best way for data collection may be to collect only the demographic information and the reason for screening failures in clinical database. The reason for screening failure should include a category of “AE” since subject can be screening failure due to AE (or precisely non-treatment emergent AE) during the screening period. The details about the AE / SAEs for screen failure subjects are not necessary to be entered into the clinical database.

Definition of Screening Failures in Clinical Trials

In all clinical trials, the typical process starts with a screening period. The screening period starts with the signing of the informed consent. During the screening period, inclusion/exclusion criteria for the study participation will be checked / tested. Subjects who meet all inclusion criteria and do not meet any exclusion criterion will be eligible to be randomized (in randomized trial) or to be dosed (in non-randomized trial). Those who are not eligible for randomization or dosing will be considered as ‘screening failures”.

It seems to be a straightforward concept. However, there could be confusions if there are subjects who are not randomized or dosed due to other reasons (for example consent withdrawal, family relocation, death during the screening period,…). These situations may not be part of the inclusion/exclusion criteria, but still cause the subjects not to be randomized or dosed.

What is the definition of “screening failures”? Will screening failures only refer to subjects who do not meet the inclusion/exclusion criteria?

In the most recent version of CDISC Clinical Research Glossary, the term Screening (of Subjects) is defined as “A process of active consideration of potential subjects for enrollment in a trial’ and the term Screen Failure is defined as “Potential subject who did not meet one or more criteria required for participation in a trial.” This definition of Screening Failure is accurate only if all other situations (such as consent withdrawal, lost to follow up,…) are part of the inclusion/exclusion criteria.

In ICH E3 (STRUCTURE AND CONTENT OF CLINICAL STUDY REPORTS), while no definition of Screening Failures are provided, it has the following statement and the example flow chart.

“…It may also be relevant to provide the number of patients screened for inclusion and a breakdown of the reasons for excluding patients during screening, if this could help clarify the appropriate patient population for eventual drug use.”

 

The annex IV b above implied that there could be multiple reasons for screening failures and inclusion/exclusion criteria would just be one of these reasons.

For example, in a clinical trial, we could have a case report form to ask the reasons for screening failures and we could have the following list of reasons:

Primary reason for screening failure:

• Adverse Event
• Patient Non-compliance
• Consent Withdrawn
• Inclusion/exclusion criteria not met
• Lost to follow-up
• Death
• Other

Therefore, a more accurate definition of Screening Failures may be as following “Potential subjects who were screened for the study participation, but were not enrolled (randomized or dosed) for the study”

INSET statement in SAS Procedures

 

Recently, I find out how convenient to include some summaries statistics in a statistical graph with a statement called INSET. An INSET statement places a box or table of summary statistics, called an inset, directly in a graph created with a CDFPLOT, HISTOGRAM, PPPLOT, PROBPLOT, or QQPLOT statement. INSET statement is available in many SAS procedures (Proc Univeriate, Proc Boxplot, Proc Lifereg,...).

 

If we run the following program in SAS, the INSET statement used in Proc Univariate will place a box on the left corner of the CDF graph to indicate the mean and standard deviation.

 

data Cord;

label Strength="Breaking Strength (psi)";

input Strength @@;

datalines;

6.94 6.97 7.11 6.95 7.12 6.70 7.13 7.34 6.90 6.83

7.06 6.89 7.28 6.93 7.05 7.00 7.04 7.21 7.08 7.01

7.05 7.11 7.03 6.98 7.04 7.08 6.87 6.81 7.11 6.74

6.95 7.05 6.98 6.94 7.06 7.12 7.19 7.12 7.01 6.84

6.91 6.89 7.23 6.98 6.93 6.83 6.99 7.00 6.97 7.01

;

run;

 

title 'Cumulative Distribution Function of Breaking Strength';

 

proc univariate data=Cord noprint;

histogram strength /normal;

cdf Strength / normal;

inset normal(mu sigma);

run;

 

 The following are more examples of using INSET statements:

 

• Adding Insets with Descriptive Statistics
• Univariate Analysis and Normality Test Using SAS
• Positioning an Inset Using Compass Point Values
• INSET statement in Proc Lifereg
• INSET statement in Proc Boxplot
• INSET statement in Proc Probit
• Creating Statistical Graphics in SAS 9.2: What Every Statistical User Should Know where TEST=LOGRANK suboption instead of INSET statement is used  
• Displaying Regression Equations and Special Characters in Regression Fit Plots where INSET statement is used in PROC SGPLOT
• Now You Can Annotate Your Statistical Graphics Procedure Graph

 

Free Lectures on Statistics and Medical Research

Now that we are in the internet era, the learning is not limited to be in the school. There are great resources on the web. The elite universities now post their video lectures for the public.

One great resource for mathematics/statistics and varriety of other topics is academicearth.org which features the lectures from Universities such as Harvard, MIT, Yale, Stanford,... For statistics,  there are six classes listed. Unfortunately, there is no topic specifically to the biostatistics. Another resource is open course which currently listed 500 free online courses from top universities.

For biostatistics, while there is no video lectures, there are recorded lectures in mp3 format. For example, there are five biostatistics classes listed at education-portal.com.

• Introduction to Biostatistics
• Methods in Biostatistics I
• Methods in Biostatistics II
• Biostatistics Lecture Series
• Biostatistics for Medical Product Regulation  

For topics in medical research (not necessarily clinical trials), there are more resources available.

• UCSF mini medical school for the public
• Pharmacy and Pharmacology IME Video Library\
• Video Sharing (GBS/CIDP)
• NPR Health Care News
• NPR Podcast
• People's Pharmacy
• iHealthTube
• Medlineplus surgery videos
• CME online education
• USMLE Podcast
• Microbiology Online Course

Confidence Intervals for difference between two proportions and for the ratio of two proportions

For clinical trials with binary outcomes, the results can usually be presented as a 2x2 contingency table as below:

	Responder	Non-responder	Total
Treatment 1	n11	n12	n1
Treatment 2	n21	n22	n2

We can then calculate the proportion of responders for two treatment groups:

       p1=n11/n1

       p2=n21/n2

We have two ways to compare two treatment groups:

• The difference between two proportions: p1-p2
• The ratio of two proportions: p1/p2

p1-p2 may be called the absolute risk difference and p1/p2 is called relative risk (RR) or risk ratio. 

The confidence interval can be constructed for the difference between two proportions and for the relative risk.

For the difference between two proportions, the asymptotic confidence interval is ca1culated using the following formula:

                                 (p1-p2) +/- Z(alpha/2)*sqrt((p1 *(1-p1)/n1)+(p2*(1-p2)/n2))

Reference: Stokes, Davis, and Kock (2000) Categorical Data Analysis using the SAS System, 2nd edition

The notations may be different in the reference book and in SAS manual, but the results should be the same.

I had a posting a while ago about “Confidence Interval for Difference in Two Proportions” where I mentioned the corrections and the SAS codes.

For relative risk, the asymptotic confidence interval is calculated using the following formula:

Exp(log(RR) +/- Z(alpha/2) * sqrt((1-p1)/(n1*p1) + (1-p2)/(n2*p2)))

Reference: Agresti A (2007) An Introduction to Categorical Data Analysis, 2nd edition, JohnWiley & Sons, Inc.,

The notations may be different in the reference book and in SAS manual, but the results should be the same.

The confidence interval for relative risk can be obtained from SAS Proc Freq and can also be manually calculated using the formula above and the formula from SAS manual.

Suppose we have study results as below:

	Success	Non-success	Total
Trt1	63	3	66
Trt2	56	13	69

data example;

  length trt $8;

  input trt $ success $ count;

  datalines;

  trt1   yes 63

  trt1   no  3

  trt2   yes 56

  trt2  no  13

;

proc freq data=example;

  weight count;

  tables trt*success/measures nopercent nocol;

  title 'outputs from SAS Proc Freq';

run;

data agresti;

  n11=63;

  n21=56;

  n1=66;

  n2=69;

  p1=n11/n1;

  p2=n21/n2;

  rr = p1/p2;

  v = (1-p1)/(n1*p1) + (1-p2)/(n2*p2);

  upper = exp(log(rr) - probit(0.025)*sqrt(v));

  lower = exp(log(rr) + probit(0.025)*sqrt(v));

run;

proc print data=agresti;

  title "using the formula from Agresti's book"

run;

data sasmanual;

  n11=63;

  n21=56;

  n1=66;

  n2=69;

  p1=n11/n1;

  p2=n21/n2;

  rr = p1/p2;

  v = (1-p1)/n11 + (1-p2)/n21;

  upper = rr * exp(-probit(0.025)*sqrt(v));

  lower = rr * exp(probit(0.025)*sqrt(v));

run;

proc print data=sasmanual;

  title "using the formula from SAS manual";

run;

I recently read a paper by Fischer et al. The confidence interval for relative risk was constructed using a method by Koopman. In Koopman’s paper “Confidence Intervals for the Ratio of Two Binomial Proportions”, a Chi-square method was proposed and the method required using numerical procedure and the iterative computations. There is no SAS program available for the calculation using Koopman's method.

There are other approaches proposed for computing confidence intervals for the ratio of two proportions. However, the method for calculating the asymptotic confidence interval adopted in SAS Proc Freq is commonly used.  

Further reading:

• Applied Statistics: Proportions, risk ratios and odds ratios
• An Introduction to Categorical Data Analysis by Agresti
• Confidence limits for the ratio of two rates based on likelihood scores: non-iterative method
• Discussions in Google forum about “Confidence interval for ratio of independent binomial proportions” 
• Fagerland “Recommended confidence intervals for two independent binomial proportions”
• Dann and Koch “Review and evaluation of methods for computing confidence intervals for the ratio of two proportions and considerations for non-inferiority clinical trials”
• Test for comparing two proportions

Multiplicity Adjustments: Gatekeeping, fixed-sequence, and fallback procedures

The multiplicity issue has evolved in last several years and a lot of new procedures have been proposed mainly in handing the issues encountered in the clinical trial and the drug development area.

 

EMA/CHMP recently released a "Concept paper on the need for a guideline on multiplicity issues in clinical trials" for seeking the public comments. In the introduction of this concept paper, it mentioned some new procedures

“The guideline is not to give advice on technical questions related to a new methodology. However, the increasing complexity of hypothesis frameworks and methods used may result in new issues and pose questions on general principles that haven’t been considered before. These include consistency problems, the construction of simultaneous confidence intervals and the usefulness of newly developed methods e.g. gatekeeping and fallback procedures as well as graphical solutions in the regulatory context.”

It is necessary to differentiate the differences among three new procedures for multiplicity adjustment:

• Gatekeeping procedure
• Fixed sequence procedure
• Fallback procedure

All these three procedures are mainly designed to deal with the issue with multiple endpoints (including the primary endpoints and the secondary endpoints).

Gatekeeping Procedure is used in the situation when there are multiple endpoints and these multiple endpoints are grouped into different families. For example, a clinical trial will typically have one or more primary endpoints (family for primary endpoints) and have multiple secondary endpoints (family for secondary endpoints). If there are many secondary endpoints, the secondary endpoints can be further divided into multiple secondary different families. With gatekeeping procedure, the families are tested in a sequential manner and the tests for subsequent families will be performed only if the tests for the previous family is significant. In other words, the families of hypotheses examined earlier serve as gatekeepers. While the term ‘gatekeeping procedure’ may not used, this approach has been implemented in many clinical trials, especially in the regulatory setting. It is very typical that the secondary endpoints will only be tested only if the primary endpoint is tested significantly. In this way, the alpha-level for primary efficacy endpoints will be tested at alpha=0.05 level and not be compromised due to the consideration of the secondary endpoints.

• The website http://multxpert.com/wiki/Gatekeeping_Procedures maintained by Alex Dmitrienko et al contains a lot of useful information about the gatekeeping procedures. 
•  A slide presentation by Branching tests in clinical trials with multiple objectives is helpful in understanding the gatekeeping procedure.
•  The gatekeeping strategy is used in NDA 22-554 GI Drugs Advisory Committee Meeting NDA 22-554 Xifaxan (Rifaximin) where the secondary endpoints were grouped as “Key Secondary Endpoints” and “Other Secondary Endpoints”. Key secondary endpoints are those designated as most clinically important with pre-specified order for their analysis. P-values and confidence intervals for all other analyses are presented with NO adjustment for multiplicity. Nominal p-values and confidence intervals are consequently exploratory and cannot be used as a basis for efficacy claims in the product label if approved.

Fixed Sequence Procedure is a stepwise multiple testing procedure that is constructed using a pre-specified sequence of hypotheses. When there are multiple endpoints, these endpoints can be ordered according to their importance. All tests will be performed at the 0.05 level following the pre-specified order. Once one hypothesis is tested not significantly, all subsequent tests will not be performed.  The advantage and disadvantage of this testing procedure are obvious: power will be maximized as long as previous hypotheses are rejected, but minimized if a previous hypothesis is not rejected.  Another drawback for this procedure is that the ordering of multiple hypotheses based on the clinical importance is subjective in nature.

• Fixed Sequence Procedure could be used under the umbrella of gatekeeping procedure for one specific family. In previous example of Xifaxan NDA, the gatekeeping procedure is used in general with considering of both primary and secondary endpoints, however the fixed sequence procedure is used in testing the key secondary endpoints.
• While it is not explicitly stated, fixed sequence procedure is actually mentioned in the EMA’s   “Points to consider on multiplicity issues in clinical trials” that is issued in 2002. In the case of “two or more primary variables ranked according to clinical relevance, no formal adjustment is necessary. However, no confirmatory claims can be based on variables that have a rank lower than or equal to that variable whose null hypothesis was the first that could not be rejected.”
• A   g
  ood reference paper is “A flexible fixed-sequence testing method for hierarchically ordered correlated multiple endpoints in clinical trials” by Huque and Alosh

The Fallback Procedure is concepturely similar to a fixed sequence test, in which hypotheses are tested in an a priori order at the full alpha level. The difference of the fallback procedure from the fixed sequence test is that the full alpha of 0.05 is split for endpoints in a pre-specified order (based on the clinical relevance) and the hypotheses in late order can still be tested (but with different alpha levels) if the previous hypothesis is not rejected. To explain how the fallback procedure differs from the fixed-sequence procedure, we can use an example from a paper “the Fallback procedure for evaluating a single family of hypotheses” by Wiens and Dmitrienko. There are five endpoints with actual p-values of 0.010, 0.060, 0.0002, 0.0004, and 0.0268. With the fixed-sequence procedure, the endpoints #3, #4, and #5 will never be tested since the endpoint #2 is not significant. However, with the fallback procedure, the endpoints #3, #4, and #5 can still be tested (just at different alpha levels).

In order	With Fixed-sequence procedure	With fallback procedure*
Endpoint #1	0.010 comparing to alpha=0.05	0.010 comparing to alpha=0.04
Endpoint #2	0.06 comparing to alpha=0.05	0.060 comparing to alpha=0.04 + 0.005 and result is not significant
Endpoint #3	Not tested due to the endpoint #2 is not significant	0.0002 comparing to alpha=0.002 and result is significant
Endpoint #4	Not tested	0.0004 comparing to alpha=0.002 + 0.002 and result is significant
Endpoint #5	Not tested	0.0268 comparing to 0.002+0.002+0.001 and result is not significant

* five endpoints are given weights for their importance and alpha levels are assigned as 0.04, 0.005, 0.002, 0.002, and 0.001 (corresponding to 0.80, 0.10, 0.04, 0.04, and 0.02 of total alpha of 0.05)

 

Additional References:

•  Free online conference on multiplicity issues in confirmatory clinical trials
• Tree-structured gatekeeping tests in clinical trials

SAS tips: converting the data between SAS data sets and Excel

Converting SAS data sets to Excel Book

 In clinical trials, the database may be stored in SAS data set format. Sometimes, there is a need to convert multiple SAS data sets into an excel book. The following program can be easily modified to serve this purpose. With the small macro using Proc Export, SAS data sets can be converted into Excel book with multiple tabs (each tab is corresponding to a SAS data set). 

libname aa "c:\Data\CRF Data\Final Data\";

%macro export(dst=);
PROC EXPORT DATA= aa.&dst

            OUTFILE= "c:\Data\CRF Data\Final Data\excel\ExcelData.xls"

            LABEL DBMS=xls REPLACE;  

   SHEET="&dst";
RUN;
%mend;

%export(dst=IE);
%export(dst=DM);
%export(dst=MH);
%export(dst=VS);
%export(dst=PE);
%export(dst=CLAB);
%export(dst=PREG);
%export(dst=DRUG);
%export(dst=AE);
%export(dst=CM);
%export(dst=COM);

In the macro above, the keyword 'LABEL' is important. With 'LABEL', the SAS variable label will be used as the column header. Without 'LABEL', the SAS variable name will be used as the column header. The keyword 'LABEL' must be placed before the keyword 'DBMS' in order to be effective.
DBMS=xls indicates that the output data file will be an excel book (no version number is needed). other options for DBMS are csv,  dlm, tab, jmp. Check SAS manual for detail. 

If you run into an error due to the Excel version issue, you may try to use xlsx engine.

%macro export(dst=);
PROC EXPORT DATA= aa.&dst

            OUTFILE= "c:\Data\CRF Data\Final Data\excel\ExcelData.xlsx"

            LABEL DBMS=xlsx REPLACE; 

   SHEET="&dst";
RUN;
%mend;

Converting Excel Book to SAS data sets

The opposite way is to convert the Excel book (with multiple table) into different SAS data sets. The program below can be modified to fulfill this task.

libname aa "c:\temp\";

%macro import(dst=);
PROC IMPORT DATAFILE= "C:\Dengc\ExcelData.xls"  OUT= aa.&dst
            DBMS=xls REPLACE;
     SHEET="&dst";
     GETNAMES=YES;
RUN;
%mend;

%import(dst=primary);
%import(dst=secondary);
%import(dst= tertiary);

In the macro above, GETNAMES=YES indicates that the first row from excel spreadsheet will be used as the SAS variable name.


Other References:

• How do I read/write Excel files in SAS?
• SAS Proc Import
• SAS Proc Export

Switching from non-inferiority to superiority - is multiplicity adjustment needed?

For a non-inferiority trial, after the non-inferiority is shown, one will typically try to show the non-inferiority. People may argue that the multiplicity adjustment arise in this situation. This can be seen in a presentation of "Branching tests in clinical trials with multiple objectives" by Alex Dmitrienko and Brian Wiens. In their presentation, the multiplicity adjustment is considered for switching from non-inferiority test to superiority test as part of the gatekeeping methods. 

However, the regulatory guidelines clearly stated that no multiplicity adjustment is needed when interpreting a non-inferiority trial as a superiority trial. In EMA's guidance "Point to consider on switching between superiority and non-inferiority", the following statement is stated:

"if the 95% confidence interval for the treatment effect no only lies entirely above -delta but also above zero then there is evidence of superiority in terms of statistical significance at the 5% level (p<0.05). In this case it is acceptable to calculate the p-value associated with a test of superiority and to evaluate whether this is sufficiently small to reject convincingly the hypothesis of no difference. There is no multiplicity argument that affects this interpretation because, in statistical terms, it corresponds to a simple closed test procedure. Usually this demonstration of a benefit is sufficient on its own, provided the safety profiles of the new agent and the comparator are similar...."

In FDA's guidance "Non-inferiority clinical trials", similar statements are included:

"In some cases, a study planned as an NI study may show superiority to the active control. ICH E-9 and FDA policy has been that such a superiority finding arising in an NI study can be interpreted without adjustment for multiplicity. Showing superiority to an active control is very persuasive with respect to the effectiveness of the test drug, because demonstrating superiority to an active drug is much more difficult than showing superiority to placebo. Similarly, a finding of less than superiority, but with a 95% CI upper bound for C-T considerably smaller than M2, is also statistically persuasive."

 The multiplicity adjustment is now everywhere. It is good to know that there is no need to do the multiplicity adjustment in the situation of interpreting a non-inferiority study as a superiority.