Recently, a group of statisticians in
Linkedin.com (presumabally all working in drug development industry) discussed the following posted questions:
"A client wants me to prepare final SAP shortly after protocol and CRFs are finalized for a Phase 3 trial, to submit to FDA prior to start of study. I find this unusual. Any experience doing so? When?"
There are responses like "I do not see why SAP need to be finalized until it is time to lock the database and unblind"; "Why do you want to wait? What will you learn or gain by waiting?"...
First of all, let's look at the ICH guidance (
E9 Statistical Principles for Clinical Trials):
"
The statistical analysis plan may be written as a separate document to be completed after finalising the protocol. In this document, a more technical and detailed elaboration of the principal features stated in the protocol may be included. The plan may include detailed procedures for executing the statistical analysis of the primary and secondary variables and other data. The plan should be reviewed and possibly updated as a result of the blind review of the data (see 7.1 for definition) and should be finalised before breaking the blind. Formal records should be kept of when the statistical analysis plan was finalised as well as when the blind was subsequently broken.
If the blind review suggests changes to the principal features stated in the protocol, these should be documented in a protocol amendment. Otherwise, it will suffice to update the statistical analysis plan with the considerations suggested from the blind review. Only results from analyses envisaged in the protocol (including amendments) can be regarded as confirmatory."
This indicated that the ICH principal is followed as long as the statistical analysis plan is finalized or signed off prior to the study unblinding (or database lock if it is open label study). I believe this is the common practice in industry.
There is certaily a trend to push for SAP signoff prior to the study start, especially for late stage trials or for trials with complicated statistical analysis.
For early phase exploratory trials,one of the purpose is to explore the adequate endpoint; control group, study design, sample size, study issues,... for the late confirmatory trials, it is acceptable not to finalize the statistical analysis plan too earlier. If it is phase III, confirmatory trial (or new term A&WC - adequate and well controlled study), it is better to have SAP signoff earlier.
If the study design is complicated or the statistical analysis is complicated (for example using Beyesian approach; using non-inferiority margin; using adaptive design,...), the statistical analysis section in the study protocol may not be sufficient and a detailed statistical analysis plan may have to be sent to FDA at the time of protocol submission.As one of the members from Linkedin commented "The more important a protocol is to the NDA/BLA (i.e., a pivitol trial), the sooner you should get it in front of the FDA for comments."
Another point is that SAP has mainly two parts: the text portion and the mock shells. We may just need to finalize the text portion of the SAP prior to the study start and design the mock up shells after the CRFs, annotations, and sample data are available. In reality, every study protocol contains a section for statistical analysis. The key elements for statistical analysis should be included in this section. If the statistical analysis section is not detailed enough, the expanded statistical analysis section (the text portion of SAP) should detail the things like: prespecified analysis method/statistical model; missing data handling and imputation; prespecified
interim analysis plan/method; multiplicity adjustment to
p value; justification for non-inferiority margin; detail adaptation method, detail bayesian method, protection of blinding; inclusion of subjects in study population,...).
SAP could become a very lengthy document. here is an example of a SAP with bayesian analysis component from FDA's website.
Several weeks ago, I attended DIA/FDA's workshop on "
Adaptive design clinical trials - discussion on FDA's draft guidance". FDA has expressed the great concern about the operational biases and the study integrity if the adaptive designs (especially those not not well accepted) are used. FDA's draft guidance on adaptive design has a specific section discussing "Role of the Prospective Statistical Analysis Plan in Adaptive Design Studies"
"
The importance of prospective specification of study design and analysis is well recognized for conventional study designs, but it is of even greater importance for many of the types of adaptive designs discussed in sections V and VI, particularly where unblinded interim analyses are planned. As a general practice, it is best that adaptive design studies have a SAP that is developed by the time the protocol is finalized. The SAP should specify all the changes prospectively planned and included in the protocol, describe the statistical methods to implement the adaptations, describe how the analysis of the data from each adaptive stage will be incorporated into the overall study results, and include the justification for the method of control of the Type I error rate and the approach to appropriately estimating treatment effects. The SAP for an adaptive trial is likely to be more detailed and complex than for a non-adaptive trial. Any design or analysis modification proposed after any unblinded interim analysis raises a concern that access to the unblinded data used in the adaptations may have influenced the decision to implement the specific change selected and thereby raises questions about the study integrity. Therefore, such modifications are generally discouraged. Nonetheless, circumstances can occur that call for the SAP to be updated or for some other flexibility for an unanticipated adaptation. The later in the study these changes or updates are made, the more a concern will arise about the revision’s impact. Generally, the justifiable reasons to do so are related to failure of the data to satisfy the statistical assumptions regarding the data (e.g., distribution, proportionality, fit of data to a model). In general, it is best that any SAP updates occur before any unblinded analyses are performed, and that there is unequivocal assurance that the blinding of the personnel determining the modification has not been compromised. A blinded steering committee can make such protocol and SAP changes, as suggested in the ICH E9 guidance and in the DMC guidance, but adaptive designs open the possibility of unintended sharing of unblinded data after the first interim analysis. Any design or analysis modifications made after an unblinded analysis, especially late in the study, may be problematic and should be accompanied by a clear, detailed description of the data firewall between the personnel with access to the unblinded analyses and those personnel making the SAP changes, along with documentation of adherence to these plans. Formal amendments to the protocol and SAP need to be made at the time of such changes (see 1377 21 CFR 312.30)"
