Recently, a group of statisticians in Linkedin.com (presumabally all working in drug development industry) discussed the following posted questions:
"A client wants me to prepare final SAP shortly after protocol and CRFs are finalized for a Phase 3 trial, to submit to FDA prior to start of study. I find this unusual. Any experience doing so? When?"
There are responses like "I do not see why SAP need to be finalized until it is time to lock the database and unblind"; "Why do you want to wait? What will you learn or gain by waiting?"...
First of all, let's look at the ICH guidance (E9 Statistical Principles for Clinical Trials):
"The statistical analysis plan may be written as a separate document to be completed after finalising the protocol. In this document, a more technical and detailed elaboration of the principal features stated in the protocol may be included. The plan may include detailed procedures for executing the statistical analysis of the primary and secondary variables and other data. The plan should be reviewed and possibly updated as a result of the blind review of the data (see 7.1 for definition) and should be finalised before breaking the blind. Formal records should be kept of when the statistical analysis plan was finalised as well as when the blind was subsequently broken.
If the blind review suggests changes to the principal features stated in the protocol, these should be documented in a protocol amendment. Otherwise, it will suffice to update the statistical analysis plan with the considerations suggested from the blind review. Only results from analyses envisaged in the protocol (including amendments) can be regarded as confirmatory."
This indicated that the ICH principal is followed as long as the statistical analysis plan is finalized or signed off prior to the study unblinding (or database lock if it is open label study). I believe this is the common practice in industry.
There is certaily a trend to push for SAP signoff prior to the study start, especially for late stage trials or for trials with complicated statistical analysis.
For early phase exploratory trials,one of the purpose is to explore the adequate endpoint; control group, study design, sample size, study issues,... for the late confirmatory trials, it is acceptable not to finalize the statistical analysis plan too earlier. If it is phase III, confirmatory trial (or new term A&WC - adequate and well controlled study), it is better to have SAP signoff earlier.
If the study design is complicated or the statistical analysis is complicated (for example using Beyesian approach; using non-inferiority margin; using adaptive design,...), the statistical analysis section in the study protocol may not be sufficient and a detailed statistical analysis plan may have to be sent to FDA at the time of protocol submission.As one of the members from Linkedin commented "The more important a protocol is to the NDA/BLA (i.e., a pivitol trial), the sooner you should get it in front of the FDA for comments."
Another point is that SAP has mainly two parts: the text portion and the mock shells. We may just need to finalize the text portion of the SAP prior to the study start and design the mock up shells after the CRFs, annotations, and sample data are available. In reality, every study protocol contains a section for statistical analysis. The key elements for statistical analysis should be included in this section. If the statistical analysis section is not detailed enough, the expanded statistical analysis section (the text portion of SAP) should detail the things like: prespecified analysis method/statistical model; missing data handling and imputation; prespecified interim analysis plan/method; multiplicity adjustment to p value; justification for non-inferiority margin; detail adaptation method, detail bayesian method, protection of blinding; inclusion of subjects in study population,...).
SAP could become a very lengthy document. here is an example of a SAP with bayesian analysis component from FDA's website.
Several weeks ago, I attended DIA/FDA's workshop on "Adaptive design clinical trials - discussion on FDA's draft guidance". FDA has expressed the great concern about the operational biases and the study integrity if the adaptive designs (especially those not not well accepted) are used. FDA's draft guidance on adaptive design has a specific section discussing "Role of the Prospective Statistical Analysis Plan in Adaptive Design Studies"
"The importance of prospective specification of study design and analysis is well recognized for conventional study designs, but it is of even greater importance for many of the types of adaptive designs discussed in sections V and VI, particularly where unblinded interim analyses are planned. As a general practice, it is best that adaptive design studies have a SAP that is developed by the time the protocol is finalized. The SAP should specify all the changes prospectively planned and included in the protocol, describe the statistical methods to implement the adaptations, describe how the analysis of the data from each adaptive stage will be incorporated into the overall study results, and include the justification for the method of control of the Type I error rate and the approach to appropriately estimating treatment effects. The SAP for an adaptive trial is likely to be more detailed and complex than for a non-adaptive trial. Any design or analysis modification proposed after any unblinded interim analysis raises a concern that access to the unblinded data used in the adaptations may have influenced the decision to implement the specific change selected and thereby raises questions about the study integrity. Therefore, such modifications are generally discouraged. Nonetheless, circumstances can occur that call for the SAP to be updated or for some other flexibility for an unanticipated adaptation. The later in the study these changes or updates are made, the more a concern will arise about the revision’s impact. Generally, the justifiable reasons to do so are related to failure of the data to satisfy the statistical assumptions regarding the data (e.g., distribution, proportionality, fit of data to a model). In general, it is best that any SAP updates occur before any unblinded analyses are performed, and that there is unequivocal assurance that the blinding of the personnel determining the modification has not been compromised. A blinded steering committee can make such protocol and SAP changes, as suggested in the ICH E9 guidance and in the DMC guidance, but adaptive designs open the possibility of unintended sharing of unblinded data after the first interim analysis. Any design or analysis modifications made after an unblinded analysis, especially late in the study, may be problematic and should be accompanied by a clear, detailed description of the data firewall between the personnel with access to the unblinded analyses and those personnel making the SAP changes, along with documentation of adherence to these plans. Formal amendments to the protocol and SAP need to be made at the time of such changes (see 1377 21 CFR 312.30)"
9 comments:
The FDA seems to have split into multiple personalities when it comes to the development of the SAP. The first is as you have described. The second has putting enough detail in the protocol and not wanting to see the analysis plan at all (but having it finalized before the blind is broken).
I'm in favor of having it done early on, especially in complex situations but also in simpler situations. First, it is easier to complete the analysis plan when the details of the study design and case report form are fresh. The alternative, especially if you are in a CRO and doing several of these, is to have to go back and review the details of the study (including discussions that did not necessarily make it into the protocol) when they have been long forgotten.
A second reason is to begin the development of statistical programs as soon as data becomes ready. This has major implications (mostly positive) for resource planning. And speaking of resource planning, turnover in staff happens from time to time and when it does having the major details written down really helps down the line at table production time.
A third reason is that completing the SAP early forces you to reflect in detail on the study design and data collection early, which in turn enables feedback on these areas at the beginning of the study.
I would love to hear when the SAP should be finalized (and approved/signed off) for open confirmatory trials. There is not much in the guidelines on this, and nothing in ICH E9 which addresses open trials.
Thanks!!
we would follow the same rule as the blinded study. the SAP must be signed off prior to the database lock as a minimum requirement although it is better to have SAP signed off way before the database lock.
Does the FDA have an obligation to respond to the SAP plan within a certain period of time, or is there no response time obligation? (drugs not biologics)
Does the FDA have a timelines by which they must respond to an SAP? Or is there no obligatory deadline for response?
No, there is no official timeline by which the FDA must respond to an SAP that is submitted to them. In most situations, FDA will provide comments on the SAP in weeks. In some situations, FDA may not provide any comment on the SAP. It varies by the review division. For studies with complex innovative trial designs, FDA must provide comments on the SAP.
Hi Dr.Deng, I just came across your blog which has so many meaningful details in clinical trials. I am a biostatistician involving in a phase 2 drug trial study in children. I would like to know if a SAP is required to submit to FDA given that the statistical analysis section is already written in the protocol. The study design is not complex. Most of analyses will be descriptive which is conducted by me and PK analysis which is conducted by a third party.
Thank you very much.
In your situation, the SAP is not required to be submitted to FDA. It is a good practice to submit the SAP to FDA for registration trials, not for early phase trials.
Hello, can you please share your thoughts on submitting the detailed SAP as a separate document to FDA for Phase 2 trial supporting accelerated approval. what should be an appropriate timing. can it be submitted as part of EOP1 meeting package. is it useful to submit it this early.
Post a Comment