Friday, April 28, 2017

CURRENT ISSUES REGARDING DATA AND SAFETY MONITORING COMMITTEES IN CLINICAL TRIALS

Last week, I attended the 10th Annual Conference on Statistical Issues in Clinical Trials held in the camps of the University of Pennsylvania. This year, the topic is about "Current Issues Regarding Data and Safety Monitoring Committees in Clinical Trials".

The conference is supposed to discuss the current issues and emerging challenges in the practice of data monitoring committees, however, these issues and challenges discussed are not new. These issues are known for long time and remain as issues.

One thing I like this one-day annual conference is that all presentation slides including the panel discussions are posted online. For DMC discussions this year, the presentation slides are available at: www.cceb.med.upenn.edu/events/10th-annual-conference-statistical-issues-clinical-trials

The early days, the term DSMB (data and safety monitoring board) was used. After FDA issued its guidance "The Establishment and Operation of Clinical Trial Data Monitoring Committees for Clinical Trial Sponsors", the term DMC (data monitoring committee) become popular. In the conference this year, a new term DSMC (data and safety monitoring committee) is also used.

There are several talks about the training for DMC members and the need to train more people who can serve on the data monitoring committee. I felt that the discussion about the training for DMC members is targeting the wrong audience. Instead of targeting the statisticians, the focus should be on those MDs in the medical field. Very often, we have difficulties to find the MDs who can serve on the data monitoring committee, let alone to fine the MDs who have prior experience serving in the data monitoring committee. For a large scale clinical trial, there may be several committees: steering committee, event or endpoint adjudication committee, and data monitoring committee. There is often a shortage of members for these committees. 

Nowadays, the study protocols are getting more and more complicated. The DMC committee members may not understand the complexity of the complicated trial design.  This is especially true for clinical trials using adaptive design and Bayesian design. This poses challenges to the DMC members.

We used to debate whether or not the data monitoring committee should be given the semi-unblinded materials for review where the treatment group is designated as "X" or "Y" instead of the true treatment assignment. The conference presentations said loudly that DMC should have sole access to interim results with complete unblinding (not semi-unblinding) on relative efficacy & relative safety of interventions.

Here are some points from Dr Fleming’s presentation “DMCs:Pomoting Best Practices to Address Emerging Challenges
Mission of the DMC
  • To Safeguard the Interests of the Study Participants
  • To Preserve Trial Integrity and Credibility to enable the clinical trial to provide timely and reliable insights to the broader clinical community
 Proposed Best Practices and Operating Principles
  • Achieving adequate training/experience in DMC process
  • Indemnification
  • Addressing confidentiality issues
  • Implementing procedures to enhance DMC independence
                DMC meeting format
                Creating an effective DMC Charter
                DMC recommendations through consensus, not by voting
                DMC contracting process
  • Defining the role of the Statistical Data Analysis Center
  • Better integration of regulatory authorities in DMC process

REFERENCES:






Friday, April 07, 2017

Estimate the number of subjects needed to detect at least one event and the rule of three

It is sometimes important to know how likely a study is to detect a relatively uncommon event, particularly if that event is severe, such as intracranial hemorrhage (ICH), progressive multifocal leukoencephalopathy (PML), bone marrow failure, or life-threatening arrhythmia. A great many people must be observed in order to have a good chance to detecting even one such event, much less to establish a relatively stable estimate of its frequency. For most clinical trials, sample size is planned to be sufficient to detect main effects in efficacy and sample size is likely to be well short of the number needed to detect these rare events. Sometimes, we do see the request from FDA for a sample size large enough to detect at least one rare event.

Cempra is a pharmaceutical company in Chapel Hill, North Carolina with a very promising antibiotics drug. However, their NDA submission was not approved by FDA, not due to the efficacy, but due to the safety issue (precisely, due to FDA’s concern about the potential liver toxicity). In FDA’s complete response letter (CRL, in other words, rejection letter), it says:
“To address this deficiency, the FDA is recommending a comparative study to evaluate the safety of solithromycin in patients with CABP. Specifically, the CRL recommends that Cempra consider a study of approximately 9,000 patients exposed to solithromycin to enable exclusion of serious drug induced liver injury (DILI) events occurring at a rate of approximately 1:3000 with a 95 percent probability.”
The request for a study with these many subjects is unusual for a pre-market study. It will certainly not be feasible for an antibacterial drug in community-acquired bacterial pneumonia indication. It is interesting to see that the FDA applied the ‘rule of three’ in proposing the sample size.

The rule of three says: to have a good chance of detecting a 1/x events, one must observe 3x people. For example, to detect at least one event if the underlying rate is 1/1,000, one would need to observe 3,000 people. In Solithromycin case, to detect at least one DILI event if the background rate of DILI event is 1/3,000, the number of subjects to be observed would be 3 x 3,000 = 9,000 subjects.

I had previously written an articleabout the rule of three from the angle of calculating the 95% confidence interval when there is no event occurred in x number of subjects. The same rule can be applied to the situation for estimating the sample size to detect at least one event.    

In one of our studies with a thrombolytic agent, FDA is concerned about the potential side effect of intracranial hemorrhage (ICH). In FDA’s comment on IND, FDA asked us to have a safety database containing at least 560 subjects exposed to the tested thrombolytic agent. They stated:
In order to permit the occurrence of 1 ICH event without necessitating halting the study, the safety database would have to have at least 560 subjects exposed to your XYZ drug with only one ICH event to keep the ICH rate under 1%. Please comment.
In this case, the background rate of ICH is assumed to be 1%. Based on the rule of three, the safety database would be 3 x 100 = 300. However, the number of 560 is coming from the direct calculation assuming a binomial distribution.

Based on an ICH rate of 1%, assuming a binomial distribution, to detect at least one ICH event, 560 subjects will be needed in order to keep upper bound of the exact two-sided 95% confidence interval below 1%.

A small SAS program below can be used for the calculation. Adjust the sample size and then see what the exact 95% confidence intervals are.

 data test;
  input ICH $ count;
  datalines;
  Have 1
  No   560
run;

proc freq data=stopping;
  weight count;
  tables ICH /  binomial (p=0.01) alpha=0.05 cl;
           *p=0.01 indicates the background rate;
  exact binomial*Obtain the exact p-value;
run;

In clinical studies with potential side effect of rare events, if we need to base the safety database on detecting at least one such event, the sample size could be very large. As we see from the calculation or simply apply the rule of three, the sample size largely depends on the assumed background event rate. It is usually the case that the background event rate is from the literature. The literature usually have variety of the results. Using DILI as an example, the background rate from the literature could vary depending on if we talk about the rate in normal population, in community-acquired bacterial pneumonia patients, or in community-acquired bacterial pneumonia patients treated with other antibiotics. 

REFERENCES:

Wednesday, March 22, 2017

Regenerative (Medicine) Advanced Therapy: from RAT to ART to RMAT

In the United States, the government can promote or incentivize the drug development in certain areas through its policies. In FDA, there are different kind of designations, each providing specific benefits or incentives for speeding up the drug development in specific areas. Here are some of the designations/vouchers,… all with purpose to stimulate the drug development in certain areas.

For granting special status to a drug or biological product (“drug”) to treat a rare disease or condition upon request of a sponsor. This status is referred to as orphan designation (or sometimes “orphan status”). Orphan designation qualifies the sponsor of the drug for various development incentives of the Orphan Drug Act, including tax credits for qualified clinical testing. A marketing application for a prescription drug product that has received orphan designation is not subject to a prescription drug user fee unless the application includes an indication for other than the rare disease or condition for which the drug was designated.
Fast track is a process designed to facilitate the development, and expedite the review of drugs to treat serious conditions and fill an unmet medical need.

A process designed to expedite the development and review of drugs which may demonstrate substantial improvement over available therapy.

These regulations allowed drugs for serious conditions that filled an unmet medical need to be approved based on a surrogate endpoint.

A Priority Review designation means FDA’s goal is to take action on an application within 6 months.

Under Section 529 to the Federal Food, Drug, and Cosmetic Act (FD&C Act), FDA will award priority review vouchers to sponsors of rare pediatric disease product applications that meet certain criteria. Under this program, a sponsor who receives an approval for a drug or biologic for a "rare pediatric disease" may qualify for a voucher that can be redeemed to receive a priority review of a subsequent marketing application for a different product.
Authorizes the FDA to award priority review vouchers to sponsors of certain tropical disease product applications that meet the criteria specified in that section

With incentives to help bring new antimicrobials to market. Antimicrobials or antibiotics drugs can be approved after being designated as a Qualified Infectious Disease Product (QIDP) under the GAIN Act. As part of this QIDP designation, FDA’s review of the drug application is expedited. The designation also qualify the drugs for five years of marketing exclusivity to be added to certain exclusivity already provided by the FDA.


With 21st Century Cure Act passed last December, we now have a new designation – Regenerative Advanced Therapy. It is interesting that Regenerative Advanced Therapy can be abbreviated as RAT – which is not a good name.In one of the webinars, Dr Frank Sasinowski questioned the term and suggested the use of Advanced Regenerative Therapy (ART). Nobody wants to have their innovative therapy being labelled as RAT, everybody wants to have their innovative therapy being labelled as ART.

Yesterday, FDA issued its first Regenerative Advanced Therapy designation to Humacyte - a company in Research Triangle Park, North Carolina for their tissue engineered vessel or human acellular vessel. In its issuance, neither RAT nor ART is used by FDA. Instead, the term is called RMAT (Regenerative Medicine Advanced Therapy). I guess that FDA realizes the term RAT is not a good one. 

With RMAT designation, the sponsor will have the breakthrough and priority review benefits as well as one aspect of fast track (rolling submission). In addition, with RMAT designation, the sponsor will have an opportunity for accelerated approval on basis of invalidated surrogate endpoint and the sponsor will have reduced phase 4 requirements if the product is approved. 

Below are some specific sections from 21st Century Cure Act regarding the Regenerative Advanced Therapy designation:

 (g) Regenerative Advanced Therapy.—
 “(1) IN GENERAL.—The Secretary, at the request of the sponsor of a drug, shall facilitate an efficient development program for, and expedite review of, such drug if the drug qualifies as a regenerative advanced therapy under the criteria described in paragraph (2).
 “(2) CRITERIA.—A drug is eligible for designation as a regenerative advanced therapy under this subsection if— 
“(A) the drug is a regenerative medicine therapy (as defined in paragraph (8)); 
“(B) the drug is intended to treat, modify, reverse, or cure a serious or life-threatening disease or condition; and 
“(C) preliminary clinical evidence indicates that the drug has the potential to address unmet medical needs for such a disease or condition.
 “(5) ACTIONS.—The sponsor of a regenerative advanced therapy shall be eligible for the actions to expedite development and review of such therapy under subsection (a)(3)(B), including early interactions to discuss any potential surrogate or intermediate endpoint to be used to support the accelerated approval of an application for the product under subsection (c).
 “(6) ACCESS TO EXPEDITED APPROVAL PATHWAYS.—An application for a regenerative advanced therapy under section 505(b)(1) of this Act or section 351(a) of the Public Health Service Act may be— 
“(A) eligible for priority review, as described in the Manual of Policies and Procedures of the Food and Drug Administration and goals identified in the letters described in section 101(b) of the Prescription Drug User Fee Amendments of 2012; and 
“(B) eligible for accelerated approval under subsection (c), as agreed upon pursuant to subsection (a)(3)(B), through, as appropriate— “(i) surrogate or intermediate endpoints reasonably likely to predict long-term clinical benefit; or “(ii) reliance upon data obtained from a meaningful number of sites, including through expansion to additional sites, as appropriate. 
“(7) POSTAPPROVAL REQUIREMENTS.—The sponsor of a regenerative advanced therapy that is granted accelerated approval and is subject to the postapproval requirements under subsection (c) may, as appropriate, fulfill such requirements, as the Secretary may require, through— 
“(A) the submission of clinical evidence, clinical studies, patient registries, or other sources of real world evidence, such as electronic health records; 
“(B) the collection of larger confirmatory data sets, as agreed upon pursuant to subsection (a)(3)(B); or 
“(C) postapproval monitoring of all patients treated with such therapy prior to approval of the therapy.
 “(8) DEFINITION.—For purposes of this section, the term ‘regenerative medicine therapy’ includes cell therapy, therapeutic tissue engineering products, human cell and tissue products, and combination products using any such therapies or products, except for those regulated solely under section 361 of the Public Health Service Act and part 1271 of title 21, Code of Federal Regulations.”.    
SEC. 3034. GUIDANCE REGARDING DEVICES USED IN THE RECOVERY, ISOLATION, OR DELIVERY OF REGENERATIVE ADVANCED THERAPIES.(a) Draft Guidance.—Not later than 1 year after the date of enactment of the 21st Century Cures Act, the Secretary of Health and Human Services, acting through the Commissioner of Food and Drugs, shall issue draft guidance clarifying how, in the context of regenerative advanced therapies, the Secretary will evaluate devices used in the recovery, isolation, or delivery of regenerative advanced therapies. In doing so, the Secretary shall specifically address— 
(1) how the Food and Drug Administration intends to simplify and streamline regulatory requirements for combination device and cell or tissue products; 
(2) what, if any, intended uses or specific attributes would result in a device used with a regenerative therapy product to be classified as a class III device; 
(3) when the Food and Drug Administration considers it is necessary, if ever, for the intended use of a device to be limited to a specific intended use with only one particular type of cell; and 
(4) application of the least burdensome approach to demonstrate how a device may be used with more than one cell type. (b) Final Guidance.—Not later than 12 months after the close of the period for public comment on the draft guidance under subsection (a), the Secretary of Health and Human Services shall finalize such guidance.  
SEC. 3035. REPORT ON REGENERATIVE ADVANCED THERAPIES.(a) Report To Congress.—Before March 1 of each calendar year, the Secretary of Health and Human Services shall, with respect to the previous calendar year, submit a report to the Committee on Health, Education, Labor, and Pensions of the Senate and the Committee on Energy and Commerce of the House of Representatives on— 
(1) the number and type of applications for approval of regenerative advanced therapies filed, approved or licensed as applicable, withdrawn, or denied; and 
(2) how many of such applications or therapies, as applicable, were granted accelerated approval or priority review. (b) Regenerative Advanced Therapy.—In this section, the term “regenerative advanced therapy” has the meaning given such term in section 506(g) of the Federal Food, Drug, and Cosmetic Act, as added by section 3033 of this Act.  
SEC. 3036. STANDARDS FOR REGENERATIVE MEDICINE AND REGENERATIVE ADVANCED THERAPIES.
Subchapter A of chapter V of the Federal Food, Drug, and Cosmetic Act (21 U.S.C. 351 et seq.) is amended by inserting after section 506F the following: 
“SEC. 506G. STANDARDS FOR REGENERATIVE MEDICINE AND REGENERATIVE ADVANCED THERAPIES. “(a) In General.—Not later than 2 years after the date of enactment of the 21st Century Cures Act, the Secretary, in consultation with the National Institute of Standards and Technology and stakeholders (including regenerative medicine and advanced therapies manufacturers and clinical trial sponsors, contract manufacturers, academic institutions, practicing clinicians, regenerative medicine and advanced therapies industry organizations, and standard setting organizations), shall facilitate an effort to coordinate and prioritize the development of standards and consensus definition of terms, through a public process, to support, through regulatory predictability, the development, evaluation, and review of regenerative medicine therapies and regenerative advanced therapies, including with respect to the manufacturing processes and controls of such products. 
“(b) Activities.— 
“(1) IN GENERAL.—In carrying out this section, the Secretary shall continue to— 
“(A) identity opportunities to help advance the development of regenerative medicine therapies and regenerative advanced therapies; 
“(B) identify opportunities for the development of laboratory regulatory science research and documentary standards that the Secretary determines would help support the development, evaluation, and review of regenerative medicine therapies and regenerative advanced therapies through regulatory predictability; and 
“(C) work with stakeholders, such as those described in subsection (a), as appropriate, in the development of such standards. 
“(2) REGULATIONS AND GUIDANCE.—Not later than 1 year after the development of standards as described in subsection (a), the Secretary shall review relevant regulations and guidance and, through a public process, update such regulations and guidance as the Secretary determines appropriate. “(c) Definitions.—For purposes of this section, the terms ‘regenerative medicine therapy’ and ‘regenerative advanced therapy’ have the meanings given such terms in section 506(g).”.

Monday, March 20, 2017

Minimization Algorithm to Achieve Treatment Balance across Strata in Stratified Randomization

In randomized controlled clinical trials, we usually need to consider if there are any prognostic factors that may have influence on the primary outcome measure. The prognostic factors usually include demographic information (gender, age), severity of the disease (mild, moderate, and severe), use of concomitant medication (patients on background therapy versus patients not on background therapy), genetic sub-type of the disease, biomarker measures at baseline…
Dealing with the prognostic factors can be at the statistical analysis stage through the sub-group analyses to investigate the impact of the prognostic factors on the treatment outcome.
However, if there are factors known to have impact on the treatment outcome, it is better to consider them in the planning stage through the stratified randomization. The prognostic factors are considered as stratified factors. In stratified randomization, the random treatment assignment is actually performed within each strata. I had an old blog article to explain the stratified randomization “Stratified randomization to achieve the balance of treatment assignment within each strata
When we plan for the stratified randomization, the number of stratification factors and number of levels or categories for each stratification factor need to be considered. The total number of strata is the multiplication of the number of levels of stratification factors.
Suppose we just have one stratification factor with two levels (age group: 'less than 65 years old' versus 'greater than and equal to 65 years old'), the total number of strata = 1 x 2 = 2. The randomization (specifically the block randomization) will be implemented within each stratum of 'less than 65 years old' or 'greater than and equal to 65 years old'.
Suppose we have two stratification factors: age group with two levels ('less than 65 years old' versus 'greater than and equal to 65 years old') and disease severity with three levels (mild, moderate, and severe), the total number of strata = 2 x 3 =6. The randomization (specifically the block randomization) will be implemented within each of the following 6 strata (combinations):
  • less than 65 years old and mild severity
  • less than 65 years old and moderate severity
  • less than 65 years old and severe severity
  • greater than and equal to 65 years old and mild severity
  • greater than and equal to 65 years old and moderate severity
  • greater than and equal to 65 years old and severe severity

The question arises when more and more stratification factors are added to the list. How many stratification factors can we have in a study while still maintaining the overall balance in treatment assignment across all of these strata?
The number of strata can increase rapidly. Suppose we have three stratification factors:
  • Age group with two levels ('less than 65 years old' versus 'greater and equal to 65 years old')
  • Disease severity with three levels (mild, moderate, and severe)
  • Geographic region with four categories (Asia, Europe, North America, and South America)

The total number of strata = 2 x 3 x 4=24. The randomization (specifically the block randomization) will be implemented within each of the 24 strata (combinations):
  • less than 65 years old, mild severity, Asia
  • less than 65 years old, moderate severity, Asia
  • less than 65 years old, severe severity, Asia
  • ……

While there is not rule how many stratification factors or strata can be used in a clinical trial, I usually stick with a number to keep the number of stratification factors no more than two or total number of strata no more than 6.
With the limited total sample size, if there are too many stratification factors or too many strata, it can actually cause the imbalance in treatment assignment on the study level. Even though the treatment assignment is intended to be balanced within each stratum by implementing the block randomization, there will be always incomplete blocks across all strata, which causes the imbalance in treatment assignment on the study level.
One approach to address this issue is an approach called ‘minimization’ algorithm. Minimization algorithm is one of the adaptive randomization approaches. It is used in the situation where there is a limited or small overall sample size, but many critical stratification factors.
Here are a couple of screen shots for illustrating how the minimization works.


The minimization requires the calculation after each subject is randomized. It is almost not possible to implement the minimization with the manual process. Luckily, with IRT (Interactive Response Technology) including IVRS (interactive voice response system) and IWRS (interactive web response system), the minimization can be implemented through programming. The algorithm for minimization is built into the IRT system and the calculation after each subject randomization is automatically calculated by the system. The IRT vendors such as conduit, sovuda, Datatrak et al can all do the implementation of the minimization algorithm.

References: 

Thursday, March 16, 2017

Block size in randomization and generating the randomization with variable block size

In clinical trials, the most popular randomization approach is probably the randomized block design. With a randomized block design, study participants (subjects) are to be divided into subgroups called blocks. The balance based on the randomization ratio is then achieved within blocks. In other words, within each block, subjects are randomly assigned to treatment different groups.

The block size must be the multiplier of the sum of the treatment ratio. For example, if the treatment assignment is A: B in 1:1 ratio, the block size must be 2, 4, 6, 8, …
If the treatment assignment is A:B in 2:1 ratio, the block size must be 3, 6, 9, 12,…
If the treatment assignment is A:B:C in 2:2:1 ratio, the block size must be 5, 10, 15…

If a block size of 5 is chosen, it indicates that within each block (every 5 subjects), 2 should randomly assigned to A and 2 should be randomly assigned to B, and 1 should be randomized assigned to C.

If a block size of 10 is chosen, it indicates that within each block (every 10 subjects), 4 should randomly assigned to A and 4 should be randomly assigned to B, and 2 should be randomized assigned to C.

To achieve the treatment balance, the smaller block size is usually chosen if central randomization (not by investigational site) is used. Central randomization is usually implemented through IRT (interactive response technology) such as IVRW (interactive voice response system) or IWRS (interactive web response system).

If the randomization is performed within each site or by site and if a smaller block size is chosen, there could be a risk of potential guess / unblinding if other subjects within the block are unblinded. For example, if the randomization is by site and if a block size of 2 is chosen, once the treatment assignment for one subject within the block is revealed, the treatment for the other subject in that block is automatically revealed. 

To prevent the potential guessing / unblinding, the following approaches may be used:
  • Choose variable block size
  • Do not disclose the block size to the sites

Note that if the randomization is centralized, there is usually not necessary to have variable block size since the randomization is across all sites and the investigator at a specific site will not be able to guess the treatment assignment based on the block size.

Following two papers discussed how to program the randomization schedule with variable block size in SAS: one using ranuni() function and one using Proc Plan.
For generating the randomization schedule with fixed block size, my SUGI paper "Generating Randomization Schedule Using SAS" is still very relevant. 

Sunday, March 05, 2017

Inclusion/exclusion violations, protocol deviations, protocol deviation waiver, rescreening, and others

In one of my previous posts, I discussed "Protocol Deviation versus Protocol Violation and its Classifications (minor, major, critical, important)". There are some more aspects related to the protocol deviations. 

Pre and post randomization protocol deviations

Protocol deviations should be collected throughout the study (beyond only the inclusion /exclusion criteria). If it is a randomized study, we can classify the protocol deviations as pre-randomization and post randomization. If it is non-randomized study, we can classify the protocol deviations as pre-dose and post-dose. 

Prior to the randomization or dosing, the eligibility criteria (inclusion and exclusion criteria) should be verified. If subjects do not meet any of the inclusion and exclusion criteria, but are enrolled into the study, the violation of inclusion/exclusion criteria must be recorded. Some people may use the term 'protocol violation' specifically for those deviations that are related to the inclusion and exclusion criteria. 

Post-randomization protocol deviations can include anything that deviates from the study protocol: study drug compliance, taking prohibited concomitant medication, subject unblinding of treatment assignments, outside the visit window, missing visits, missing procedures,…

According to the SDTM implementation guide, the pre-randomization protocol deviations (i.e., violation of inclusion/exclusion criteria) should be collected in IE domain and the post-randomization protocol deviations need to be collected in DV domain.

Can we allow the protocol deviation waiver? 

A protocol waiver is an intentional deviation from the approved protocol, such as the enrollment of a participant in violation of the protocol’s inclusion/exclusion criteria. Most sites realize that it is necessary to obtain the sponsor’s approval prior to implementing a protocol waiver. It is also necessary, however, to obtain the IRB’s approval prior to implementing a protocol waiver, unless the change is deemed necessary to eliminate an apparent immediate hazard for study participants' safety. In practice, many clinical trial sponsors will allow the protocol deviation waiver and allow the subjects without meeting one or more inclusion / exclusion criteria to be enrolled into the study. Here are some discussions about the waiver for protocol deviations: 

EMA on its website has a specific question about the protocol waivers:
Adherence to the protocol is a fundamental part of the conduct of a clinical study. Any significant change to the protocol should be submitted as an amendment to the competent regulatory authority and ethics committee. Significant changes to the protocol include any change in inclusion and exclusion criteria, addition or deletion of tests, dosing, duration of treatment etc (see the definition of a substantial amendment in the 'detailed guidance for the request for authorisation of a clinical trial on a medicinal product for human use to the competent authorities, notification of substantial amendments and declaration of the end of the trial' published by the European Commission in chapter I, volume 10 of the rules governing medicinal products in the European Community).  Deviations from the inclusion/exclusion criteria of the protocol might erode the scientific and ethical value of the protocol and its authorisation and might have an impact on the processes put in place for the care and safety of the study subjects.
Sponsors and investigators should not use systems of prospectively approving protocol deviations, in order to effectively widen the scope of a protocol.   Protocol design should be appropriate to the populations required and if the protocol design is defective, the protocol should be amended.
GCP does permit deviations from the protocol when necessary to eliminate immediate hazards to the subjects but this should not normally arise in the context of inclusion/exclusion criteria, since the subject is not yet fully included in the trial at that point in the process GCP inspectors have observed a number of sponsors implementing systems where the investigator can contact the sponsor, usually the Medical Monitor, and request a prospective approval to deviate from the inclusion and/or exclusion criteria.  The use of such systematic waiver systems in clinical trials is not considered to be appropriate and studies using such a system might be regarded as non-compliant with GCP.
Can we allow rescreening for subjects who failed one of the inclusion / exclusion criteria?

For some inclusion/exclusion criteria that are based on the laboratory measurements, lung functional test, six-minutes walking test,… subjects may be screening failures due to not meeting one of the criteria. For example, if one of the inclusion criteria requires hemoglobin level must be greater than 9 g/dl, a subject may just miss the criteria (for example, hemoglobin level is 8.9 g/dl) that could be caused by the measurement error. If the protocol specifies that the rescreening is allowed, the subject may come back for a rescreening and have another lab test for hemoglobin level. I have seen many clinical trial protocols that allow the rescreening, especially in the chronic disease clinical trials.

It is obvious that the rescreening will not be feasible for clinical trials in acute diseases for example, in clinical trials in ischemic stroke patients.   

Where to document and maintain the protocol deviations?

The protocol deviations are usually documented by the clinical team (study manager and clinical monitors) and oversight group. In early days, the protocol deviation may just be entered and maintained in an excel spreadsheet. Nowadays, the protocol deviations are usually documented and maintained in CTMS (clinical trial management system).

Can protocol deviations be collected through EDC?

The protocol deviations can also be directly documented and maintained within EDC system where a separate case report form (CRF) is designed specifically for collecting the protocol deviations. Clinical monitors (CRAs) are given the access to enter and maintain the protocol deviation through EDC system. The investigator/study coordinators at sites will not have access to the protocol deviation CRF. 

Site level versus subject level protocol deviations
While majority of the protocol deviations are on the subject level, the protocol deviations can be on the site-level that have impacts on all subjects enrolled at that specific site.

The site level and subject level protocol deviation need to be distinguished in the protocol deviation tracking.

CDISC SDTM implementation guide indicates that the protocol deviations will be captured in DV domain (see the explanations of the DV domain below). Notice that current SDTM standard is designed for subject data with the only exception of Trial Design info. Therefore, the DV domain is only for the subject-level protocol deviations. If there are site level protocol deviation (violations), the suggestion is not to be included in the DV dataset, but included in protocol deviation tracking and described in the clinical study report. It is also a good practice to indicate in Study Data Reviewer’s Guide how the site-level protocol deviations are handled.



How will the protocol deviation information be used?

The protocol deviation information will be converted into the data set as part of the clinical database.

The protocol deviations in DV domain will be provided in data listing and will be summarized (by treatment group and by protocol deviation category). The violation of the inclusion/exclusion criteria may be separately listed and summarized.

In clinical study report, according to ICH E3, section 10.2 is for describing the inclusion / exclusion criteria violation and protocol deviations


Tuesday, February 21, 2017

Constructing stopping rule for safety monitoring

In many early phase clinical trials, if there is a known risk or theoretically potential risk for patients' safety, it is very common that a DSMB (data safety monitoring board) will be established to review the data on a on-going basis. Usually, there is probably a stopping rule for safety - the stopping rule based on the accumulated safety data that is different from the stopping rule for each individual patient.

The stopping rule can be based on the expert's opinion and more commonly can be derived from the statistical standpoint based on the known background rate. Here are some articles about the safety monitoring and the stopping rule for safety:
In fact, an easy way to construct a stopping rule is to utilize the hypothesis testing approach. The idea is to rule out that the observed event rate is higher than the background event rate and is to stop the trial if excessive event rate is observed.

Here is an example from an early phase clinical trial for ischemic stroke where a known risk for the experimental drug is to cause excessive symptomatic intracranial hemorrhage (SICH). The plan is to enroll and dose up to 20 subjects. The literature reviews reveal that the background rate for SICH in ischemic stroke patients is about 10%. We can then construct a stopping rule based on the background rate for SICH of 10%.

The DSMB is empowered to stop the trial whenever they deem fit to protect the safety of patients. DSMB will consider adopting a stopping rule that evaluates the rate of SICH after 5 enrollments, requiring suspension of trial entry if the observed rate of SICH be sufficient to reject the null hypothesis that the true SICH proportion (P) is 10% or less in favor of the (one-sided) alternative hypothesis that the true SCIH proportion is more than 10%. The table below depicts the stopping guidelines. Exact binomial methods are used to compute the p values and confidence limits. One-sided lower 90% confidence limit is calculated.

Number of Subjects Enrolled in the Study
Number of SICH Needed to Reject Null Hypothesis
Observed Rate of SICH (%)
One-sided p values
Lower 90% Confidence Limit
~5
2
40%
0.0815
11.22%
6-10
3
30%
0.0702
11.58%
11-15
4
27%
0.0556
12.18%
16-20
5
25%
0.0432
12.69%

If 2 SICH events are observed in less than and equal to 5 subjects enrolled, the lower confidence limit of 11.22% will be exceeding the 10% background rate. The stopping rule for safety will be triggered and the study should be stopped.

The SAS program for calculating the stopping rule will be something like below. Notice that we obtain two-sided 80% confidence interval in order to obtain one-sided 90% lower limit. Try-and-error method can be employed to find the lowest number of SICHs that will have lower 90% confidence limit exceed the background event rate (10%).

data stopping;
  input scenario SICH $ count;
  datalines;
  1 Have 2
  1 No   3
  2 Have 3
  2 No   7
  3 Have 4
  3 No   11
  4 Have 5
  4 No   15
run;

proc freq data=stopping;
  weight count;
  tables SICH /  binomial (p=0.10) alpha=0.20 cl;
                    **p=0.10 option indicates the background rate to compare with, here we assume the
                        SICH rate of 10%;
                    ** Alpha=0.20 to obtain two-sided 80% confidence interval;
  exact binomial;  *Obtain the exact p-value;
  by scenario;
run;