Visit Window for Longitudinal Studies

In the previous post, the time window was discussed. The time window is an issue in studies with relatively shorter durations (such as the clinical pharmacology studies and clinical trials in analgesic drugs). The measures outside the time window can still be used in analysis by using the actual time in calculations. The impact is relatively neglectable.   

In clinical trials with a longitudinal design where the study subjects are followed up at pre-specified schedules for a long period of time, the visit window is an issue to be handled. The visit schedules may not be followed due to the reasons of 1) patient side (for example, travel arrangement); 2) investigator side (for example, investigator may have other responsibilities and not able to see the patient on a specific day); 3) the study procedure cannot be performed on the specified visit day (for example, the CT, MRI, … may not be arranged at the exact visit date).

It is common that the visit window is allowed and specified in the protocol, in this way, not every out of window visit will be recorded as the protocol deviation. The visit window needs to be protocol specific and needs to specified according to the length of the study and interval of the study visits. A study with every four-week visit schedule may have a visit window +/- 1 week; a study with every six-month visit schedule may have a visit window +/- a month.

It is inevitable to have always some subjects with visit outside the visit window. The study will also allow having unscheduled visits and early termination visit (the early termination visit can occur at any time during the study). These visits and measures outside the visit window will need to be used in the analysis so that the missing data can be minimized, and the more representative measures are selected to be used in the analysis. This needs to be handled through programming (usually in analysis dataset programming or CDISC ADaM dataset programming). Generally, two steps are needed:

Specify the Visit Slotting Algorithm

Based on the actual visit date, allocate the actual measure to the specific visit. An example below is for a six-month (24 weeks exactly) study. The visit slotting algorithm can be specified as the following:

Nominal Visit (Scheduled Visit)	Scheduled Visit Day (Study Day)	Slotting Intervals
Baseline	1	-3 to 1 day
Week 1	8	2 to 11 days
Week 2	15	12 to 22 days
Week 4	29	23 to 43 days
Week 8	57	44 to 71 days
Week 12	85	72 to 99 days
Week 16	113	100 to 127 days
Week 20	141	128 to 155 days
Week 24	169	156 to 14 days after the last treatment dose

Selecting the Value for Analysis 

After all the observations have been slotted based on the algorithm above, it is very possible that there are multiple valid observations for an assessment within an assigned analysis visit. Only one of these observations will be used for summary statistics and analyses. In the ADaM program, this is to determine the ANL01FL (‘Y’ if the observation is selected for analysis)

The observation to be used is determined using the following hierarchy (in decreasing order):

The observation closest to the target study day

The latter observation, if 2 observations are equally close to the target study day

Impact on the Missing Data Imputation

If the missing value needs to be imputed, the imputation should be implemented after the above two steps. For missing values where the last observation carried forward (LOCF) algorithm is applied, it is always the last valid observation on treatment carried forward, even though this might not be the observation obtained by the above hierarchy and used in the summaries by visit window.

Impact on analyses using a mixed model such as MMRM (mixed model repeat measure)

After the slotting and determining the analysis flag (ANL01FL), the mixed model will be based on the data with ANL01FL =’Y’ (i.e., some observations within a visit window may be excluded from the analysis).  Can we use all observations and use the actual visit day in the mixed model analysis? If you try, you may run into the issue that the model does not converge. 

Time Windows for PK Blood Sampling

In the drug development process, especially the early non-clinical or clinical studies, it is critical to understand the DMPK. DMPK, or Drug Metabolism and Pharmacokinetics, is an important part of studies often referred to as ADME (Absorption, Distribution, Metabolism, and Elimination).   

• Absorption (how much and how fast, often referred to as the absorbed fraction or bioavailability)
• Distribution (where the drug is distributed, how fast and how extensive)
• Metabolism (how fast, what mechanism/route, what metabolite is formed, and whether they are active or toxic)
• Elimination (how fast, which route)

In clinical pharmacology studies, the purpose of the study is to characterize the pharmacokinetics profile to assess the bioavailability and bioequivalence. In clinical pharmacology studies, usually, the series of blood samples will be collected for measuring the concentration of the study drug and its metabolites. Based on the measured concentrations from the series blood samples, pharmacokinetics parameters can be calculated. The common pharmacokinetics parameters are AUC (area under the time-concentration curve), Cmax (maximum concentration), Tmax (time to maximum concentration), T1/2 (half life),…

When designing a clinical pharmacology study, it is critical to decide and select adequate time points for blood sampling. The ideal blood sampling scheme will include the time points close to the Tmax (so that we can get a good estimate for Cmax and Tmax) and have good spaced time points to characterize the elimination phase (so that we can get a good estimate for T1/2).

From the practicality standpoint, it is not easy to draw the blood samples at the exact time according to the sampling scheme specified in the study protocol. For the phase I study using healthy volunteers at confined clinical research unit, the time windows can be easily controlled. However, for PK studies in patients across many investigational sites, it was very difficult to keep all blood draws within the narrow windows.

It is very common that time windows are allowed for these blood sampling time points. For example, for sampling time at 1 hour (60 minutes) after the study drug administration, we may add a time window to allow the 1-hour sample to be drawn any time between 55 – 65 minutes after the study drug administration, denotes as 60 +/- 5 minutes. A time window of 5 minutes is allowed for this time point. If the blood drawn is outside the time window (for example outside 55 - 65 minutes), a protocol deviation will be recorded. If the data entry is through an EDC (electronic data capture) system, the edit checks are usually built in to track the deviations for sampling time outside windows.

There are some articles arguing the necessity of the time windows.

• Using Sampling “Windows” for PK Blood Samples
• Should you Include a +/- Time Window Around Collection of PK/PD Samples in a Clinical Trial?

The arguments are 1) when we calculate the pharmacokinetic parameters, the actual sampling times are used. The deviations from the nominal time or pre-specified time point have little impact on the calculation of the PK parameters. 2) if the time windows are set too tight and any outside time windows are recorded as protocol deviation, there will be a lot of unnecessary protocol deviations being recorded. If any outside time window is flagged/alerted during the data entry in EDC, it can be very annoying to the data entry person.

My experience is that we can keep the time windows in the study protocol, but not set the time windows too tight. The sites usually need to have some instructions about the time windows, not purely for the PK parameter calculations, but more for operations. The time windows can be set up as a suggestion and we can use the word ‘should’, not ‘must’ for following the time windows. In this way, not any out of window blood drawn will be automatically recorded as a protocol deviation. 

FDA launches new digital tool to help capture real world data from patients to help inform regulatory decision-making

FDA In Brief: FDA launches new digital tool to help capture real world data from patients to help inform regulatory decision-making

FDA’s MyStudies Application (App)

“There are a lot of new ways that we can use real world evidence to help inform regulatory decisions around medical products as the collection of this data gets more widespread and reliable. Better capture of real world data, collected from a variety of sources, has the potential to make our new drug development process more efficient, improve safety and help lower the cost of product development. If done right, it can also help make sure doctors and patients are better informed about the clinical use of new products, enabling them to make more effective, efficient medical choices. This will ultimately help us achieve better outcomes, and safer and more efficient use of expensive technology,” said FDA Commissioner Scott Gottlieb, M.D. “Today we’re unveiling the new MyStudies app for the collection of real world data. This digital platform enables developers to adapt our technology to advance new ways to access and use data collected directly from patients—with the necessary controls in place to ensure patient privacy. Our hope is that the collection of more real world data directly from patients, using a secure app, will lead to more efficient product development and assist with safety monitoring.”

Today the U.S. Food and Drug Administration is announcing the MyStudies app, a new mobile technology to foster the collection of real world evidence via patients’ mobile devices. Real world data can be collected from a variety of sources, such as electronic health records, claims and billing activities, and product and disease registries, as well as from patient-generated data including in home-use settings, or from data gathered from other sources, such as mobile devices.

As part of the agency’s work to foster greater opportunities around real world evidence, the FDA partnered with Kaiser Permanente on a pilot study to measure the functionality and engagement of the MyStudies app. Based on the successful outcome of the study, the FDA is now releasing the open source code and technical documents that will allow researchers and developers to customize and use the FDA’s newly created MyStudies app to expand the diversity of health information available for clinical trials and studies, while directly capturing the perspective of patients. By providing the open source code, the agency is providing a tool that sponsors and developers can adapt to advance their specific clinical trial and real world evidence needs, while also remaining compliant with the FDA’s regulations and guidance for data authenticity, integrity and confidentiality.

For example, patients may be able to securely enroll in and contribute data to traditional clinical trials, pragmatic trials, observational studies and registries. Sponsors may be able to customize their apps to administer questionnaires assessing patient-reported outcomes, symptom scales or patient reports of prescription and over-the-counter medication use.

The overall effort was led by David Martin, M.D., associate director for real world evidence in the Office of Medical Policy in the FDA’s Center for Drug Evaluation and Research, with a grant from the U.S. Department of Health and Human Services’ Patient Centered Outcomes Research Trust Fund. The open source code that serves as the foundation of the MyStudies app, as well as specifications for a secure patient data storage environment, were developed through a collaboration with Harvard Pilgrim Health Care Institute, LabKey and Boston Technology Corporation

Fitting Compartment Models Using PROC NLMIXED

There is a new article on SAS's website "Fitting Compartment Models Using PROC NLMIXED". It is nice to know that SAS Proc NLMIXED can be used for fitting the compartment model in pharmacokinetic (PK) analyses.

Usually, for PK analyses, the non-compartment model will be used - i.e., Cmax, Tmax,... are based on the observed values and AUC is calculated using the trapezoid rule. I would say that for more than 90% of times, the non-compartment model will be good enough for the PK analyses.

However, in some situations, the non-compartment model or non-compartment analysis (NCA) will not be adequate. One-compartment model with first-order elimination and two-compartment model with first-order distribution and elimination processes may be more appropriate.

For PK analyses using NCA, the most commonly used software is Phoenix WinNonLin that was developed by Pharsight and now owned by Certara. For PK analyses based on compartment models, the most commonly used software is NONMEM that was developed by UCSF and now managed by ICON plc.

Potential Unblinding due to Imbalance in Adverse Event Profiles

In a previous post “Is blinded study really blinded? - assessment of blinding/unblinding in clinical trials”, we discussed the potential unblinding if the subjects on different treatment groups experienced a different type of adverse events. For the ethic reasons, the informed consent form needs to list all the potential side effects of the experimental drug. As required by GCP, the investigator’s brochure needs to discuss the details of the potential side effects caused by the experimental drug. If there is an obvious imbalance in adverse event profiles between the two treatment groups (experiment drug versus control), it is possible that the study blinding is not well maintained and the patient or the investigator can guess or predict based on the AE profile which treatment group the patient is on.

This potential unblinding issue can be even more evident if the crossover design is employed where the same subject will have an chance to experience two different drugs. If there are differences in side effects, the subjects will be likely to feel the difference.

In FDA’s recent guidance for industry “Hematologic Malignancy and Oncologic Disease: Considerations for Use of Placebos and Blinding in Randomized Controlled Clinical Trials for Drug Product Development”, the potential unblinding issue was brought up.

“In many cases, because of the toxicity profile of the active treatment, patients and investigators may infer which treatment patients are receiving and thus use of a placebo control may not, in fact, blind the treatment. …”

The guidance also cited the ethic issue for maintaining blinding when the unblinding is necessary to give the best care for the subjects (patients).

“Continued blinding of patients and investigators at the time of disease progression or occurrence of serious adverse events presents additional challenges. For example, in a blinded immunotherapy trial, a patient who develops adverse events on the control arm may receive unnecessary treatments (e.g., immunosuppressive drug products including a high dose of glucocorticoids, cyclophosphamide, interleukin-6 antagonist, or infliximab) for management of adverse events incorrectly attributed to the investigational drug product. Maintaining the blind after disease progression could also affect a patient’s subsequent therapy, potentially preventing a patient who had been on a placebo arm from receiving an approved therapy, or delaying or preventing the patient’s entry into other clinical trials (for those trials of similar drug products that may have specific exclusion criteria based on prior treatment with an active drug or class of drugs). Unblinding would allow informed decision-making with respect to additional treatment 61 options (see below). “

In an analysis by Shah et al Adverse events appear to unblind clinical trials in irritable bowel syndrome, the result “suggests that higher AE incidence on active therapy is associated with more beneficial patient‐reported outcomes in IBS clinical trials. This raises the issue of spontaneous unblinding”

Maintaining the blinding is one of the cornerstones of the RCT (randomized, controlled trial). As mentioned in an article by Stefan Dürr Avoid Unintentional Unblinding In Clinical Trials

“The knowledge of the patient’s treatment could potentially lead to conscious or unconscious bias in the way the site staff recruits the patients (selection bias), how they are treated (performance bias), the assessment of endpoints (detection bias), the handling of withdrawals, and how data is excluded from analysis. The patient attitude to the treatment if known can lead to a difference in reporting symptoms (response bias) or withdrawal from the study (attrition bias). To reduce the chance of bias to the data, clinical trials should be conducted in a double-blind design whenever possible.”

However, in some situation especially when there is an imbalance in the side effects of two treatment groups, blinding may either be difficult to maintain or not be ethical to implement, and alternative approaches are needed. As suggested in FDA's guidance "

“Hematologic Malignancy and Oncologic Disease: Considerations for Use of Placebos and Blinding in Randomized Controlled Clinical Trials for Drug Product Development”, an open-label study or an add-on trial design may be more appropriate. In an early post, I also discussed some blinding techniques. 

Latin Squares for Constructing "Williams Designs" - Balanced for First-order Carry-over (Residual) Effects

Courtesy from the website http://statpages.info/latinsq.html, the list below contains the Williams design with 2 to 26 treatment groups. It is unlikely to have a clinical trial with 26 treatment groups. The most number of treatment groups I see is a study with 6 treatment groups. 

In one of my previous posts, I described the Williams design and its usefulness in phase I or clinical pharmacology studies. In a SAS SUGI paper "Generating Randomization Schedules Using SAS Programming", I described how to use SAS program to construct the randomization schedule for a study with Williams design. 

With the limited sample size, the Williams design is the most efficient design and minimizes the potential impact of the carryover effect. In Williams design with an even number of n treatment groups, there will be n number of sequences or an "n by n" square. Each treatment only occurs one time in one sequence, in one period. Furthermore, each treatment only follows another treatment one time. 

If the number of treatments (n) is an odd number, there will be 2 x n number of sequences or two n by n squares. Each treatment only occurs once in one sequence and in two periods. Each treatment only follows another treatment twice. 

With the Latin Square listed below, we can easily construct the crossover design with treatments, periods, and sequences. For example, a study design with 3 treatment groups will have the following assignments: three treatment groups (A, B, C), three periods (period 1, period 2, and period 3), and six sequences (ABC, BCA, CAB, CBA, ACB, and BAC). 

Treatment Sequence	Period 1	Period 2	Period 3
1	A	B	C
2	B	C	A
3	C	A	B
4	C	B	A
5	A	C	B
6	B	A	C

• Since it is the crossover design, there should be a washout period between each period. It is very common that the washout period is 5 times the half-life. 
• Randomization is for the sequence, not the treatment group. 
• Once we know the number of subjects is needed for each sequence, we will know the total number of subjects for the entire study. 
• Usually, the mixed model is needed for statistical analysis. The statistical model will need to consider the treatment, period, and sequence. An example of the SAS codes for analysis was provided in one of the previous posts "Cookbook SAS Codes for Bioequivalence Test in 2x2x2 Crossover Design"

Williams Design by the Number of Treatment Groups:

Size = 2

A B 
B A 
 

Size = 3

A B C 
B C A 
C A B 
C B A 
A C B 
B A C 
 

Size = 4

A B D C 
B C A D 
C D B A 
D A C B 
 

Size = 5

A B E C D 
B C A D E 
C D B E A 
D E C A B 
E A D B C 
D C E B A 
E D A C B 
A E B D C 
B A C E D 
C B D A E 
 

Size = 6

A B F C E D 
B C A D F E 
C D B E A F 
D E C F B A 
E F D A C B 
F A E B D C 
 

Size = 7

A B G C F D E 
B C A D G E F 
C D B E A F G 
D E C F B G A 
E F D G C A B 
F G E A D B C 
G A F B E C D 
E D F C G B A 
F E G D A C B 
G F A E B D C 
A G B F C E D 
B A C G D F E 
C B D A E G F 
D C E B F A G 
 

Size = 8

A B H C G D F E 
B C A D H E G F 
C D B E A F H G 
D E C F B G A H 
E F D G C H B A 
F G E H D A C B 
G H F A E B D C 
H A G B F C E D 
 

Size = 9

A B I C H D G E F 
B C A D I E H F G 
C D B E A F I G H 
D E C F B G A H I 
E F D G C H B I A 
F G E H D I C A B 
G H F I E A D B C 
H I G A F B E C D 
I A H B G C F D E 
F E G D H C I B A 
G F H E I D A C B 
H G I F A E B D C 
I H A G B F C E D 
A I B H C G D F E 
B A C I D H E G F 
C B D A E I F H G 
D C E B F A G I H 
E D F C G B H A I 
 

Size = 10

A B J C I D H E G F 
B C A D J E I F H G 
C D B E A F J G I H 
D E C F B G A H J I 
E F D G C H B I A J 
F G E H D I C J B A 
G H F I E J D A C B 
H I G J F A E B D C 
I J H A G B F C E D 
J A I B H C G D F E 
 
 
Size = 11

A B K C J D I E H F G 
B C A D K E J F I G H 
C D B E A F K G J H I 
D E C F B G A H K I J 
E F D G C H B I A J K 
F G E H D I C J B K A 
G H F I E J D K C A B 
H I G J F K E A D B C 
I J H K G A F B E C D 
J K I A H B G C F D E 
K A J B I C H D G E F 
G F H E I D J C K B A 
H G I F J E K D A C B 
I H J G K F A E B D C 
J I K H A G B F C E D 
K J A I B H C G D F E 
A K B J C I D H E G F 
B A C K D J E I F H G 
C B D A E K F J G I H 
D C E B F A G K H J I 
E D F C G B H A I K J 
F E G D H C I B J A K 
 

Size = 12

A B L C K D J E I F H G 
B C A D L E K F J G I H 
C D B E A F L G K H J I 
D E C F B G A H L I K J 
E F D G C H B I A J L K 
F G E H D I C J B K A L 
G H F I E J D K C L B A 
H I G J F K E L D A C B 
I J H K G L F A E B D C 
J K I L H A G B F C E D 
K L J A I B H C G D F E 
L A K B J C I D H E G F 
 
 
Size = 13

A B M C L D K E J F I G H 
B C A D M E L F K G J H I 
C D B E A F M G L H K I J 
D E C F B G A H M I L J K 
E F D G C H B I A J M K L 
F G E H D I C J B K A L M 
G H F I E J D K C L B M A 
H I G J F K E L D M C A B 
I J H K G L F M E A D B C 
J K I L H M G A F B E C D 
K L J M I A H B G C F D E 
L M K A J B I C H D G E F 
M A L B K C J D I E H F G 
H G I F J E K D L C M B A 
I H J G K F L E M D A C B 
J I K H L G M F A E B D C 
K J L I M H A G B F C E D 
L K M J A I B H C G D F E 
M L A K B J C I D H E G F 
A M B L C K D J E I F H G 
B A C M D L E K F J G I H 
C B D A E M F L G K H J I 
D C E B F A G M H L I K J 
E D F C G B H A I M J L K 
F E G D H C I B J A K M L 
G F H E I D J C K B L A M 
 
 
Size = 14

A B N C M D L E K F J G I H 
B C A D N E M F L G K H J I 
C D B E A F N G M H L I K J 
D E C F B G A H N I M J L K 
E F D G C H B I A J N K M L 
F G E H D I C J B K A L N M 
G H F I E J D K C L B M A N 
H I G J F K E L D M C N B A 
I J H K G L F M E N D A C B 
J K I L H M G N F A E B D C 
K L J M I N H A G B F C E D 
L M K N J A I B H C G D F E 
M N L A K B J C I D H E G F 
N A M B L C K D J E I F H G 
 
 
Size = 15

A B O C N D M E L F K G J H I 
B C A D O E N F M G L H K I J 
C D B E A F O G N H M I L J K 
D E C F B G A H O I N J M K L 
E F D G C H B I A J O K N L M 
F G E H D I C J B K A L O M N 
G H F I E J D K C L B M A N O 
H I G J F K E L D M C N B O A 
I J H K G L F M E N D O C A B 
J K I L H M G N F O E A D B C 
K L J M I N H O G A F B E C D 
L M K N J O I A H B G C F D E 
M N L O K A J B I C H D G E F 
N O M A L B K C J D I E H F G 
O A N B M C L D K E J F I G H 
I H J G K F L E M D N C O B A 
J I K H L G M F N E O D A C B 
K J L I M H N G O F A E B D C 
L K M J N I O H A G B F C E D 
M L N K O J A I B H C G D F E 
N M O L A K B J C I D H E G F 
O N A M B L C K D J E I F H G 
A O B N C M D L E K F J G I H 
B A C O D N E M F L G K H J I 
C B D A E O F N G M H L I K J 
D C E B F A G O H N I M J L K 
E D F C G B H A I O J N K M L 
F E G D H C I B J A K O L N M 
G F H E I D J C K B L A M O N 
H G I F J E K D L C M B N A O 
 

Size = 16

A B P C O D N E M F L G K H J I 
B C A D P E O F N G M H L I K J 
C D B E A F P G O H N I M J L K 
D E C F B G A H P I O J N K M L 
E F D G C H B I A J P K O L N M 
F G E H D I C J B K A L P M O N 
G H F I E J D K C L B M A N P O 
H I G J F K E L D M C N B O A P 
I J H K G L F M E N D O C P B A 
J K I L H M G N F O E P D A C B 
K L J M I N H O G P F A E B D C 
L M K N J O I P H A G B F C E D 
M N L O K P J A I B H C G D F E 
N O M P L A K B J C I D H E G F 
O P N A M B L C K D J E I F H G 
P A O B N C M D L E K F J G I H 
 

Size = 17

A B Q C P D O E N F M G L H K I J 
B C A D Q E P F O G N H M I L J K 
C D B E A F Q G P H O I N J M K L 
D E C F B G A H Q I P J O K N L M 
E F D G C H B I A J Q K P L O M N 
F G E H D I C J B K A L Q M P N O 
G H F I E J D K C L B M A N Q O P 
H I G J F K E L D M C N B O A P Q 
I J H K G L F M E N D O C P B Q A 
J K I L H M G N F O E P D Q C A B 
K L J M I N H O G P F Q E A D B C 
L M K N J O I P H Q G A F B E C D 
M N L O K P J Q I A H B G C F D E 
N O M P L Q K A J B I C H D G E F 
O P N Q M A L B K C J D I E H F G 
P Q O A N B M C L D K E J F I G H 
Q A P B O C N D M E L F K G J H I 
J I K H L G M F N E O D P C Q B A 
K J L I M H N G O F P E Q D A C B 
L K M J N I O H P G Q F A E B D C 
M L N K O J P I Q H A G B F C E D 
N M O L P K Q J A I B H C G D F E 
O N P M Q L A K B J C I D H E G F 
P O Q N A M B L C K D J E I F H G 
Q P A O B N C M D L E K F J G I H 
A Q B P C O D N E M F L G K H J I 
B A C Q D P E O F N G M H L I K J 
C B D A E Q F P G O H N I M J L K 
D C E B F A G Q H P I O J N K M L 
E D F C G B H A I Q J P K O L N M 
F E G D H C I B J A K Q L P M O N 
G F H E I D J C K B L A M Q N P O 
H G I F J E K D L C M B N A O Q P 
I H J G K F L E M D N C O B P A Q 
 
 

Size = 18

A B R C Q D P E O F N G M H L I K J 
B C A D R E Q F P G O H N I M J L K 
C D B E A F R G Q H P I O J N K M L 
D E C F B G A H R I Q J P K O L N M 
E F D G C H B I A J R K Q L P M O N 
F G E H D I C J B K A L R M Q N P O 
G H F I E J D K C L B M A N R O Q P 
H I G J F K E L D M C N B O A P R Q 
I J H K G L F M E N D O C P B Q A R 
J K I L H M G N F O E P D Q C R B A 
K L J M I N H O G P F Q E R D A C B 
L M K N J O I P H Q G R F A E B D C 
M N L O K P J Q I R H A G B F C E D 
N O M P L Q K R J A I B H C G D F E 
O P N Q M R L A K B J C I D H E G F 
P Q O R N A M B L C K D J E I F H G 
Q R P A O B N C M D L E K F J G I H 
R A Q B P C O D N E M F L G K H J I 
 

Size = 19

A B S C R D Q E P F O G N H M I L J K 
B C A D S E R F Q G P H O I N J M K L 
C D B E A F S G R H Q I P J O K N L M 
D E C F B G A H S I R J Q K P L O M N 
E F D G C H B I A J S K R L Q M P N O 
F G E H D I C J B K A L S M R N Q O P 
G H F I E J D K C L B M A N S O R P Q 
H I G J F K E L D M C N B O A P S Q R 
I J H K G L F M E N D O C P B Q A R S 
J K I L H M G N F O E P D Q C R B S A 
K L J M I N H O G P F Q E R D S C A B 
L M K N J O I P H Q G R F S E A D B C 
M N L O K P J Q I R H S G A F B E C D 
N O M P L Q K R J S I A H B G C F D E 
O P N Q M R L S K A J B I C H D G E F 
P Q O R N S M A L B K C J D I E H F G 
Q R P S O A N B M C L D K E J F I G H 
R S Q A P B O C N D M E L F K G J H I 
S A R B Q C P D O E N F M G L H K I J 
K J L I M H N G O F P E Q D R C S B A 
L K M J N I O H P G Q F R E S D A C B 
M L N K O J P I Q H R G S F A E B D C 
N M O L P K Q J R I S H A G B F C E D 
O N P M Q L R K S J A I B H C G D F E 
P O Q N R M S L A K B J C I D H E G F 
Q P R O S N A M B L C K D J E I F H G 
R Q S P A O B N C M D L E K F J G I H 
S R A Q B P C O D N E M F L G K H J I 
A S B R C Q D P E O F N G M H L I K J 
B A C S D R E Q F P G O H N I M J L K 
C B D A E S F R G Q H P I O J N K M L 
D C E B F A G S H R I Q J P K O L N M 
E D F C G B H A I S J R K Q L P M O N 
F E G D H C I B J A K S L R M Q N P O 
G F H E I D J C K B L A M S N R O Q P 
H G I F J E K D L C M B N A O S P R Q 
I H J G K F L E M D N C O B P A Q S R 
J I K H L G M F N E O D P C Q B R A S 
 

Size = 20

A B T C S D R E Q F P G O H N I M J L K 
B C A D T E S F R G Q H P I O J N K M L 
C D B E A F T G S H R I Q J P K O L N M 
D E C F B G A H T I S J R K Q L P M O N 
E F D G C H B I A J T K S L R M Q N P O 
F G E H D I C J B K A L T M S N R O Q P 
G H F I E J D K C L B M A N T O S P R Q 
H I G J F K E L D M C N B O A P T Q S R 
I J H K G L F M E N D O C P B Q A R T S 
J K I L H M G N F O E P D Q C R B S A T 
K L J M I N H O G P F Q E R D S C T B A 
L M K N J O I P H Q G R F S E T D A C B 
M N L O K P J Q I R H S G T F A E B D C 
N O M P L Q K R J S I T H A G B F C E D 
O P N Q M R L S K T J A I B H C G D F E 
P Q O R N S M T L A K B J C I D H E G F 
Q R P S O T N A M B L C K D J E I F H G 
R S Q T P A O B N C M D L E K F J G I H 
S T R A Q B P C O D N E M F L G K H J I 
T A S B R C Q D P E O F N G M H L I K J 
 

Size = 21

A B U C T D S E R F Q G P H O I N J M K L 
B C A D U E T F S G R H Q I P J O K N L M 
C D B E A F U G T H S I R J Q K P L O M N 
D E C F B G A H U I T J S K R L Q M P N O 
E F D G C H B I A J U K T L S M R N Q O P 
F G E H D I C J B K A L U M T N S O R P Q 
G H F I E J D K C L B M A N U O T P S Q R 
H I G J F K E L D M C N B O A P U Q T R S 
I J H K G L F M E N D O C P B Q A R U S T 
J K I L H M G N F O E P D Q C R B S A T U 
K L J M I N H O G P F Q E R D S C T B U A 
L M K N J O I P H Q G R F S E T D U C A B 
M N L O K P J Q I R H S G T F U E A D B C 
N O M P L Q K R J S I T H U G A F B E C D 
O P N Q M R L S K T J U I A H B G C F D E 
P Q O R N S M T L U K A J B I C H D G E F 
Q R P S O T N U M A L B K C J D I E H F G 
R S Q T P U O A N B M C L D K E J F I G H 
S T R U Q A P B O C N D M E L F K G J H I 
T U S A R B Q C P D O E N F M G L H K I J 
U A T B S C R D Q E P F O G N H M I L J K 
L K M J N I O H P G Q F R E S D T C U B A 
M L N K O J P I Q H R G S F T E U D A C B 
N M O L P K Q J R I S H T G U F A E B D C 
O N P M Q L R K S J T I U H A G B F C E D 
P O Q N R M S L T K U J A I B H C G D F E 
Q P R O S N T M U L A K B J C I D H E G F 
R Q S P T O U N A M B L C K D J E I F H G 
S R T Q U P A O B N C M D L E K F J G I H 
T S U R A Q B P C O D N E M F L G K H J I 
U T A S B R C Q D P E O F N G M H L I K J 
A U B T C S D R E Q F P G O H N I M J L K 
B A C U D T E S F R G Q H P I O J N K M L 
C B D A E U F T G S H R I Q J P K O L N M 
D C E B F A G U H T I S J R K Q L P M O N 
E D F C G B H A I U J T K S L R M Q N P O 
F E G D H C I B J A K U L T M S N R O Q P 
G F H E I D J C K B L A M U N T O S P R Q 
H G I F J E K D L C M B N A O U P T Q S R 
I H J G K F L E M D N C O B P A Q U R T S 
J I K H L G M F N E O D P C Q B R A S U T 
K J L I M H N G O F P E Q D R C S B T A U 
 


Size = 22

A B V C U D T E S F R G Q H P I O J N K M L 
B C A D V E U F T G S H R I Q J P K O L N M 
C D B E A F V G U H T I S J R K Q L P M O N 
D E C F B G A H V I U J T K S L R M Q N P O 
E F D G C H B I A J V K U L T M S N R O Q P 
F G E H D I C J B K A L V M U N T O S P R Q 
G H F I E J D K C L B M A N V O U P T Q S R 
H I G J F K E L D M C N B O A P V Q U R T S 
I J H K G L F M E N D O C P B Q A R V S U T 
J K I L H M G N F O E P D Q C R B S A T V U 
K L J M I N H O G P F Q E R D S C T B U A V 
L M K N J O I P H Q G R F S E T D U C V B A 
M N L O K P J Q I R H S G T F U E V D A C B 
N O M P L Q K R J S I T H U G V F A E B D C 
O P N Q M R L S K T J U I V H A G B F C E D 
P Q O R N S M T L U K V J A I B H C G D F E 
Q R P S O T N U M V L A K B J C I D H E G F 
R S Q T P U O V N A M B L C K D J E I F H G 
S T R U Q V P A O B N C M D L E K F J G I H 
T U S V R A Q B P C O D N E M F L G K H J I 
U V T A S B R C Q D P E O F N G M H L I K J 
V A U B T C S D R E Q F P G O H N I M J L K 
 


Size = 23

A B W C V D U E T F S G R H Q I P J O K N L M 
B C A D W E V F U G T H S I R J Q K P L O M N 
C D B E A F W G V H U I T J S K R L Q M P N O 
D E C F B G A H W I V J U K T L S M R N Q O P 
E F D G C H B I A J W K V L U M T N S O R P Q 
F G E H D I C J B K A L W M V N U O T P S Q R 
G H F I E J D K C L B M A N W O V P U Q T R S 
H I G J F K E L D M C N B O A P W Q V R U S T 
I J H K G L F M E N D O C P B Q A R W S V T U 
J K I L H M G N F O E P D Q C R B S A T W U V 
K L J M I N H O G P F Q E R D S C T B U A V W 
L M K N J O I P H Q G R F S E T D U C V B W A 
M N L O K P J Q I R H S G T F U E V D W C A B 
N O M P L Q K R J S I T H U G V F W E A D B C 
O P N Q M R L S K T J U I V H W G A F B E C D 
P Q O R N S M T L U K V J W I A H B G C F D E 
Q R P S O T N U M V L W K A J B I C H D G E F 
R S Q T P U O V N W M A L B K C J D I E H F G 
S T R U Q V P W O A N B M C L D K E J F I G H 
T U S V R W Q A P B O C N D M E L F K G J H I 
U V T W S A R B Q C P D O E N F M G L H K I J 
V W U A T B S C R D Q E P F O G N H M I L J K 
W A V B U C T D S E R F Q G P H O I N J M K L 
M L N K O J P I Q H R G S F T E U D V C W B A 
N M O L P K Q J R I S H T G U F V E W D A C B 
O N P M Q L R K S J T I U H V G W F A E B D C 
P O Q N R M S L T K U J V I W H A G B F C E D 
Q P R O S N T M U L V K W J A I B H C G D F E 
R Q S P T O U N V M W L A K B J C I D H E G F 
S R T Q U P V O W N A M B L C K D J E I F H G 
T S U R V Q W P A O B N C M D L E K F J G I H 
U T V S W R A Q B P C O D N E M F L G K H J I 
V U W T A S B R C Q D P E O F N G M H L I K J 
W V A U B T C S D R E Q F P G O H N I M J L K 
A W B V C U D T E S F R G Q H P I O J N K M L 
B A C W D V E U F T G S H R I Q J P K O L N M 
C B D A E W F V G U H T I S J R K Q L P M O N 
D C E B F A G W H V I U J T K S L R M Q N P O 
E D F C G B H A I W J V K U L T M S N R O Q P 
F E G D H C I B J A K W L V M U N T O S P R Q 
G F H E I D J C K B L A M W N V O U P T Q S R 
H G I F J E K D L C M B N A O W P V Q U R T S 
I H J G K F L E M D N C O B P A Q W R V S U T 
J I K H L G M F N E O D P C Q B R A S W T V U 
K J L I M H N G O F P E Q D R C S B T A U W V 
L K M J N I O H P G Q F R E S D T C U B V A W 
 

Size = 24

A B X C W D V E U F T G S H R I Q J P K O L N M 
B C A D X E W F V G U H T I S J R K Q L P M O N 
C D B E A F X G W H V I U J T K S L R M Q N P O 
D E C F B G A H X I W J V K U L T M S N R O Q P 
E F D G C H B I A J X K W L V M U N T O S P R Q 
F G E H D I C J B K A L X M W N V O U P T Q S R 
G H F I E J D K C L B M A N X O W P V Q U R T S 
H I G J F K E L D M C N B O A P X Q W R V S U T 
I J H K G L F M E N D O C P B Q A R X S W T V U 
J K I L H M G N F O E P D Q C R B S A T X U W V 
K L J M I N H O G P F Q E R D S C T B U A V X W 
L M K N J O I P H Q G R F S E T D U C V B W A X 
M N L O K P J Q I R H S G T F U E V D W C X B A 
N O M P L Q K R J S I T H U G V F W E X D A C B 
O P N Q M R L S K T J U I V H W G X F A E B D C 
P Q O R N S M T L U K V J W I X H A G B F C E D 
Q R P S O T N U M V L W K X J A I B H C G D F E 
R S Q T P U O V N W M X L A K B J C I D H E G F 
S T R U Q V P W O X N A M B L C K D J E I F H G 
T U S V R W Q X P A O B N C M D L E K F J G I H 
U V T W S X R A Q B P C O D N E M F L G K H J I 
V W U X T A S B R C Q D P E O F N G M H L I K J 
W X V A U B T C S D R E Q F P G O H N I M J L K 
X A W B V C U D T E S F R G Q H P I O J N K M L 
 
 

Size = 25

A B Y C X D W E V F U G T H S I R J Q K P L O M N 
B C A D Y E X F W G V H U I T J S K R L Q M P N O 
C D B E A F Y G X H W I V J U K T L S M R N Q O P 
D E C F B G A H Y I X J W K V L U M T N S O R P Q 
E F D G C H B I A J Y K X L W M V N U O T P S Q R 
F G E H D I C J B K A L Y M X N W O V P U Q T R S 
G H F I E J D K C L B M A N Y O X P W Q V R U S T 
H I G J F K E L D M C N B O A P Y Q X R W S V T U 
I J H K G L F M E N D O C P B Q A R Y S X T W U V 
J K I L H M G N F O E P D Q C R B S A T Y U X V W 
K L J M I N H O G P F Q E R D S C T B U A V Y W X 
L M K N J O I P H Q G R F S E T D U C V B W A X Y 
M N L O K P J Q I R H S G T F U E V D W C X B Y A 
N O M P L Q K R J S I T H U G V F W E X D Y C A B 
O P N Q M R L S K T J U I V H W G X F Y E A D B C 
P Q O R N S M T L U K V J W I X H Y G A F B E C D 
Q R P S O T N U M V L W K X J Y I A H B G C F D E 
R S Q T P U O V N W M X L Y K A J B I C H D G E F 
S T R U Q V P W O X N Y M A L B K C J D I E H F G 
T U S V R W Q X P Y O A N B M C L D K E J F I G H 
U V T W S X R Y Q A P B O C N D M E L F K G J H I 
V W U X T Y S A R B Q C P D O E N F M G L H K I J 
W X V Y U A T B S C R D Q E P F O G N H M I L J K 
X Y W A V B U C T D S E R F Q G P H O I N J M K L 
Y A X B W C V D U E T F S G R H Q I P J O K N L M 
N M O L P K Q J R I S H T G U F V E W D X C Y B A 
O N P M Q L R K S J T I U H V G W F X E Y D A C B 
P O Q N R M S L T K U J V I W H X G Y F A E B D C 
Q P R O S N T M U L V K W J X I Y H A G B F C E D 
R Q S P T O U N V M W L X K Y J A I B H C G D F E 
S R T Q U P V O W N X M Y L A K B J C I D H E G F 
T S U R V Q W P X O Y N A M B L C K D J E I F H G 
U T V S W R X Q Y P A O B N C M D L E K F J G I H 
V U W T X S Y R A Q B P C O D N E M F L G K H J I 
W V X U Y T A S B R C Q D P E O F N G M H L I K J 
X W Y V A U B T C S D R E Q F P G O H N I M J L K 
Y X A W B V C U D T E S F R G Q H P I O J N K M L 
A Y B X C W D V E U F T G S H R I Q J P K O L N M 
B A C Y D X E W F V G U H T I S J R K Q L P M O N 
C B D A E Y F X G W H V I U J T K S L R M Q N P O 
D C E B F A G Y H X I W J V K U L T M S N R O Q P 
E D F C G B H A I Y J X K W L V M U N T O S P R Q 
F E G D H C I B J A K Y L X M W N V O U P T Q S R 
G F H E I D J C K B L A M Y N X O W P V Q U R T S 
H G I F J E K D L C M B N A O Y P X Q W R V S U T 
I H J G K F L E M D N C O B P A Q Y R X S W T V U 
J I K H L G M F N E O D P C Q B R A S Y T X U W V 
K J L I M H N G O F P E Q D R C S B T A U Y V X W 
L K M J N I O H P G Q F R E S D T C U B V A W Y X 
M L N K O J P I Q H R G S F T E U D V C W B X A Y 
 


Size = 26

A B Z C Y D X E W F V G U H T I S J R K Q L P M O N 
B C A D Z E Y F X G W H V I U J T K S L R M Q N P O 
C D B E A F Z G Y H X I W J V K U L T M S N R O Q P 
D E C F B G A H Z I Y J X K W L V M U N T O S P R Q 
E F D G C H B I A J Z K Y L X M W N V O U P T Q S R 
F G E H D I C J B K A L Z M Y N X O W P V Q U R T S 
G H F I E J D K C L B M A N Z O Y P X Q W R V S U T 
H I G J F K E L D M C N B O A P Z Q Y R X S W T V U 
I J H K G L F M E N D O C P B Q A R Z S Y T X U W V 
J K I L H M G N F O E P D Q C R B S A T Z U Y V X W 
K L J M I N H O G P F Q E R D S C T B U A V Z W Y X 
L M K N J O I P H Q G R F S E T D U C V B W A X Z Y 
M N L O K P J Q I R H S G T F U E V D W C X B Y A Z 
N O M P L Q K R J S I T H U G V F W E X D Y C Z B A 
O P N Q M R L S K T J U I V H W G X F Y E Z D A C B 
P Q O R N S M T L U K V J W I X H Y G Z F A E B D C 
Q R P S O T N U M V L W K X J Y I Z H A G B F C E D 
R S Q T P U O V N W M X L Y K Z J A I B H C G D F E 
S T R U Q V P W O X N Y M Z L A K B J C I D H E G F 
T U S V R W Q X P Y O Z N A M B L C K D J E I F H G 
U V T W S X R Y Q Z P A O B N C M D L E K F J G I H 
V W U X T Y S Z R A Q B P C O D N E M F L G K H J I 
W X V Y U Z T A S B R C Q D P E O F N G M H L I K J 
X Y W Z V A U B T C S D R E Q F P G O H N I M J L K 
Y Z X A W B V C U D T E S F R G Q H P I O J N K M L 
Z A Y B X C W D V E U F T G S H R I Q J P K O L N M 

Randomized Withdrawal Design - Examples for Defining the Criteria for Run-in and Randomized Withdrawal Periods

In a previous post, we discussed the "Randomized Withdrawal Design and Randomized Discontinuation Trial". The randomized withdrawal design and the randomized discontinuation design may be used interchangeably. The randomized withdrawal design is one of the clinical trial designs with enrichment strategy and is more efficient design if it is applied in the appropriate situation.

The diagram for a typical randomized withdrawal design will be something like below (with efficacy measure in the run-in period).

or with safety or tolerability as the measure in the run-in period.

The randomized withdrawal design contains a run-in period and a randomized, controlled period. For each period, the criteria will need to be defined. For open-label run-in period, the criteria are needed to define what is considered as 'responder' (for efficacy measure) or 'tolerable' (for safety measure). For the randomized, controlled period where the formal hypothesis testing is based on, the study endpoint will need to be defined.

The table below listed some examples of clinical trials using the randomized withdrawal design. The criteria for the run-in period and for randomized withdrawal period are listed. 

	Criteria for Responder or tolerability	Endpoint for Randomized Withdrawal Period
ICE Study (immunoglobulin in CIDP)	10-point INCAT score improve by 1 point	Time to relapse where the relapse was defined as 10-point INCAT score worsen by 1 point
Droxidopa for Symptomatic Neurogenic Orthostatic Hypotension	based on symptom and BP response where the response is determined by improvement of at least one point on a symptom question (Orthostatic Hypotension Symptom Assessment (OHSA) Item 1) and an improvement in SBP of at least 10 mmHg at 3 minutes post-standing]	the mean change from Randomization (Visit 4) to the End of Study Visit (Visit 5) in the OHSA Item 1 (dizziness, lightheadedness, feeling faint or feeling like you might black out) score
Nifedipine for vasospastic angina	Stable	recurrence of severe vasospastic angina leading to study withdrawal
Gamma-hydroxybutyrate (GBH, sodium oxybate) for treatment of cataplexy	Median Attacks every 2 weeks	Median Attacks every 2 weeks
Double-blind, Placebo-controlled, Randomised Withdrawal, Extension, Safety and Efficacy Study of LDX in Children and Adolescents Aged 6-17	Completed a minimum of 4 weeks of double-blind treatment, reached Visit 4 and completed the 1-week post-treatment washout in the antecedent study (SPD489-325), without experiencing any clinically significant AEs that would preclude exposure to LDX	Treatment failure defined as 50% increase (worsening) in Attention Deficit Hyperactivity Disorder Rating Scale (ADHD-RS-IV) total score and greater than and equal to 2 point increase (worsening) in the Clinical Global Impression-Severity of Illness (CGI-S) score observed at any visit during the randomized withdrawal period compared to the respective scores at baseline of randomized withdrawal period.
A double-blind, placebo-controlled, randomized withdrawal study of lurasidone for the maintenance of efficacy in patients with schizophrenia	To achieve and maintain clinical stability for at least 12 weeks during the open-label stabilization phase and to have remained on a stable dose of lurasidone for four weeks prior to randomization. Clinical stability was defined as a PANSS total score less than or equal to 70, with PANSS item scores less than or equal to 4 on all positive subscale items and the item for "uncooperativeness”, (item G8), and a CGI-S score less than 4.	Time to relapse (based on Kaplan–Meier survival analysis), with relapse defined as greater than and equal to 1 of the following during the double-blind phase: (1) An increase of greater than and equal to 25% from double-blind baseline in PANSS total score and CGI-S worsening of greater than and equal to 1 point for two consecutive visits no more than ten days apart. (2) At any single visit, a PANSS item score of greater than and equal to 5 (moderately severe) on hostility or uncooperativeness, or a PANSS item score of greater than and equal to 5 on two or more items of unusual thought content, delusions, conceptual disorganization, or hallucinatory behavior. (3) Initiation of supplemental treatment with an antipsychotic agent other than lurasidone, an increased dose of an antidepressant or mood stabilizer, an increase in lorazepam (or benzodiazepine equivalent) dose by greater than and equal to 2 mg/d for at least 3 days, or electroconvulsive therapy. (4) Insufficient clinical response or exacerbation of underlying disease reported as an adverse event, as determined by the study investigator. (5) Deliberate self-injury or repeated aggressive behavior, active suicidal or homicidal ideation or attempt. (6) Psychiatric hospitalization due to worsening schizophrenia. Secondary
A Randomized Withdrawal, Placebo-Controlled Study Evaluating the Efficacy and Tolerability of Tapentadol Extended Release in Patients With Chronic Painful Diabetic Peripheral Neuropathy	Patients who tolerated tapentadol ER and had 1-point improvement in average pain intensity from the pre-titration evaluation period to the last 3 days of the open-label titration period were randomly assigned (1:1) to receive tapentadol ER or placebo during a subsequent 12-week double-blind maintenance phase	The primary efficacy endpoint was the mean change in average pain intensity from baseline to week 12
Long-term Maintenance of Response Across Multiple Fibromyalgia Symptom Domains in a Randomized Withdrawal Study of Pregabalin	meeting response criteria for pain [Z50% reduction in pain 100-mm Visual Analog Scale (VAS) score from OL baseline] and PGIC (self-rating of much improved or very much improved) at the end of the OL treatment phase	Time to LTR (loss of therapeutic response)
The randomized discontinuation trial: a phase II design to assess growth-inhibitory agents	Radiologic stable	Fraction with radiologic stable disease