FDA launches new digital tool to help capture real world data from patients to help inform regulatory decision-making

FDA In Brief: FDA launches new digital tool to help capture real world data from patients to help inform regulatory decision-making

FDA’s MyStudies Application (App)

“There are a lot of new ways that we can use real world evidence to help inform regulatory decisions around medical products as the collection of this data gets more widespread and reliable. Better capture of real world data, collected from a variety of sources, has the potential to make our new drug development process more efficient, improve safety and help lower the cost of product development. If done right, it can also help make sure doctors and patients are better informed about the clinical use of new products, enabling them to make more effective, efficient medical choices. This will ultimately help us achieve better outcomes, and safer and more efficient use of expensive technology,” said FDA Commissioner Scott Gottlieb, M.D. “Today we’re unveiling the new MyStudies app for the collection of real world data. This digital platform enables developers to adapt our technology to advance new ways to access and use data collected directly from patients—with the necessary controls in place to ensure patient privacy. Our hope is that the collection of more real world data directly from patients, using a secure app, will lead to more efficient product development and assist with safety monitoring.”

Today the U.S. Food and Drug Administration is announcing the MyStudies app, a new mobile technology to foster the collection of real world evidence via patients’ mobile devices. Real world data can be collected from a variety of sources, such as electronic health records, claims and billing activities, and product and disease registries, as well as from patient-generated data including in home-use settings, or from data gathered from other sources, such as mobile devices.

As part of the agency’s work to foster greater opportunities around real world evidence, the FDA partnered with Kaiser Permanente on a pilot study to measure the functionality and engagement of the MyStudies app. Based on the successful outcome of the study, the FDA is now releasing the open source code and technical documents that will allow researchers and developers to customize and use the FDA’s newly created MyStudies app to expand the diversity of health information available for clinical trials and studies, while directly capturing the perspective of patients. By providing the open source code, the agency is providing a tool that sponsors and developers can adapt to advance their specific clinical trial and real world evidence needs, while also remaining compliant with the FDA’s regulations and guidance for data authenticity, integrity and confidentiality.

For example, patients may be able to securely enroll in and contribute data to traditional clinical trials, pragmatic trials, observational studies and registries. Sponsors may be able to customize their apps to administer questionnaires assessing patient-reported outcomes, symptom scales or patient reports of prescription and over-the-counter medication use.

The overall effort was led by David Martin, M.D., associate director for real world evidence in the Office of Medical Policy in the FDA’s Center for Drug Evaluation and Research, with a grant from the U.S. Department of Health and Human Services’ Patient Centered Outcomes Research Trust Fund. The open source code that serves as the foundation of the MyStudies app, as well as specifications for a secure patient data storage environment, were developed through a collaboration with Harvard Pilgrim Health Care Institute, LabKey and Boston Technology Corporation

Fitting Compartment Models Using PROC NLMIXED

There is a new article on SAS's website "Fitting Compartment Models Using PROC NLMIXED". It is nice to know that SAS Proc NLMIXED can be used for fitting the compartment model in pharmacokinetic (PK) analyses.

Usually, for PK analyses, the non-compartment model will be used - i.e., Cmax, Tmax,... are based on the observed values and AUC is calculated using the trapezoid rule. I would say that for more than 90% of times, the non-compartment model will be good enough for the PK analyses.

However, in some situations, the non-compartment model or non-compartment analysis (NCA) will not be adequate. One-compartment model with first-order elimination and two-compartment model with first-order distribution and elimination processes may be more appropriate.

For PK analyses using NCA, the most commonly used software is Phoenix WinNonLin that was developed by Pharsight and now owned by Certara. For PK analyses based on compartment models, the most commonly used software is NONMEM that was developed by UCSF and now managed by ICON plc.

Potential Unblinding due to Imbalance in Adverse Event Profiles

In a previous post “Is blinded study really blinded? - assessment of blinding/unblinding in clinical trials”, we discussed the potential unblinding if the subjects on different treatment groups experienced a different type of adverse events. For the ethic reasons, the informed consent form needs to list all the potential side effects of the experimental drug. As required by GCP, the investigator’s brochure needs to discuss the details of the potential side effects caused by the experimental drug. If there is an obvious imbalance in adverse event profiles between the two treatment groups (experiment drug versus control), it is possible that the study blinding is not well maintained and the patient or the investigator can guess or predict based on the AE profile which treatment group the patient is on.

This potential unblinding issue can be even more evident if the crossover design is employed where the same subject will have an chance to experience two different drugs. If there are differences in side effects, the subjects will be likely to feel the difference.

In FDA’s recent guidance for industry “Hematologic Malignancy and Oncologic Disease: Considerations for Use of Placebos and Blinding in Randomized Controlled Clinical Trials for Drug Product Development”, the potential unblinding issue was brought up.

“In many cases, because of the toxicity profile of the active treatment, patients and investigators may infer which treatment patients are receiving and thus use of a placebo control may not, in fact, blind the treatment. …”

The guidance also cited the ethic issue for maintaining blinding when the unblinding is necessary to give the best care for the subjects (patients).

“Continued blinding of patients and investigators at the time of disease progression or occurrence of serious adverse events presents additional challenges. For example, in a blinded immunotherapy trial, a patient who develops adverse events on the control arm may receive unnecessary treatments (e.g., immunosuppressive drug products including a high dose of glucocorticoids, cyclophosphamide, interleukin-6 antagonist, or infliximab) for management of adverse events incorrectly attributed to the investigational drug product. Maintaining the blind after disease progression could also affect a patient’s subsequent therapy, potentially preventing a patient who had been on a placebo arm from receiving an approved therapy, or delaying or preventing the patient’s entry into other clinical trials (for those trials of similar drug products that may have specific exclusion criteria based on prior treatment with an active drug or class of drugs). Unblinding would allow informed decision-making with respect to additional treatment 61 options (see below). “

In an analysis by Shah et al Adverse events appear to unblind clinical trials in irritable bowel syndrome, the result “suggests that higher AE incidence on active therapy is associated with more beneficial patient‐reported outcomes in IBS clinical trials. This raises the issue of spontaneous unblinding”

Maintaining the blinding is one of the cornerstones of the RCT (randomized, controlled trial). As mentioned in an article by Stefan Dürr Avoid Unintentional Unblinding In Clinical Trials

“The knowledge of the patient’s treatment could potentially lead to conscious or unconscious bias in the way the site staff recruits the patients (selection bias), how they are treated (performance bias), the assessment of endpoints (detection bias), the handling of withdrawals, and how data is excluded from analysis. The patient attitude to the treatment if known can lead to a difference in reporting symptoms (response bias) or withdrawal from the study (attrition bias). To reduce the chance of bias to the data, clinical trials should be conducted in a double-blind design whenever possible.”

However, in some situation especially when there is an imbalance in the side effects of two treatment groups, blinding may either be difficult to maintain or not be ethical to implement, and alternative approaches are needed. As suggested in FDA's guidance "

“Hematologic Malignancy and Oncologic Disease: Considerations for Use of Placebos and Blinding in Randomized Controlled Clinical Trials for Drug Product Development”, an open-label study or an add-on trial design may be more appropriate. In an early post, I also discussed some blinding techniques. 

Latin Squares for Constructing "Williams Designs" - Balanced for First-order Carry-over (Residual) Effects

Courtesy from the website http://statpages.info/latinsq.html, the list below contains the Williams design with 2 to 26 treatment groups. It is unlikely to have a clinical trial with 26 treatment groups. The most number of treatment groups I see is a study with 6 treatment groups. 

In one of my previous posts, I described the Williams design and its usefulness in phase I or clinical pharmacology studies. In a SAS SUGI paper "Generating Randomization Schedules Using SAS Programming", I described how to use SAS program to construct the randomization schedule for a study with Williams design. 

With the limited sample size, the Williams design is the most efficient design and minimizes the potential impact of the carryover effect. In Williams design with an even number of n treatment groups, there will be n number of sequences or an "n by n" square. Each treatment only occurs one time in one sequence, in one period. Furthermore, each treatment only follows another treatment one time. 

If the number of treatments (n) is an odd number, there will be 2 x n number of sequences or two n by n squares. Each treatment only occurs once in one sequence and in two periods. Each treatment only follows another treatment twice. 

With the Latin Square listed below, we can easily construct the crossover design with treatments, periods, and sequences. For example, a study design with 3 treatment groups will have the following assignments: three treatment groups (A, B, C), three periods (period 1, period 2, and period 3), and six sequences (ABC, BCA, CAB, CBA, ACB, and BAC). 

Treatment Sequence	Period 1	Period 2	Period 3
1	A	B	C
2	B	C	A
3	C	A	B
4	C	B	A
5	A	C	B
6	B	A	C

• Since it is the crossover design, there should be a washout period between each period. It is very common that the washout period is 5 times the half-life. 
• Randomization is for the sequence, not the treatment group. 
• Once we know the number of subjects is needed for each sequence, we will know the total number of subjects for the entire study. 
• Usually, the mixed model is needed for statistical analysis. The statistical model will need to consider the treatment, period, and sequence. An example of the SAS codes for analysis was provided in one of the previous posts "Cookbook SAS Codes for Bioequivalence Test in 2x2x2 Crossover Design"

Williams Design by the Number of Treatment Groups:

Size = 2

A B 
B A 
 

Size = 3

A B C 
B C A 
C A B 
C B A 
A C B 
B A C 
 

Size = 4

A B D C 
B C A D 
C D B A 
D A C B 
 

Size = 5

A B E C D 
B C A D E 
C D B E A 
D E C A B 
E A D B C 
D C E B A 
E D A C B 
A E B D C 
B A C E D 
C B D A E 
 

Size = 6

A B F C E D 
B C A D F E 
C D B E A F 
D E C F B A 
E F D A C B 
F A E B D C 
 

Size = 7

A B G C F D E 
B C A D G E F 
C D B E A F G 
D E C F B G A 
E F D G C A B 
F G E A D B C 
G A F B E C D 
E D F C G B A 
F E G D A C B 
G F A E B D C 
A G B F C E D 
B A C G D F E 
C B D A E G F 
D C E B F A G 
 

Size = 8

A B H C G D F E 
B C A D H E G F 
C D B E A F H G 
D E C F B G A H 
E F D G C H B A 
F G E H D A C B 
G H F A E B D C 
H A G B F C E D 
 

Size = 9

A B I C H D G E F 
B C A D I E H F G 
C D B E A F I G H 
D E C F B G A H I 
E F D G C H B I A 
F G E H D I C A B 
G H F I E A D B C 
H I G A F B E C D 
I A H B G C F D E 
F E G D H C I B A 
G F H E I D A C B 
H G I F A E B D C 
I H A G B F C E D 
A I B H C G D F E 
B A C I D H E G F 
C B D A E I F H G 
D C E B F A G I H 
E D F C G B H A I 
 

Size = 10

A B J C I D H E G F 
B C A D J E I F H G 
C D B E A F J G I H 
D E C F B G A H J I 
E F D G C H B I A J 
F G E H D I C J B A 
G H F I E J D A C B 
H I G J F A E B D C 
I J H A G B F C E D 
J A I B H C G D F E 
 
 
Size = 11

A B K C J D I E H F G 
B C A D K E J F I G H 
C D B E A F K G J H I 
D E C F B G A H K I J 
E F D G C H B I A J K 
F G E H D I C J B K A 
G H F I E J D K C A B 
H I G J F K E A D B C 
I J H K G A F B E C D 
J K I A H B G C F D E 
K A J B I C H D G E F 
G F H E I D J C K B A 
H G I F J E K D A C B 
I H J G K F A E B D C 
J I K H A G B F C E D 
K J A I B H C G D F E 
A K B J C I D H E G F 
B A C K D J E I F H G 
C B D A E K F J G I H 
D C E B F A G K H J I 
E D F C G B H A I K J 
F E G D H C I B J A K 
 

Size = 12

A B L C K D J E I F H G 
B C A D L E K F J G I H 
C D B E A F L G K H J I 
D E C F B G A H L I K J 
E F D G C H B I A J L K 
F G E H D I C J B K A L 
G H F I E J D K C L B A 
H I G J F K E L D A C B 
I J H K G L F A E B D C 
J K I L H A G B F C E D 
K L J A I B H C G D F E 
L A K B J C I D H E G F 
 
 
Size = 13

A B M C L D K E J F I G H 
B C A D M E L F K G J H I 
C D B E A F M G L H K I J 
D E C F B G A H M I L J K 
E F D G C H B I A J M K L 
F G E H D I C J B K A L M 
G H F I E J D K C L B M A 
H I G J F K E L D M C A B 
I J H K G L F M E A D B C 
J K I L H M G A F B E C D 
K L J M I A H B G C F D E 
L M K A J B I C H D G E F 
M A L B K C J D I E H F G 
H G I F J E K D L C M B A 
I H J G K F L E M D A C B 
J I K H L G M F A E B D C 
K J L I M H A G B F C E D 
L K M J A I B H C G D F E 
M L A K B J C I D H E G F 
A M B L C K D J E I F H G 
B A C M D L E K F J G I H 
C B D A E M F L G K H J I 
D C E B F A G M H L I K J 
E D F C G B H A I M J L K 
F E G D H C I B J A K M L 
G F H E I D J C K B L A M 
 
 
Size = 14

A B N C M D L E K F J G I H 
B C A D N E M F L G K H J I 
C D B E A F N G M H L I K J 
D E C F B G A H N I M J L K 
E F D G C H B I A J N K M L 
F G E H D I C J B K A L N M 
G H F I E J D K C L B M A N 
H I G J F K E L D M C N B A 
I J H K G L F M E N D A C B 
J K I L H M G N F A E B D C 
K L J M I N H A G B F C E D 
L M K N J A I B H C G D F E 
M N L A K B J C I D H E G F 
N A M B L C K D J E I F H G 
 
 
Size = 15

A B O C N D M E L F K G J H I 
B C A D O E N F M G L H K I J 
C D B E A F O G N H M I L J K 
D E C F B G A H O I N J M K L 
E F D G C H B I A J O K N L M 
F G E H D I C J B K A L O M N 
G H F I E J D K C L B M A N O 
H I G J F K E L D M C N B O A 
I J H K G L F M E N D O C A B 
J K I L H M G N F O E A D B C 
K L J M I N H O G A F B E C D 
L M K N J O I A H B G C F D E 
M N L O K A J B I C H D G E F 
N O M A L B K C J D I E H F G 
O A N B M C L D K E J F I G H 
I H J G K F L E M D N C O B A 
J I K H L G M F N E O D A C B 
K J L I M H N G O F A E B D C 
L K M J N I O H A G B F C E D 
M L N K O J A I B H C G D F E 
N M O L A K B J C I D H E G F 
O N A M B L C K D J E I F H G 
A O B N C M D L E K F J G I H 
B A C O D N E M F L G K H J I 
C B D A E O F N G M H L I K J 
D C E B F A G O H N I M J L K 
E D F C G B H A I O J N K M L 
F E G D H C I B J A K O L N M 
G F H E I D J C K B L A M O N 
H G I F J E K D L C M B N A O 
 

Size = 16

A B P C O D N E M F L G K H J I 
B C A D P E O F N G M H L I K J 
C D B E A F P G O H N I M J L K 
D E C F B G A H P I O J N K M L 
E F D G C H B I A J P K O L N M 
F G E H D I C J B K A L P M O N 
G H F I E J D K C L B M A N P O 
H I G J F K E L D M C N B O A P 
I J H K G L F M E N D O C P B A 
J K I L H M G N F O E P D A C B 
K L J M I N H O G P F A E B D C 
L M K N J O I P H A G B F C E D 
M N L O K P J A I B H C G D F E 
N O M P L A K B J C I D H E G F 
O P N A M B L C K D J E I F H G 
P A O B N C M D L E K F J G I H 
 

Size = 17

A B Q C P D O E N F M G L H K I J 
B C A D Q E P F O G N H M I L J K 
C D B E A F Q G P H O I N J M K L 
D E C F B G A H Q I P J O K N L M 
E F D G C H B I A J Q K P L O M N 
F G E H D I C J B K A L Q M P N O 
G H F I E J D K C L B M A N Q O P 
H I G J F K E L D M C N B O A P Q 
I J H K G L F M E N D O C P B Q A 
J K I L H M G N F O E P D Q C A B 
K L J M I N H O G P F Q E A D B C 
L M K N J O I P H Q G A F B E C D 
M N L O K P J Q I A H B G C F D E 
N O M P L Q K A J B I C H D G E F 
O P N Q M A L B K C J D I E H F G 
P Q O A N B M C L D K E J F I G H 
Q A P B O C N D M E L F K G J H I 
J I K H L G M F N E O D P C Q B A 
K J L I M H N G O F P E Q D A C B 
L K M J N I O H P G Q F A E B D C 
M L N K O J P I Q H A G B F C E D 
N M O L P K Q J A I B H C G D F E 
O N P M Q L A K B J C I D H E G F 
P O Q N A M B L C K D J E I F H G 
Q P A O B N C M D L E K F J G I H 
A Q B P C O D N E M F L G K H J I 
B A C Q D P E O F N G M H L I K J 
C B D A E Q F P G O H N I M J L K 
D C E B F A G Q H P I O J N K M L 
E D F C G B H A I Q J P K O L N M 
F E G D H C I B J A K Q L P M O N 
G F H E I D J C K B L A M Q N P O 
H G I F J E K D L C M B N A O Q P 
I H J G K F L E M D N C O B P A Q 
 
 

Size = 18

A B R C Q D P E O F N G M H L I K J 
B C A D R E Q F P G O H N I M J L K 
C D B E A F R G Q H P I O J N K M L 
D E C F B G A H R I Q J P K O L N M 
E F D G C H B I A J R K Q L P M O N 
F G E H D I C J B K A L R M Q N P O 
G H F I E J D K C L B M A N R O Q P 
H I G J F K E L D M C N B O A P R Q 
I J H K G L F M E N D O C P B Q A R 
J K I L H M G N F O E P D Q C R B A 
K L J M I N H O G P F Q E R D A C B 
L M K N J O I P H Q G R F A E B D C 
M N L O K P J Q I R H A G B F C E D 
N O M P L Q K R J A I B H C G D F E 
O P N Q M R L A K B J C I D H E G F 
P Q O R N A M B L C K D J E I F H G 
Q R P A O B N C M D L E K F J G I H 
R A Q B P C O D N E M F L G K H J I 
 

Size = 19

A B S C R D Q E P F O G N H M I L J K 
B C A D S E R F Q G P H O I N J M K L 
C D B E A F S G R H Q I P J O K N L M 
D E C F B G A H S I R J Q K P L O M N 
E F D G C H B I A J S K R L Q M P N O 
F G E H D I C J B K A L S M R N Q O P 
G H F I E J D K C L B M A N S O R P Q 
H I G J F K E L D M C N B O A P S Q R 
I J H K G L F M E N D O C P B Q A R S 
J K I L H M G N F O E P D Q C R B S A 
K L J M I N H O G P F Q E R D S C A B 
L M K N J O I P H Q G R F S E A D B C 
M N L O K P J Q I R H S G A F B E C D 
N O M P L Q K R J S I A H B G C F D E 
O P N Q M R L S K A J B I C H D G E F 
P Q O R N S M A L B K C J D I E H F G 
Q R P S O A N B M C L D K E J F I G H 
R S Q A P B O C N D M E L F K G J H I 
S A R B Q C P D O E N F M G L H K I J 
K J L I M H N G O F P E Q D R C S B A 
L K M J N I O H P G Q F R E S D A C B 
M L N K O J P I Q H R G S F A E B D C 
N M O L P K Q J R I S H A G B F C E D 
O N P M Q L R K S J A I B H C G D F E 
P O Q N R M S L A K B J C I D H E G F 
Q P R O S N A M B L C K D J E I F H G 
R Q S P A O B N C M D L E K F J G I H 
S R A Q B P C O D N E M F L G K H J I 
A S B R C Q D P E O F N G M H L I K J 
B A C S D R E Q F P G O H N I M J L K 
C B D A E S F R G Q H P I O J N K M L 
D C E B F A G S H R I Q J P K O L N M 
E D F C G B H A I S J R K Q L P M O N 
F E G D H C I B J A K S L R M Q N P O 
G F H E I D J C K B L A M S N R O Q P 
H G I F J E K D L C M B N A O S P R Q 
I H J G K F L E M D N C O B P A Q S R 
J I K H L G M F N E O D P C Q B R A S 
 

Size = 20

A B T C S D R E Q F P G O H N I M J L K 
B C A D T E S F R G Q H P I O J N K M L 
C D B E A F T G S H R I Q J P K O L N M 
D E C F B G A H T I S J R K Q L P M O N 
E F D G C H B I A J T K S L R M Q N P O 
F G E H D I C J B K A L T M S N R O Q P 
G H F I E J D K C L B M A N T O S P R Q 
H I G J F K E L D M C N B O A P T Q S R 
I J H K G L F M E N D O C P B Q A R T S 
J K I L H M G N F O E P D Q C R B S A T 
K L J M I N H O G P F Q E R D S C T B A 
L M K N J O I P H Q G R F S E T D A C B 
M N L O K P J Q I R H S G T F A E B D C 
N O M P L Q K R J S I T H A G B F C E D 
O P N Q M R L S K T J A I B H C G D F E 
P Q O R N S M T L A K B J C I D H E G F 
Q R P S O T N A M B L C K D J E I F H G 
R S Q T P A O B N C M D L E K F J G I H 
S T R A Q B P C O D N E M F L G K H J I 
T A S B R C Q D P E O F N G M H L I K J 
 

Size = 21

A B U C T D S E R F Q G P H O I N J M K L 
B C A D U E T F S G R H Q I P J O K N L M 
C D B E A F U G T H S I R J Q K P L O M N 
D E C F B G A H U I T J S K R L Q M P N O 
E F D G C H B I A J U K T L S M R N Q O P 
F G E H D I C J B K A L U M T N S O R P Q 
G H F I E J D K C L B M A N U O T P S Q R 
H I G J F K E L D M C N B O A P U Q T R S 
I J H K G L F M E N D O C P B Q A R U S T 
J K I L H M G N F O E P D Q C R B S A T U 
K L J M I N H O G P F Q E R D S C T B U A 
L M K N J O I P H Q G R F S E T D U C A B 
M N L O K P J Q I R H S G T F U E A D B C 
N O M P L Q K R J S I T H U G A F B E C D 
O P N Q M R L S K T J U I A H B G C F D E 
P Q O R N S M T L U K A J B I C H D G E F 
Q R P S O T N U M A L B K C J D I E H F G 
R S Q T P U O A N B M C L D K E J F I G H 
S T R U Q A P B O C N D M E L F K G J H I 
T U S A R B Q C P D O E N F M G L H K I J 
U A T B S C R D Q E P F O G N H M I L J K 
L K M J N I O H P G Q F R E S D T C U B A 
M L N K O J P I Q H R G S F T E U D A C B 
N M O L P K Q J R I S H T G U F A E B D C 
O N P M Q L R K S J T I U H A G B F C E D 
P O Q N R M S L T K U J A I B H C G D F E 
Q P R O S N T M U L A K B J C I D H E G F 
R Q S P T O U N A M B L C K D J E I F H G 
S R T Q U P A O B N C M D L E K F J G I H 
T S U R A Q B P C O D N E M F L G K H J I 
U T A S B R C Q D P E O F N G M H L I K J 
A U B T C S D R E Q F P G O H N I M J L K 
B A C U D T E S F R G Q H P I O J N K M L 
C B D A E U F T G S H R I Q J P K O L N M 
D C E B F A G U H T I S J R K Q L P M O N 
E D F C G B H A I U J T K S L R M Q N P O 
F E G D H C I B J A K U L T M S N R O Q P 
G F H E I D J C K B L A M U N T O S P R Q 
H G I F J E K D L C M B N A O U P T Q S R 
I H J G K F L E M D N C O B P A Q U R T S 
J I K H L G M F N E O D P C Q B R A S U T 
K J L I M H N G O F P E Q D R C S B T A U 
 


Size = 22

A B V C U D T E S F R G Q H P I O J N K M L 
B C A D V E U F T G S H R I Q J P K O L N M 
C D B E A F V G U H T I S J R K Q L P M O N 
D E C F B G A H V I U J T K S L R M Q N P O 
E F D G C H B I A J V K U L T M S N R O Q P 
F G E H D I C J B K A L V M U N T O S P R Q 
G H F I E J D K C L B M A N V O U P T Q S R 
H I G J F K E L D M C N B O A P V Q U R T S 
I J H K G L F M E N D O C P B Q A R V S U T 
J K I L H M G N F O E P D Q C R B S A T V U 
K L J M I N H O G P F Q E R D S C T B U A V 
L M K N J O I P H Q G R F S E T D U C V B A 
M N L O K P J Q I R H S G T F U E V D A C B 
N O M P L Q K R J S I T H U G V F A E B D C 
O P N Q M R L S K T J U I V H A G B F C E D 
P Q O R N S M T L U K V J A I B H C G D F E 
Q R P S O T N U M V L A K B J C I D H E G F 
R S Q T P U O V N A M B L C K D J E I F H G 
S T R U Q V P A O B N C M D L E K F J G I H 
T U S V R A Q B P C O D N E M F L G K H J I 
U V T A S B R C Q D P E O F N G M H L I K J 
V A U B T C S D R E Q F P G O H N I M J L K 
 


Size = 23

A B W C V D U E T F S G R H Q I P J O K N L M 
B C A D W E V F U G T H S I R J Q K P L O M N 
C D B E A F W G V H U I T J S K R L Q M P N O 
D E C F B G A H W I V J U K T L S M R N Q O P 
E F D G C H B I A J W K V L U M T N S O R P Q 
F G E H D I C J B K A L W M V N U O T P S Q R 
G H F I E J D K C L B M A N W O V P U Q T R S 
H I G J F K E L D M C N B O A P W Q V R U S T 
I J H K G L F M E N D O C P B Q A R W S V T U 
J K I L H M G N F O E P D Q C R B S A T W U V 
K L J M I N H O G P F Q E R D S C T B U A V W 
L M K N J O I P H Q G R F S E T D U C V B W A 
M N L O K P J Q I R H S G T F U E V D W C A B 
N O M P L Q K R J S I T H U G V F W E A D B C 
O P N Q M R L S K T J U I V H W G A F B E C D 
P Q O R N S M T L U K V J W I A H B G C F D E 
Q R P S O T N U M V L W K A J B I C H D G E F 
R S Q T P U O V N W M A L B K C J D I E H F G 
S T R U Q V P W O A N B M C L D K E J F I G H 
T U S V R W Q A P B O C N D M E L F K G J H I 
U V T W S A R B Q C P D O E N F M G L H K I J 
V W U A T B S C R D Q E P F O G N H M I L J K 
W A V B U C T D S E R F Q G P H O I N J M K L 
M L N K O J P I Q H R G S F T E U D V C W B A 
N M O L P K Q J R I S H T G U F V E W D A C B 
O N P M Q L R K S J T I U H V G W F A E B D C 
P O Q N R M S L T K U J V I W H A G B F C E D 
Q P R O S N T M U L V K W J A I B H C G D F E 
R Q S P T O U N V M W L A K B J C I D H E G F 
S R T Q U P V O W N A M B L C K D J E I F H G 
T S U R V Q W P A O B N C M D L E K F J G I H 
U T V S W R A Q B P C O D N E M F L G K H J I 
V U W T A S B R C Q D P E O F N G M H L I K J 
W V A U B T C S D R E Q F P G O H N I M J L K 
A W B V C U D T E S F R G Q H P I O J N K M L 
B A C W D V E U F T G S H R I Q J P K O L N M 
C B D A E W F V G U H T I S J R K Q L P M O N 
D C E B F A G W H V I U J T K S L R M Q N P O 
E D F C G B H A I W J V K U L T M S N R O Q P 
F E G D H C I B J A K W L V M U N T O S P R Q 
G F H E I D J C K B L A M W N V O U P T Q S R 
H G I F J E K D L C M B N A O W P V Q U R T S 
I H J G K F L E M D N C O B P A Q W R V S U T 
J I K H L G M F N E O D P C Q B R A S W T V U 
K J L I M H N G O F P E Q D R C S B T A U W V 
L K M J N I O H P G Q F R E S D T C U B V A W 
 

Size = 24

A B X C W D V E U F T G S H R I Q J P K O L N M 
B C A D X E W F V G U H T I S J R K Q L P M O N 
C D B E A F X G W H V I U J T K S L R M Q N P O 
D E C F B G A H X I W J V K U L T M S N R O Q P 
E F D G C H B I A J X K W L V M U N T O S P R Q 
F G E H D I C J B K A L X M W N V O U P T Q S R 
G H F I E J D K C L B M A N X O W P V Q U R T S 
H I G J F K E L D M C N B O A P X Q W R V S U T 
I J H K G L F M E N D O C P B Q A R X S W T V U 
J K I L H M G N F O E P D Q C R B S A T X U W V 
K L J M I N H O G P F Q E R D S C T B U A V X W 
L M K N J O I P H Q G R F S E T D U C V B W A X 
M N L O K P J Q I R H S G T F U E V D W C X B A 
N O M P L Q K R J S I T H U G V F W E X D A C B 
O P N Q M R L S K T J U I V H W G X F A E B D C 
P Q O R N S M T L U K V J W I X H A G B F C E D 
Q R P S O T N U M V L W K X J A I B H C G D F E 
R S Q T P U O V N W M X L A K B J C I D H E G F 
S T R U Q V P W O X N A M B L C K D J E I F H G 
T U S V R W Q X P A O B N C M D L E K F J G I H 
U V T W S X R A Q B P C O D N E M F L G K H J I 
V W U X T A S B R C Q D P E O F N G M H L I K J 
W X V A U B T C S D R E Q F P G O H N I M J L K 
X A W B V C U D T E S F R G Q H P I O J N K M L 
 
 

Size = 25

A B Y C X D W E V F U G T H S I R J Q K P L O M N 
B C A D Y E X F W G V H U I T J S K R L Q M P N O 
C D B E A F Y G X H W I V J U K T L S M R N Q O P 
D E C F B G A H Y I X J W K V L U M T N S O R P Q 
E F D G C H B I A J Y K X L W M V N U O T P S Q R 
F G E H D I C J B K A L Y M X N W O V P U Q T R S 
G H F I E J D K C L B M A N Y O X P W Q V R U S T 
H I G J F K E L D M C N B O A P Y Q X R W S V T U 
I J H K G L F M E N D O C P B Q A R Y S X T W U V 
J K I L H M G N F O E P D Q C R B S A T Y U X V W 
K L J M I N H O G P F Q E R D S C T B U A V Y W X 
L M K N J O I P H Q G R F S E T D U C V B W A X Y 
M N L O K P J Q I R H S G T F U E V D W C X B Y A 
N O M P L Q K R J S I T H U G V F W E X D Y C A B 
O P N Q M R L S K T J U I V H W G X F Y E A D B C 
P Q O R N S M T L U K V J W I X H Y G A F B E C D 
Q R P S O T N U M V L W K X J Y I A H B G C F D E 
R S Q T P U O V N W M X L Y K A J B I C H D G E F 
S T R U Q V P W O X N Y M A L B K C J D I E H F G 
T U S V R W Q X P Y O A N B M C L D K E J F I G H 
U V T W S X R Y Q A P B O C N D M E L F K G J H I 
V W U X T Y S A R B Q C P D O E N F M G L H K I J 
W X V Y U A T B S C R D Q E P F O G N H M I L J K 
X Y W A V B U C T D S E R F Q G P H O I N J M K L 
Y A X B W C V D U E T F S G R H Q I P J O K N L M 
N M O L P K Q J R I S H T G U F V E W D X C Y B A 
O N P M Q L R K S J T I U H V G W F X E Y D A C B 
P O Q N R M S L T K U J V I W H X G Y F A E B D C 
Q P R O S N T M U L V K W J X I Y H A G B F C E D 
R Q S P T O U N V M W L X K Y J A I B H C G D F E 
S R T Q U P V O W N X M Y L A K B J C I D H E G F 
T S U R V Q W P X O Y N A M B L C K D J E I F H G 
U T V S W R X Q Y P A O B N C M D L E K F J G I H 
V U W T X S Y R A Q B P C O D N E M F L G K H J I 
W V X U Y T A S B R C Q D P E O F N G M H L I K J 
X W Y V A U B T C S D R E Q F P G O H N I M J L K 
Y X A W B V C U D T E S F R G Q H P I O J N K M L 
A Y B X C W D V E U F T G S H R I Q J P K O L N M 
B A C Y D X E W F V G U H T I S J R K Q L P M O N 
C B D A E Y F X G W H V I U J T K S L R M Q N P O 
D C E B F A G Y H X I W J V K U L T M S N R O Q P 
E D F C G B H A I Y J X K W L V M U N T O S P R Q 
F E G D H C I B J A K Y L X M W N V O U P T Q S R 
G F H E I D J C K B L A M Y N X O W P V Q U R T S 
H G I F J E K D L C M B N A O Y P X Q W R V S U T 
I H J G K F L E M D N C O B P A Q Y R X S W T V U 
J I K H L G M F N E O D P C Q B R A S Y T X U W V 
K J L I M H N G O F P E Q D R C S B T A U Y V X W 
L K M J N I O H P G Q F R E S D T C U B V A W Y X 
M L N K O J P I Q H R G S F T E U D V C W B X A Y 
 


Size = 26

A B Z C Y D X E W F V G U H T I S J R K Q L P M O N 
B C A D Z E Y F X G W H V I U J T K S L R M Q N P O 
C D B E A F Z G Y H X I W J V K U L T M S N R O Q P 
D E C F B G A H Z I Y J X K W L V M U N T O S P R Q 
E F D G C H B I A J Z K Y L X M W N V O U P T Q S R 
F G E H D I C J B K A L Z M Y N X O W P V Q U R T S 
G H F I E J D K C L B M A N Z O Y P X Q W R V S U T 
H I G J F K E L D M C N B O A P Z Q Y R X S W T V U 
I J H K G L F M E N D O C P B Q A R Z S Y T X U W V 
J K I L H M G N F O E P D Q C R B S A T Z U Y V X W 
K L J M I N H O G P F Q E R D S C T B U A V Z W Y X 
L M K N J O I P H Q G R F S E T D U C V B W A X Z Y 
M N L O K P J Q I R H S G T F U E V D W C X B Y A Z 
N O M P L Q K R J S I T H U G V F W E X D Y C Z B A 
O P N Q M R L S K T J U I V H W G X F Y E Z D A C B 
P Q O R N S M T L U K V J W I X H Y G Z F A E B D C 
Q R P S O T N U M V L W K X J Y I Z H A G B F C E D 
R S Q T P U O V N W M X L Y K Z J A I B H C G D F E 
S T R U Q V P W O X N Y M Z L A K B J C I D H E G F 
T U S V R W Q X P Y O Z N A M B L C K D J E I F H G 
U V T W S X R Y Q Z P A O B N C M D L E K F J G I H 
V W U X T Y S Z R A Q B P C O D N E M F L G K H J I 
W X V Y U Z T A S B R C Q D P E O F N G M H L I K J 
X Y W Z V A U B T C S D R E Q F P G O H N I M J L K 
Y Z X A W B V C U D T E S F R G Q H P I O J N K M L 
Z A Y B X C W D V E U F T G S H R I Q J P K O L N M 

Randomized Withdrawal Design - Examples for Defining the Criteria for Run-in and Randomized Withdrawal Periods

In a previous post, we discussed the "Randomized Withdrawal Design and Randomized Discontinuation Trial". The randomized withdrawal design and the randomized discontinuation design may be used interchangeably. The randomized withdrawal design is one of the clinical trial designs with enrichment strategy and is more efficient design if it is applied in the appropriate situation.

The diagram for a typical randomized withdrawal design will be something like below (with efficacy measure in the run-in period).

or with safety or tolerability as the measure in the run-in period.

The randomized withdrawal design contains a run-in period and a randomized, controlled period. For each period, the criteria will need to be defined. For open-label run-in period, the criteria are needed to define what is considered as 'responder' (for efficacy measure) or 'tolerable' (for safety measure). For the randomized, controlled period where the formal hypothesis testing is based on, the study endpoint will need to be defined.

The table below listed some examples of clinical trials using the randomized withdrawal design. The criteria for the run-in period and for randomized withdrawal period are listed. 

	Criteria for Responder or tolerability	Endpoint for Randomized Withdrawal Period
ICE Study (immunoglobulin in CIDP)	10-point INCAT score improve by 1 point	Time to relapse where the relapse was defined as 10-point INCAT score worsen by 1 point
Droxidopa for Symptomatic Neurogenic Orthostatic Hypotension	based on symptom and BP response where the response is determined by improvement of at least one point on a symptom question (Orthostatic Hypotension Symptom Assessment (OHSA) Item 1) and an improvement in SBP of at least 10 mmHg at 3 minutes post-standing]	the mean change from Randomization (Visit 4) to the End of Study Visit (Visit 5) in the OHSA Item 1 (dizziness, lightheadedness, feeling faint or feeling like you might black out) score
Nifedipine for vasospastic angina	Stable	recurrence of severe vasospastic angina leading to study withdrawal
Gamma-hydroxybutyrate (GBH, sodium oxybate) for treatment of cataplexy	Median Attacks every 2 weeks	Median Attacks every 2 weeks
Double-blind, Placebo-controlled, Randomised Withdrawal, Extension, Safety and Efficacy Study of LDX in Children and Adolescents Aged 6-17	Completed a minimum of 4 weeks of double-blind treatment, reached Visit 4 and completed the 1-week post-treatment washout in the antecedent study (SPD489-325), without experiencing any clinically significant AEs that would preclude exposure to LDX	Treatment failure defined as 50% increase (worsening) in Attention Deficit Hyperactivity Disorder Rating Scale (ADHD-RS-IV) total score and greater than and equal to 2 point increase (worsening) in the Clinical Global Impression-Severity of Illness (CGI-S) score observed at any visit during the randomized withdrawal period compared to the respective scores at baseline of randomized withdrawal period.
A double-blind, placebo-controlled, randomized withdrawal study of lurasidone for the maintenance of efficacy in patients with schizophrenia	To achieve and maintain clinical stability for at least 12 weeks during the open-label stabilization phase and to have remained on a stable dose of lurasidone for four weeks prior to randomization. Clinical stability was defined as a PANSS total score less than or equal to 70, with PANSS item scores less than or equal to 4 on all positive subscale items and the item for "uncooperativeness”, (item G8), and a CGI-S score less than 4.	Time to relapse (based on Kaplan–Meier survival analysis), with relapse defined as greater than and equal to 1 of the following during the double-blind phase: (1) An increase of greater than and equal to 25% from double-blind baseline in PANSS total score and CGI-S worsening of greater than and equal to 1 point for two consecutive visits no more than ten days apart. (2) At any single visit, a PANSS item score of greater than and equal to 5 (moderately severe) on hostility or uncooperativeness, or a PANSS item score of greater than and equal to 5 on two or more items of unusual thought content, delusions, conceptual disorganization, or hallucinatory behavior. (3) Initiation of supplemental treatment with an antipsychotic agent other than lurasidone, an increased dose of an antidepressant or mood stabilizer, an increase in lorazepam (or benzodiazepine equivalent) dose by greater than and equal to 2 mg/d for at least 3 days, or electroconvulsive therapy. (4) Insufficient clinical response or exacerbation of underlying disease reported as an adverse event, as determined by the study investigator. (5) Deliberate self-injury or repeated aggressive behavior, active suicidal or homicidal ideation or attempt. (6) Psychiatric hospitalization due to worsening schizophrenia. Secondary
A Randomized Withdrawal, Placebo-Controlled Study Evaluating the Efficacy and Tolerability of Tapentadol Extended Release in Patients With Chronic Painful Diabetic Peripheral Neuropathy	Patients who tolerated tapentadol ER and had 1-point improvement in average pain intensity from the pre-titration evaluation period to the last 3 days of the open-label titration period were randomly assigned (1:1) to receive tapentadol ER or placebo during a subsequent 12-week double-blind maintenance phase	The primary efficacy endpoint was the mean change in average pain intensity from baseline to week 12
Long-term Maintenance of Response Across Multiple Fibromyalgia Symptom Domains in a Randomized Withdrawal Study of Pregabalin	meeting response criteria for pain [Z50% reduction in pain 100-mm Visual Analog Scale (VAS) score from OL baseline] and PGIC (self-rating of much improved or very much improved) at the end of the OL treatment phase	Time to LTR (loss of therapeutic response)
The randomized discontinuation trial: a phase II design to assess growth-inhibitory agents	Radiologic stable	Fraction with radiologic stable disease

FDA Guidance: Expansion Cohorts: Use in First-In-Human Clinical Trials to Expedite Development of Oncology Drugs and Biologics

Two years ago, I had a post discussing "Dose Cohort Expansion Study - A Never Ending Phase I Study Design". Last month, FDA officially issued the guidance for industry "Expansion Cohorts: Use in First-In-Human Clinical Trials to Expedite Development of Oncology Drugs and Biologics".

With this guidance, we expect that more sponsors will adopt this study design in oncology trials and beyond the oncology trials.

The dose cohort expansion study design completely changed the traditional phased approaches for clinical trials for drug development. The phases I, II, III, IV are no longer critical.

Splitting p-value and estimate of treatment difference

When we perform the statistical test to compare the difference between two treatment groups, we usually construct a test statistic, calculate the treatment difference, and then obtain the p-value corresponding to the test statistic and the treatment difference.

For example, for a study with primary efficacy endpoint of proportion of subjects with hemostasis, Treatment difference was estimated using risk ratio and the corresponding p-value was calculated to indicate if the risk ratio is statistically significant. 

Another example is for a study with 6-Min walk distance (6MWD) as the primary endpoint. The treatment difference was estimated using least-squares mean difference. The corresponding p-value was calculated using the analysis of covariance (ANCOVA) approach. 

In these examples, the p-value and the estimate of the treatment difference were from the same test statistic. 

We see many examples where two different methods are used for estimating the treatment difference and for calculating the p-value - I call it 'splitting the p-value and the estimate of the treatment difference'. Here are two situations where this splitting situation occurs.

Analysis of Time to Event: log-rank test for calculating the p-value and proportional hazard model for estimating the hazard ratio 

In clinical trials with time to event endpoint, it is very common to provide the Kaplan-Meier estimate and calculate the p-value using log-rank test - a non-parametric method. The treatment difference or magnitude of treatment effect is usually measured using hazard ratio. Kaplan-Meier estimate does not give an estimate of the hazard ratio. The hazard ratio needs to be estimated using the Proportional Hazard model (or Cox regression model). 

In a study by Sitbon et al, the primary efficacy endpoint was a composite endpoint of time to death or a complication related to PAH. The hazard ratio and p-value were provided in the primary efficacy table below. However, it needs to be noted that two different methods were used to calculate the hazard ratio and the p-value. As stated in the Statistical Analysis section, the statistical methods were provided as the following:  

"In time-to-event analyses, end points were estimated with the use of the Kaplan–Meier method and were analyzed with the use of the log-rank test. Hazard ratios with 99% confidence intervals (for primary and secondary end points) and 95% confidence intervals (for exploratory end points) were estimated with the use of proportional-hazard models."

 

p-value corresponding to the hazard ratio can also be obtained from the proportional hazard model, but is usually not presented in the place where p-value from the log-rank test is provided - to avoid the confusion about two different p-values. 

Why do we present the hazard ratio from one method and p-value from another method? why can't we present both the hazard ratio and the corresponding p-value from the proportional hazard model? 

   

Wilcoxon Rank Sum Test to calculate the p-value and Hodges-Lemman method to calculate the difference in median

Wilcoxon is also called Mann Whitney U Test and is a non-parametric method to compare the difference in medians for non-normal distributed data. Wilcoxon rank sum test converts the original data into ranks and the p-value is calculated to compare the total ranks between groups. However, the statistics of ranks has no meaning in measuring the magnitude of the treatment effect. Therefore, when the Wilcoxon method is used to calculate the p-value, a different method needs to be employed to estimate the treatment difference (magnitude of the treatment effect). Hodges-Lehmann method is now commonly used to estimate the treatment difference - location shift in medians between two treatment groups. 

Here is a link to FDA's statistical review for Xermelo (telotristat ethyl) oral tablets in indication of  Carcinoid Syndrome. p-value was calculated from Wilcoxon rank test and treatment difference (location shift in median) and confidence interval were calculated from Hodges Lehmann method.

The primary efficacy endpoints in studies LX301 and LX303 were analyzed by the blocked 2- sample Wilcoxon rank sum statistic stratified by the baseline urinary 5-HIAA levels (≤ upper limit of normal reference range [ULN], >ULN, and Unknown). Descriptive statistics of the primary endpoints and the Hodges-Lehmann estimator of location shift with its respective CLs were reported for each comparison.

In a paper by Jing et al, "Efficacy and Safety of Oral Treprostinil Monotherapy for the Treatment of Pulmonary Arterial Hypertension A Randomized, Controlled Trial", the endpoint of 6MWD was analyzed using Wilcoxon method for p-values and using the Hodges-Lemman estimator for the treatment effect (location shift in medians).