A clinical trial with sample size = 1?

In the October issue of New England Journal of Medicine, Kim and Hu et al from Boston Children's Hospital published a paper "Patient-Customized Oligonucleotide Therapy for a Rare Genetic Disease" for their study with only one patient - the so-called 'N of 1' or 'N of one' trial. 

Drs. Woodcock and Marks from FDA wrote an editorial for this study "Drug Regulation in the Era of Individualized Therapies". 

While the "N of 1" design fits into the paradigm of patient-centric drug development and precision medicine, a sample size of 1 doesn't fit into the current drug development and drug approval process.  We can see this from the editorial by Drs Woodcock and Marks. They raised a long list of questions with no answers: 

In these “N-of-one” situations, 

• what type of evidence is needed before exposing a human to a new drug? 

• Even in rapidly progressing, fatal illnesses, precipitating severe complications or death is not acceptable, so what is the minimum assurance of safety that is needed? 

• How persuasive should the mechanistic or functional data be?

• How should the dose and regimen be selected?

• How much characterization of the product should be undertaken? 

• How should the urgency of the patient’s situation or the number of people who could ultimately be treated affect the decisionmaking process?

• In addition, how will efficacy be evaluated? At the very least, during the time needed to discover and develop an intervention, quantifiable, objective measures of the patient’s disease status should be identified and tracked, since, in an N-of-one experiment, evaluation of disease trends before and after treatment will usually be the primary method of assessing effectiveness. In this regard, there is precedent for the application of new efficacy measures to the study of small numbers of patients.

In a previous post, "How Low in Sample Size Can We Go? FDA approves ultra-orphan drug on a 4-patient trial", we discussed a case that FDA approved a drug based on a 4-patient trial - that was the drug approval with the fewest sample size I was aware of. 

With four patients, we can still do some statistical calculations, for example, mean and standard deviation. With one patient, no statistical calculation is needed. 

The previous ASA president, Dr. Barry D. Nussbaum wrote in his president's corner article "Bigger Isn’t Always Better When It Comes to Data" about a sample size one - he was sarcastic!. 

"While I am thinking in terms of humorous determinations of sample size, I sometimes suggest we should stop at samples of size one, since otherwise variance starts to get in the way. I always have a smile, but sometimes think the audience is taking me seriously."

But the YouTube video clip he suggested was interesting - a dialog between a biologist and a statistician about the sample size. 

"This reminded me of a YouTube video clip many of you may have seen in which a scientist and statistician try to collaborate. The scientist is hung up on a sample of size three since that is what was always used. The clip is humorous, and in reflection, sad as well. Look for yourself at https://goo.gl/9qdfjK"

In my previous post "N of 1 Clinical Trial Design and its Use in Rare Disease Studies", "N of 1" design is really meant for a study with multiple crossovers in the same patient, but will at least several sets of patients - some replicates are needed. 

Real-World Evidence for Regulatory Decision Making - Some Updates

This year, real-world data and real-world evidence is the theme in every conference in the clinical trial and drug development area. Just a couple of months ago, I had discussed "Generate Real-World Data (RWD) and Real-World Evidence (RWE) for Regulatory Purposes". Last month at the annual Regulatory-Industry Statistics Workshop, the real-world evidence was again the main topic. Below are some topics (with slides) that were discussed in the workshop.

• Keynote speaking by Jacqueline Corrigan-Curay, US FDA CDER Office of Medical Policy: FDA’s Real-World Evidence Program: An Update
• Real-World Data, Machine Learning, and Causal Inference: Presentation by Jie Chen, Merck Research Laboratories
• FDA Guidance on Meta-Analysis of Safety Evaluation: Presentation by Eugenio Andraca-Carrera, FDA/CDER/OTS/OB
• Continuum Safety Evidence in Decision-Making: From RCTs to RWE: Presentation by Ed Whalen, Pfizer Inc.
• The Use of Real-World Evidence in Regulatory Decision-Making for Medical Products: How We Get There from Here:  Presentation by Mark Levenson, FDA/CDER
• The Use of RWE for Label Expansion: Presentation by Weili He, AbbVie
• The Use of RWD/RWE to Inform Clinical Trial Design: Presentation by Martin Ho, FDA/CDRH
• Adjusting diagnostic test performance for numerous confounders: Presentation by Gene Anthony Pennello, FDA
• Approaches and Challenges in Accounting for Baseline and Post-Baseline Characteristics when Comparing Two Treatments in an Observational/Non-Randomized Setting: Presentation by Joe Massaro, Boston University

There is also an upcoming conference "Real-World Evidence Conference" this November 2019 in Cambridge, MA

Where is the real-world data coming from?
The real-world data will mainly come from the claim data, the EHR (electronic health records), registry, and maybe social media data. The data from a clinical trial using the real-world device (for example, using actigraphy/accelerometry to measure the patient's function in the real world) may also be considered as real-world data - different types and more acceptable real-world data.

Even though the real-world evidence is discussed everywhere, the application and the acceptability of the real-world data are still limited. It is unlikely to have real-world data or real-world evidence to replace the clinical trials, especially the gold standard of RCT (randomized, controlled trials).  In this DIA discussion between Ms. Kunz and Ms. Mahoney, "Advancing the Use of Real-World Evidence for Regulatory Decision-Making", the potential application of real-world evidence was mentioned to be for label expansion and for fulfilling the post-marketing requirement. I would say that real-world evidence may also be applied in the regulatory decision making for ultra-rare diseases.

For real-world data, data quality is always an issue and a concern. In a recent presentation by Dr. Bob Temple, "Leveraging Randomized Designs toGenerate RWE", he discussed the areas and examples that real-world evidence was used. He also had the following comments about data quality and precision.

In a most recent article, US FDA's Temple On Real-World Evidence: 'I Find The Whole Thing Very Frustrating'. and also this article: Real-World Evidence: Sponsors Look To US FDA Drug Reviews For Potential Pitfalls.

Officials from the European Medicines Agency (EMA) said in an article published recently in Clinical Pharmacology & Therapeutics that there will need to be adequate statistical methods to extract, analyze and interpret real-world evidence before they can translate into credible evidence.

Are novel, non‐randomised analytic methods fit for decision‐making? The need for prospective, controlled and transparent validation