(Bayes) Success Run Theorem for Sample Size Estimation in Medical Device Trial

In a recent discussion about the sample size requirement for a clinical trial in a medical device field, one of my colleagues recommended an approach of using “success run theorem” to estimate the sample size. ‘Success run theorem’ may also be called ‘Bayes success run theorem’. In process validation field, it is a typical method based on a binomial distribution that leads to a defined sample size.  

Application of success run theorem depends on the reliability of the new process (or new device). In medical device trials, the reliability is the probability that an item (i.e. the device) will carry out its function satisfactorily for the stated period when used according to the specified conditions. A reliability of 95% means that a medical device will be functional without problem for 95% of times.

With the success run theorem, we will calculate the sample size so that we have 95% confidence interval to run the device without failure (reliability). Usually, people use 95% confidence interval to achieve 95% reliability. With ‘success run theorem’, the sample size can be calculated as:

                                 N = ln(1-C)/ln( R)

Where N is the sample size needed, C is the confidence interval, and R is the reliability.

With typical 95% confidence interval to achieve 95% reliability, a sample size of 57 will be needed. 

The website below contains the explanation how the success run theorem formula is derived. With C = 1 – R^(n+1), we would have N = [ln(1-C)/ln(R)] – 1, slightly different from the formula above.

 How do you derive the Success-Run Theorem from the traditional form of Bayes Theorem?

This derivation above is based on uniform prior for reliability (a conservative assumption) which assumes no information from predicate devices and the same weight to every reliability value to fall anywhere between 0 to 1.

In medical device field, devices evolve and they are constantly being improved. When we evaluate a new device or next generation device, there is usually some prior information that can be based on. Therefore, instead of uniform prior for reliability, Bayesian technique with mixture of beta priors for reliability can be applied. Using mixtures of beta priors for reliability, we will be able to incorporate historical information from predicate device to decrease the sample size requirement.

We have seen this application in the field of automotive electronics attribute testing, but have not seen any application in FDA regulatory medical device testing.

References:  

• Mark Durivage (2017) How To Establish The Number Of Runs Required For Process Validation
• Mark Durivage (2016)
• Risk-Based Approaches To Establishing Sample Sizes For Process Validation
• Mark Durivage (2014) Sample Sizes: How Many Do I Need?
• Howdo you derive the Success-Run Theorem from the traditional form of BayesTheorem?
• Kleyner, S Bhagath, M. Gasparini, and J. Robinson (1997) Bayesian Techniques to Reduce the Sample Size in Automotive Electronics Attribute Testing