A surrogate endpoint is intended to substitute for a clinical endpoint. A surrogate endpoint is expected to predict clinical benefit (or harm, or lack of benefit) based on epidemiologic, therapeutic, pathophysiologic or other scientific evidence. In clinical trials, a surrogate endpoint (or marker) is a measure of effect of a certain treatment that may correlate with a real endpoint but doesn't necessarily have a guaranteed relationship. The National Institutes of Health (USA) define surrogate endpoint as "a biomarker intended to substitute for a clinical endpoint"
Biomarkers are biological substances or features that can be used to indicate normal biological processes, disease processes, or responses to therapy. Biomarkers can be physiological indicators, such as heart rate or blood pressure, or they can be molecules in the tissues, blood, or other body fluids. For example, elevated blood levels of a protein called prostate specific antigen is a molecular biomarker for prostate cancer.
Biomarker and surrogate endpoint are often used interchangeably. However, there a subtle difference. Surrogate endpoints may not just be biomarkers and could include the imaging measurements (such as CT bone/lung densitometry, arteriogram...).
Just recently, I noticed that there are quite some works done in the area of statistical validadtion for surrogate endpoints. In the medical community, people may simply think that a biomarker can be a surrogate endpoint if the correlation between a surrogate endpoint and an established clinical endpoint are observed. However, the correlation is only one of the criteria (or requirement) for a biomarker to be a valid surrogate endpoint. To validate a surrogate endpoint, there have been a lot of discussions about the statistical approach in validating the surrogate endpoint.
in their paper titled "Surrogate end points in clinical trials: are we being misled?" (1996), Fleming and DeMets provided many examples of the surrogate endpoints and pointed out that these surrogate endpoints often fail in formal statistical validation.
The issues with surrogate endpoint is actually discussed in ICH E9 Statistical Principles for Clinical Trials
Surrogate Variables (2.2.6)
When direct assessment of the clinical benefit to the subject through observing
actual clinical efficacy is not practical, indirect criteria (surrogate variables — see
Glossary) may be considered. Commonly accepted surrogate variables are used in
a number of indications where they are believed to be reliable predictors of
clinical benefit. There are two principal concerns with the introduction of any
proposed surrogate variable. First, it may not be a true predictor of the clinical
outcome of interest. For example, it may measure treatment activity associated
with one specific pharmacological mechanism, but may not provide full information
on the range of actions and ultimate effects of the treatment, whether positive or
negative. There have been many instances where treatments showing a highly
positive effect on a proposed surrogate have ultimately been shown to be
detrimental to the subjects' clinical outcome; conversely, there are cases of
treatments conferring clinical benefit without measurable impact on proposed
surrogates. Second, proposed surrogate variables may not yield a quantitative
measure of clinical benefit that can be weighed directly against adverse effects.
Statistical criteria for validating surrogate variables have been proposed but the
experience with their use is relatively limited. In practice, the strength of the
evidence for surrogacy depends upon (i) the biological plausibility of the
relationship, (ii) the demonstration in epidemiological studies of the prognostic
value of the surrogate for the clinical outcome, and (iii) evidence from clinical
trials that treatment effects on the surrogate correspond to effects on the clinical
outcome. Relationships between clinical and surrogate variables for one product
do not necessarily apply to a product with a different mode of action for treating the
same disease.
Some key references:
1. Prentice, R. L. (1989). Surrogate endpoints in clinical trials: Definition and operational criteria. Statistics in Medicine 8 431–440
2. Freedman L, Graubard B (1992). Statistical validation of intermediate endpoints for chronic
diseases. Statistics in Medicine
3. Lin DY, Fleming TR, DeGruttola V. (1997) Estimating the proportion of treatment effect explained by a surrogate endpoint. Statistics in Medicine, 16:1515-1527
4. A framework for biomarker and surrogate endpoint in drug development by Janet Woodcock
5. Surrogate Markers - Their Role in Regulatory Decision Process
6. Statistical Validadtion of surrogate markers
7. Fleming and DeMets (1996) Surrogate End Points in Clinical Trials: Are We Being Misled? Ann Mem Med. 1996; 125:605-613
Saturday, May 23, 2009
Friday, May 15, 2009
Imaging analysis in clinical trial
1) Disease diagnosis as part of inclusion/exclusion criteria
2) Safety assessment
3) Clinical efficacy endpoint
There are many medical imaging technologies. Here are just a list of some:
1) x-ray
2) CT scan
3) MRI
4) PET scan
5) Ultrasound
6) arteriogram or angiography
7) venogram
There is benefit and risk in using the medical imaging in clinical trials. Some imaging can pose extra safety issues. For example, x-ray, CT scan, PET scan can put the study subjects at extra radiological exposure. Arteriogram and CT/A can expose the subjects to additional contrast medium or dyes which may have its own safety issue.
Medical imaging is always a surrogate endpoint. The technician plays the important role in obtaining the imaging. The standardization and calibration are always important in order to obtain the reliable data especially in longitudinal studies. The interpretation of the imaging results depend on who read the imaging. There could be substantial variation between different readers. Therefore, the central reading is very important if the medical imaging is used in clinical trial. There are quite some articles discussing the imaging in clinical trials in the Applied Clinical Trial magazine.
There are several specialty medical imaging vendors on the market. Some of them are listed below:
1) BioClinica or Bio-imaging
2) Biomedical Systems
3) Perceptive (part of Parexel)
4) Synarc
FDA and EMEA has issued several guidance on imaging used in clinical trial. For example:
This guidance discussed the clinical trials with imaging endpoints - i.e., the reading from medical imaging is used as the efficacy endpoint. Examples are: RECIST criteria for assessing the tumor size for solid tumor based on FDG-PET or MRI. Lung density measure by CT scan to assess emphysema.
2) FDA guidance Developing Imaging Drug and Biological Products,
- Part 1: Conducting Clinical Safety Assessments) (Imaging Drug Guidance Part 1) ;
- Part 2: Clinical Indications (Imaging Drug Guidance Part 2);
- Part 3: Design, Analysis and Interpretation of Clinical Studies (Imaging Drug Guidance Part 3)
3) FDA guidance on "New Contrast Imaging Indication Considerations for Devices and Approved Drug and Biological Products"
4) Developing Medical Imaging Drugs and Biologics
5) EMEA Concept Paper on "The Harmonisation and Update of the Clinical Aspects in the Authorised Conditions of Use for Radiopharmaceuticals and Other Diagnostic Medical Products"
Additional reading:
4) Developing Medical Imaging Drugs and Biologics
5) EMEA Concept Paper on "The Harmonisation and Update of the Clinical Aspects in the Authorised Conditions of Use for Radiopharmaceuticals and Other Diagnostic Medical Products"
Additional reading:
Sunday, May 03, 2009
Adjustment for multiplicity
one of the issues in statistics field is the adjustment for multiplicity - adjustment of alpha level for multiple tests. The multiplicity can arise in many different situations in clinical trials; some of them are listed below:
- Multiple arms
- Co-primary endpoints
- Multiple statistical approaches for the same endpoint
- Interim analysis
- More than one doses vs. Placebo
- Meta analysis
- Sub group analysis
There are tons of articles about the multiplicity, but there are few guidances from the regulatory bodies. While the multiplicity issues arise, the common understanding is that the adjustment needs to be made. However, there is no guidance on which approach should be used. The adjustment approach could be the very conservative approach (e.g., Bonferroni) or less conservative (e.g., Hochberg). One could evaluate the various approaches and determine which adjusmtent approach is best suited to the situation in study.
While we are still waiting for FDA's guidance on multiplicity issue (hopefully it will come out in 2009), EMEA has issued a PtC (point to consider) document on multiplicity. The document provide guidances on when an adjustment for multiplicity should be implemented.
While there are so many articles related to multiplicity, I find the following articles suitable for my taste and with practical discussions.
- Proschan and Waclawiw (2000) Practical Guidelines for Multiplicity Adjustment in Clinical Trials. Controlled Clinical Trial
- Capizzi and Zhang (1996) Testing the Hypothesis that Matters for Multiple Primary Endpoints. Drug Information Journal
- Koch and Gansky (1996) Statistical Considerations for Multiplicity in Confirmatory Protocols. Drug information Journal
- Wright (1992) Adjust p values for simutaneous inference. Biometrics
It is always useful to refer to the statistical review documents for previous NDA/BLA to see which kind of approaches have been used in drug approval process. Three approaches below seem to stand out. These three approaches are also mentioned in
- Hochberg procedure
- Bonferroni-Holm procedure
- Hierarchical order for testing null hypotheses
while not exactly the same, In a CDRH guidance on
"Clinical Investigations of Devices Indicated for the Treatment of Urinary Incontinence ", it states “The primary statistical challenge in supporting the indication for use or device performance in the labeling is in making multiple assessments of the secondary endpoint data without increasing the type 1 error rate above an acceptable level (typically 5%). There are many valid multiplicity adjustment strategies available for use to maintain the type 1 error rate at or below the specified level, three of which are listed below:
· Bonferroni procedure;
· Hierarchical closed test procedure; and
· Holm’s step-down procedure. "
Hochberg procedure is based on Hochberg's paper in 1988. It has been used in several NDA/BLA submissions. For example, in Tysabri BLA, it is stated
"Hochberg procedure for multiple comparisons was used for the evaluation of the primary endpoints. For 2 endpoints, the Hochberg procedure results in the following rule: if the maximum of the 2 p-values is less than 0.05, then both hypotheses are rejected and claim the statistical significance for both endpoints. Otherwise, if the minimum of the 2 p-values needs to be less than 0.025 for claiming the statistical significance".
Bonferroni-Holm procedure is based on Holm's paper in 1979 (Holm, S (1979): "A simple sequentially rejective multiple test procedure", Scandinavian Journal of Statistics, 6:65–70). It is a modification to the original method. This method may also be called Holm-Bonferroni approach or Bonferroni-Holm correction. This approach was employed in Flomax NDA (020579). and BLA for HFM-582 (STN 125057).
Both Holm's procedure and Hochberg's procedure are the modifications from the Bonferroni procedure. Holm's procedure is called 'step-down procedure' and Hochberg's procedure is called 'step-up procedure'. An article by Huang and Hsu titled "Hochberg's step-up method: cutting corners off Holm's step-down method" (Biometrika 2007 94(4):965-975) provided a good comparison of these two procedures.
Benjamin-Hochberg also proposed a new procedure which controls the FDR (false discovery rate) instead of controling the overall alpha level. The original paper by Benjamin and Hochberg is titled "controlling the false discovery rate: a practical and powerful approach to multiple testing" appeared in Journal of the Royal Statistical Society. it is interesting that the FDR and Benjamin-Hochberg procedure has been pretty often used in the gene identification/microarray area. A nice comparison of Bonferroni-Holm approach and Benjamin-Hochberg approach is from this website. Another good summary is the slides from 2004 FDA/Industry statistics worshop.
Hierarchical order for testing null hypotheses was cited in EMEA's guidance as
"Two or more primary variables ranked according to clinical relevance. No formal adjustment is necessary. Howeveer, no confirmatory claims can be based on variables that have a rank lower than or equal to that variable whose null hypothesis was the first that could not be rejected. "
This approach can be explained as a situation where a primary endpoint and several other secondary endpoints are defined. The highest ranked hypothesis is similar to the primary endpoint and the lower ranked endpoints are similar to the secondary endpoints.
In one of my old studies, we hypothsized the comparisons as something like below:
"A closed test procedure with the following sort order will be used for the pairwise comparisons. The second hypothesis will be tested only if the first hypothesis has been rejected, thus maintaining the overall significance level at 5%.
1. The contrast between drug 400mg and placebo (two-sided, alpha = 0.05)(H01 : mu of 400 mg = mu of placebo)
2. The contrast between drug 400 mg and a comparator (two-sided, alpha = 0.05)(H02 : mu of 400 mg = mu of the comparator) "
Friday, May 01, 2009
Understanding person-year or patient-year
When I studied the public health many years ago, in occupational health class, the term 'person year' was pretty often used. Since the length of exposure to the health hazard is different for different workers, it is necessary to calculate the person year. The total person year (summation of person year from all workers exposed to certain industry hazard) will then be used to calculate the rate (such as death rate, mortality rate,...). When same logic is used in the clinical setting or in clinical trial field, the similar term 'patient year' is used. The terms 'person year' and 'patient year' are used interchangeably.
The rates are represented as “per person-time” to provide more accurate comparisons among groups when follow-up time (i.e., patient exposure time) is not the same in all groups. Theoretically, we can express a rate of events per patient year, but the rate would be typically be a fraction or too small. In practice, the rate can be expressed as per 100, 1000, 100,000, 1 million patient-years or patient-years at risk.
“Patient-year at risk” means that the denominator of the rate calculation is ascertained by adding exposure times of all patients, where each patient’s exposure time is defined as days spent in a pre-determined time period (i.e., a year), censored only by events such as death or disenrollment, or the end of the time period. Divide the total number of days by 365 or 365.25 to get the actual year value.
“Patient-year” means that the denominator of the rate calculation is ascertained by counting all patients who are in the pre-determined time period for at least one day.
The expressions “per 100,000 patient-years at risk” and “per million patient-years” are just different ways of normalizing the rates to better present them. Thus, a hospitalization rate of 0.0000015 per patient-year, can also be expressed as 1.5 per million patient-years.
CTSpedia.org provided pretty detail explanation about the person-time (person year is just a special case of the person-time). An example of calculating death rate using patient year is illustrated from Organ Donor website.
The rate expressed in 'patient year' has been used in many different scenarios. For example, the following paragraph from a website have used 'The number of exacerbations per patient year'; 'the number of exacerbation days per patient year',...
In FDA guidance "Efficacy, Safety, and Pharmacokinetic Studies to Support Marketing of Immune Globulin Intravenous (Human) as Replacement Therapy for Primary Humoral Immunodeficiency", the rate of SBI (serious bacterial infection) is per person-year.
"The protocol should prospectively define the study analyses. We expect that the data analyses presented in the BLA will be consistent with the analytical plan submitted to the IND. Based on our examination of historical data, we believe that a statistical demonstration of a serious infection rate per person-year less than 1.0 is adequate to provide substantial evidence of efficacy. You may test the null hypothesis that the serious infection rate is greater than or equal to 1.0 per person-year at the 0.01 level of significance or, equivalently, the upper one-sided 99% confidence limit would be less than 1.0. "
"We recommend that you provide in the BLA descriptive statistics for the number of serious infection episodes per person-year during the period of study observation."
The rates are represented as “per person-time” to provide more accurate comparisons among groups when follow-up time (i.e., patient exposure time) is not the same in all groups. Theoretically, we can express a rate of events per patient year, but the rate would be typically be a fraction or too small. In practice, the rate can be expressed as per 100, 1000, 100,000, 1 million patient-years or patient-years at risk.
“Patient-year at risk” means that the denominator of the rate calculation is ascertained by adding exposure times of all patients, where each patient’s exposure time is defined as days spent in a pre-determined time period (i.e., a year), censored only by events such as death or disenrollment, or the end of the time period. Divide the total number of days by 365 or 365.25 to get the actual year value.
“Patient-year” means that the denominator of the rate calculation is ascertained by counting all patients who are in the pre-determined time period for at least one day.
The expressions “per 100,000 patient-years at risk” and “per million patient-years” are just different ways of normalizing the rates to better present them. Thus, a hospitalization rate of 0.0000015 per patient-year, can also be expressed as 1.5 per million patient-years.
CTSpedia.org provided pretty detail explanation about the person-time (person year is just a special case of the person-time). An example of calculating death rate using patient year is illustrated from Organ Donor website.
The rate expressed in 'patient year' has been used in many different scenarios. For example, the following paragraph from a website have used 'The number of exacerbations per patient year'; 'the number of exacerbation days per patient year',...
"Additionally, tiotropium significantly reduced the number of exacerbations (0.853 vs 1.051 exacerbations per patient-year; p=0.003) (1) and number of exacerbation days (mean: 12.61 vs 15.96 days per patient year; p is less than 0.001). Similarly, tiotropium significantly reduced the frequency of exacerbation related hospitalizations (0.177 vs 0.253 means hospitalizations per patient year, p=0.013)(1) and the number of hospitalization days (1.433 vs. 1.702, mean days per patient year, p=0.001) compared to placebo. In addition, a reduction in the number of treatment days (antibiotic or steroids) (p is less than 0.001) and unscheduled visits to health care providers for exacerbations (p = 0.017) were also significantly reduced with tiotropium compared to placebo."
In FDA guidance "Efficacy, Safety, and Pharmacokinetic Studies to Support Marketing of Immune Globulin Intravenous (Human) as Replacement Therapy for Primary Humoral Immunodeficiency", the rate of SBI (serious bacterial infection) is per person-year.
"The protocol should prospectively define the study analyses. We expect that the data analyses presented in the BLA will be consistent with the analytical plan submitted to the IND. Based on our examination of historical data, we believe that a statistical demonstration of a serious infection rate per person-year less than 1.0 is adequate to provide substantial evidence of efficacy. You may test the null hypothesis that the serious infection rate is greater than or equal to 1.0 per person-year at the 0.01 level of significance or, equivalently, the upper one-sided 99% confidence limit would be less than 1.0. "
"We recommend that you provide in the BLA descriptive statistics for the number of serious infection episodes per person-year during the period of study observation."
Saturday, April 25, 2009
Acronym related to Clinical trials in EU countries
In order to conduct a clinical trial in the EC, the sponsor must first submit a valid request for authorisation to the Competent Authority of the Member State where they propose to conduct the trial. This request is known as the Clinical Trial Application (CTA). The content of this application will then be assessed by the competent authority and/or the Ethics Committee to ensure that the anticipated therapeutic benefits to the patient justify any foreseeable risks before a favourable opinion is issued to allow the trial to proceed.
The safety of subjects participating in a clinical trial is the main reason behind many of the changes brought about by the Directive and thus why the need for a common system of authorization has also come about. This requirement within the pharmaceutical industry was previously only applicable to commercial products. However, this change now means that all facilities used for the manufacture or import of Investigational Medicinal Products (IMPs) will be subject to an inspection by the competent authority.
This is to ensure that the principles of Good Manufacturing Practice (GMP) as led down in Annex 13 to the EU guide to Good Manufacturing Practice are being adhered to. On the basis of this inspection, they may become licensed by the competent authority. This authorisation takes the form of a Manufacturing Authorisation for IMPs or MA for IMPs.
Yet, one additional aspect must be fulfilled in order for a facility to be granted a licence. This is the need for the manufacturer or importer to have a Qualified Person (QP) permanently and continuously at their disposal. This person will be named on the licence and will be responsible for the release of batches of clinical trial material before they can be used in a clinical trial.
Several scenarios present themselves. The first one is when the IMP has been manufactured within Europe. This is no doubt the simplest case for the QP when discharging their duties. In order to release material of European origin, they must confirm that each batch has been manufactured and checked in compliance with GMP, the Product Specification File (PSF) and the request for authorisation to conduct the trial, i.e. the CTA.Another scenario exists when a comparator product from outside the EU, with a marketing authorisation (MA) in that country is to be used as an IMP. Under such circumstances the QP can perform release, if documentation is available to certify its manufacture to standards at least equivalent to European GMP. However, in the absence of such documentation, the QP must ensure that each lot undergoes all relevant analyses, tests or checks to confirm its quality.
This can sometimes prove difficult and therefore it is important that the sponsor gives purchase of comparators due consideration. One piece of advice would be that, if possible, comparators should be sourced within Europe or from countries where Mutual Recognition Agreements are already in existence, such as Canada, Australia, New Zealand, Switzerland and Japan. These Mutual Recognition Agreements are based on trust and confidence and are therefore very useful when it comes to importing comparators, as they aim to remove barriers to trade and promote standardization of GMP.
MA: Marketing authorization
MAH: Marketing authorization holder
CA: Competent Authority
QP: Qualified Person
MRP: Mutual Recognition Process
MRA: Mutual Recognition Agreement
EMEA: European Medicines Agency
- The European Medicines Agency (EMEA) is a decentralised body of the European Union with headquarters in London. Its main responsibility is the protection and promotion of public and animal health, through the evaluation and supervision of medicines for human and veterinary use.
BPWP: blood product working party
- The Blood Products Working Party (BPWP) was set up by the CHMP to provide recommendations on all matters relating directly or indirectly to efficacy and safety aspects of blood products.
CHMP: The Committee for Medicinal Products
- The Committee for Medicinal Products for Human Use (CHMP) is responsible for preparing the Agency's opinions on all questions concerning medicinal products for human use, in accordance with Regulation (EC) No 726/2004.
NfG: Notes for Guidance
PtC: Point to Consider
The Paul-Ehrlich-Institut is an institution of the Federal Republic of Germany. It reports to the Bundesministerium für GesundheitSimilar to CBER of FDA.
(Federal Ministry of Health).
RMS: Reference Member State
Concerned Member States
Application for variation to a marketing authorisation = sNDA or sBLA
MHRA: Medicines and Healthcare products Regulatory Agency - An executive agency of the Department of Health in UK - simiar to FDA in US
SPC or SmPC: Summary of Product Characteristics - similar to Label or Package Insert in US.
- Pescription medications are regulated by governmental bodies to assure quality and appropriate use. In the US, the FDA regulates medications, and requires "labels" to be approved. "Package inserts" are written for health care providers. They contain very detailed information about different drugs. Frequently, there are also official documents for patients, called Patient Information leaflets. The manufacturers prepare this information, and the FDA approves it (sometimes after considerable discussions and negotiations!).
- In Europe, a similar process is used, with the "label" called the Summary of Product Characteristics (SPC, or SmPC). The patient-oriented document is called a "Package Leaflet" or "Patient Information Leaflet" (PILs)
IPMD: The Investigational Medicinal Product Dossier (IMPD)- similar to IND submission in US. IMPD needs to be submitted to the concerned competent authority (CA) in order to obtain the authorization of conducting the clinical trial.
CTA: Clinical Trial Application - similar to IND (investigational new drug)
NICE: The National Institute for Health and Clinical Excellence a counterpart in US is the Agency for Healthcare Research and Quality.
- NICE is a special health aurhority of the National Health Service (NHS) in England and Wales. It was set up as the National Institute for Clinical Excellence in 1999, and on 1 April 2005 joined with the Health Development Agency to become the new National Institute for Health and Clinical Excellence (still abbreviated as NICE).
- NICE publishes clinical appraisals of whether particular treatments should be considered worthwhile by the NHS. These appraisals are based primarily on cost-effectiveness.
Sunday, April 19, 2009
Risk management, pharmacoepidemiology, and pharmacovigilence
Risk Management:
Risk management is the overall and continuing process of minimizing risks throughout a product's lifecycle to optimize its benefit/risk balance. Risk information emerges continuously throughout a product's lifecycle, during both the investigation and marketing phases through both labeled and off-label uses. FDA considers risk management to be a continuous process of (1) learning about and interpreting a product's benefits and risks, (2) designing and implementing interventions to minimize a product's risks, (3) evaluating interventions in light of new knowledge that is acquired over time, and (4) revising interventions when appropriate.
Pharmacoepidemiology:
Pharmacovigilance (PVG):
Pharmacovigilance is generally regarded as all postapproval scientific and data gathering activities relating to the detection, assessment, understanding, and prevention of adverse events or any other product-related problems. This includes the use of pharmacoepidemiologic studies.
Patient registry:
The term "registry" as used in pharmacovigilance and pharmacoepidemiology is often given different meanings. For the purpose of this concept paper, we are defining a registry as a systematic collection of defined events or product exposures in a defined patient population for a defined period of time. Through the creation of registries, a sponsor can monitor for safety signals identified from spontaneous case reports, literature reports, or other sources, and evaluate factors that affect the risk of adverse outcomes, such as dose, timing of exposure, or other patient characteristics.
REMS: Risk Evaluation and Mitigation Strategy
Risk management is the overall and continuing process of minimizing risks throughout a product's lifecycle to optimize its benefit/risk balance. Risk information emerges continuously throughout a product's lifecycle, during both the investigation and marketing phases through both labeled and off-label uses. FDA considers risk management to be a continuous process of (1) learning about and interpreting a product's benefits and risks, (2) designing and implementing interventions to minimize a product's risks, (3) evaluating interventions in light of new knowledge that is acquired over time, and (4) revising interventions when appropriate.
Pharmacoepidemiology:
pharmacoepidemiology is the study of the utilization and effects of drugs in large numbers of patients. It can be viewed as an epidemiological discipline with particular focus on drugs.The process of identifying and responding to safety issues about drugs.
Pharmacovigilance (PVG):
Pharmacovigilance is generally regarded as all postapproval scientific and data gathering activities relating to the detection, assessment, understanding, and prevention of adverse events or any other product-related problems. This includes the use of pharmacoepidemiologic studies.
Patient registry:
The term "registry" as used in pharmacovigilance and pharmacoepidemiology is often given different meanings. For the purpose of this concept paper, we are defining a registry as a systematic collection of defined events or product exposures in a defined patient population for a defined period of time. Through the creation of registries, a sponsor can monitor for safety signals identified from spontaneous case reports, literature reports, or other sources, and evaluate factors that affect the risk of adverse outcomes, such as dose, timing of exposure, or other patient characteristics.
REMS: Risk Evaluation and Mitigation Strategy
A Risk Evaluation and Mitigation Strategy (REMS) is a strategy to manage a known or potential serious risk associated with a drug or biological product. A REMS will be required if FDA finds that a REMS is necessary to ensure that the benefits of the drug or biological product outweigh the risks of the product, and FDA notifies the sponsor. A REMS can include a Medication Guide, Patient Package Insert, a communication plan, elements to assure safe use, and an implementation system, and must include a timetable for assessment of the REMS. Some drug and biological products that previously were approved/licensed with risk minimization action plans (RiskMAPs) will now be deemed to have REMS.
For more information, see FDA's website about FDAAA.
also, the following website may be useful:
Sunday, April 12, 2009
Effect Size
Effect size (ES) is a name given to a family of indices that measure the magnitude of a treatment effect. Unlike significance tests, these indices are independent of sample size. Effect size has been frequently linked to the power analysis (or sample size calculation) and Meta analysis.
The concept of effect size seems to come from Cohen's book "Statistical Power Analysis for the Behavioral Sciences". Effect size is not just for the continuous variable, it could also be for rates and proportions, and other type of data.
The following weblinks provide good summaries on effect size:
The concept of effect size seems to come from Cohen's book "Statistical Power Analysis for the Behavioral Sciences". Effect size is not just for the continuous variable, it could also be for rates and proportions, and other type of data.
The following weblinks provide good summaries on effect size:
- Effect size (ES)
- Effect Size Substantive Interpretation Guidelines: Issues in the Interpretation of Effect Sizes
- Power Analysis, Statistical Significance, & Effect Size
- Effect size on Wikipedia
Recently, I came across a paper that described the use of effect size as Benchmarks for Interpreting Change - one of many ways to determine the sensitivity of the measurement and subsequently the minimal clinically important difference (MCID) or minimal important difference (MID).
In a paper by Kazis LE, Anderson JJ, Meenan RF (Effect sizes for interpreting changes in health status. Med Care 1989;27:S178-S189), they described the following:
"Effect size as used in this study is calcu-lated by taking the difference between the means before treatment and after treatment and dividing it by the standard deviation of the same measure before treatment. This method of calculating effect sizes can be expressed mathematically as ES = (mi - m2)/sl, where m, is the pretreatment mean, m2 the posttreatment mean, and s, the pretreatment standard deviation. In this instance the before-treatment scores are used as a proxy for control group scores. This approach treats the effect size as a standard measure of change in a "before and after study" context. We are interested in the magnitude or size of the change rather than statistical significance, so we use the standard deviation at baseline rather than the standard deviation of the differ-ence between the means.8 Effect sizes can be used to translate changes in health status into a standard unit of measurement that will provide a clearer interpretation of the results. This can be ac-complished by using effect sizes as bench-marks for measuring changes or as a means for taking comparisons between measures in the same study or across studies. "
Here the effect size is not to compare the two treatmetn groups, rather compare the differences pre and post. The formula for effect size can be explicitly rewritten to represent the mean change from pre treatment to the post treatment divided by the standard deviation of the baseline measures (effect size = (mui - mu0/SDmu0; mui = mean value of the post-baseline measure; mu0 = mean value at baseline).
If we calculate the effect size for both treatment group and placebo group, we should expect a very small effect size for Placebo group and a rather large effect size for treatment group - an indicator of a good measurement.
Sunday, April 05, 2009
Least squares means (marginal means) vs. means
If you work with SAS, you probably heard and used the term 'least squares means' very often. Least squares means (LS Means) are actually a sort of SAS jargon. Least square means is actually referred to as marginal means (or sometimes EMM - estimated marginal means). In an analysis of covariance model, they are the group means after having controlled for a covariate (i.e. holding it constant at some typical value of the
The mean value for Treatment A is simply the summation of all measures divided by the total number of observations (Mean for treatment A = 24/5 = 4.8); similarly the Mean for treatment B = 26/5 = 5.2. Mean for treatmeng A > Mean for treatment B.
After the mean for each cell is calculated, the least squares means are simply the average of these means. For treatment A, the LS mean is (3+7.5)/2 = 5.25; for treatment B, it is (5.5+5)/2=5.25. The LS Mean for both treatment groups are identical.
covariate, such as its mean value).
I often find that it is neccessary to use a very simple example to illulatrate the difference between LS Means and Means to my non-statistician colleagues. I made up the data in Table 1 above. There are two treatment groups (treatment A and treatment B) that are measured at two centers (Center 1 and Center 2).
The mean value for Treatment A is simply the summation of all measures divided by the total number of observations (Mean for treatment A = 24/5 = 4.8); similarly the Mean for treatment B = 26/5 = 5.2. Mean for treatmeng A > Mean for treatment B.
Table 2 shows the calculation of least squares means. First step is to calculate the means for each cell of treatment and center combination. The mean 9/3=3 for treatment A and center 1 combination; 7.5 for treatment A and center 2 combination; 5.5 for treatment B and center 1 combination; and 5 for treatment B and center 2 combination.
After the mean for each cell is calculated, the least squares means are simply the average of these means. For treatment A, the LS mean is (3+7.5)/2 = 5.25; for treatment B, it is (5.5+5)/2=5.25. The LS Mean for both treatment groups are identical.
It is easy to show the simple calculation of means and LS means in the above table with two factors. In clinical trials, the statistical model often needs to be adjusted for multiple factors including both categorical (treatment, center, gender) and continuous covariates (baseline measures). The calculation of LS mean is not easy to demonstrate. However, the LS mean should be used when the inferential comparison needs to be made. Typically, the means and LS means should point to the same direction (while with different values) for treatment comparison. Occasionally, they could point to the different directions (treatment A better than treatment B according to mean values; treatment B better than treatment A according to LS Mean).
SAS procedure GLM has a nice discussion about the comparison of Least Square Means vs. Means. A small article "Means vs LS Means and Type I vs Type III Sum of Squares"by Dan may also help.
Sunday, March 29, 2009
Jadad Scale to assess the quality of clinical trials
In a cost-effectiveness assessment report, a detail descriptions were provided for the approaches in choosing the clinical trial data for meta analysis. After many clinical trials are selected, a 'Jadad Scale' was used to assess the quality of clinical trials.
Jadad Scale sometimes known as Jadad scoring or the Oxford quality scoring system, is a procedure to independently assess the methodological quality of a clincal trial. It is the most widely used such assessment in the world.
The Jadad score was used as the 'gold standard' to assess the methodological quality of studies. This validated score lies in the range 0-5. Studies are scored according to the presence of three key methodological features of randomization, blinding and accountability of all patients, including withdrawals.
According to NIH website Appendix E: The Jadad Score
A Method for assessing the quality of controlled clinical trials
Basic Jadad Score is assessed based on the answer to the following 5 questions.
The maximum score is 5.
Question Yes No
1. Was the study described as random? 1 0
2. Was the randomization scheme described and appropriate? 1 0
3. Was the study described as double-blind? 1 0
4. Was the method of double blinding appropriate? (Were both the patient and the assessor appropriately blinded?) 1 0
5. Was there a description of dropouts and withdrawals? 1 0
Quality Assessment Based on Jadad Score
Range of Score Quality
0–2 Low
3–5 High
Wikipedia has a pretty good summary of the use of Jadad Scale.
Jadad Scale has been frequently used as a study selection criteria when the literature review or meta analysis are performed.
References:
1. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17:1-12.
Jadad Scale sometimes known as Jadad scoring or the Oxford quality scoring system, is a procedure to independently assess the methodological quality of a clincal trial. It is the most widely used such assessment in the world.
The Jadad score was used as the 'gold standard' to assess the methodological quality of studies. This validated score lies in the range 0-5. Studies are scored according to the presence of three key methodological features of randomization, blinding and accountability of all patients, including withdrawals.
According to NIH website Appendix E: The Jadad Score
A Method for assessing the quality of controlled clinical trials
Basic Jadad Score is assessed based on the answer to the following 5 questions.
The maximum score is 5.
Question Yes No
1. Was the study described as random? 1 0
2. Was the randomization scheme described and appropriate? 1 0
3. Was the study described as double-blind? 1 0
4. Was the method of double blinding appropriate? (Were both the patient and the assessor appropriately blinded?) 1 0
5. Was there a description of dropouts and withdrawals? 1 0
Quality Assessment Based on Jadad Score
Range of Score Quality
0–2 Low
3–5 High
Wikipedia has a pretty good summary of the use of Jadad Scale.
Jadad Scale has been frequently used as a study selection criteria when the literature review or meta analysis are performed.
References:
1. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17:1-12.
Stratified randomization to achieve the balance of treatment assignment within each strata
Stratified randomization refers to the situation in which strata are constructed based on values of prognostic variables or baseline covariates and a randomization scheme is performed separately within each stratum. One misconception is to think that the stratified randomization is going to require the equal number of subjects for each strata.
For example, suppose that in a two-arm, parallel design study, we would like to stratify the randomization for age group (<18 versus >=18 years old). But we don't know how many subjects in each age group we could enroll. The purpose is to make sure that within each age group, there are equal numbers of subjects assigned to treatment A or treatment B.
After the study, there may be quite different total number of subjects in each age group, but within each age group, there should be approximately equal number of subjects in treatment A or treatment B.
The strata size usually vary (maybe there are relatively fewer young males and young females with the disease of interest). The objective of stratified randomization is to ensure balance of the treatment groups with respect to the various combinations of the prognostic variables. Simple randomization will not ensure that these groups are balanced within these strata so permuted blocks are used within each stratum are used to achieve balance.
When the stratified randomization is utilized, the # of stratification factors is typically limited to 1 or 2. The number of strata is exponentially increased if too many randomization factors are included. For example, if we have 4 stratification factors and each factor has two levels, then the # of strata = 2^4 = 16 strata, which is not practical.
If there are too many strata in relation to the target sample size, then some of the strata will be empty or sparse. This can be taken to the extreme such that each stratum consists of only one patient each, which in effect would yield a similar result as simple randomization. Keep the number of strata used to a minimum for good effect.
I have also seen a trial to require the equal number of subjects for each strata and with each strata, then equal number of subjects assigned to two treatment groups. In a trial to study the IBS (irritable bowel Syndrome), the protocol required the equal number of subjects in two type of IBSs (IBS-C vs. IBS-M). Within IBS-C or IBS-M group, there should be equal number of subjects assigned to treatment A or treatment B. The things turned out not nice because there were a lot of more subjects with IBS-C than IBS-M. During the study, while enrollment target for IBS-C was achieved, there was still a lot of IBS-M subjects to be enrolled.
IBS-C=Irritable Bowel Syndrome (constipation dominant)
IBS-M=Irritable Bowel Syndrome (mixed - constipation and diarrria)
For example, suppose that in a two-arm, parallel design study, we would like to stratify the randomization for age group (<18 versus >=18 years old). But we don't know how many subjects in each age group we could enroll. The purpose is to make sure that within each age group, there are equal numbers of subjects assigned to treatment A or treatment B.
After the study, there may be quite different total number of subjects in each age group, but within each age group, there should be approximately equal number of subjects in treatment A or treatment B.
The strata size usually vary (maybe there are relatively fewer young males and young females with the disease of interest). The objective of stratified randomization is to ensure balance of the treatment groups with respect to the various combinations of the prognostic variables. Simple randomization will not ensure that these groups are balanced within these strata so permuted blocks are used within each stratum are used to achieve balance.
When the stratified randomization is utilized, the # of stratification factors is typically limited to 1 or 2. The number of strata is exponentially increased if too many randomization factors are included. For example, if we have 4 stratification factors and each factor has two levels, then the # of strata = 2^4 = 16 strata, which is not practical.
If there are too many strata in relation to the target sample size, then some of the strata will be empty or sparse. This can be taken to the extreme such that each stratum consists of only one patient each, which in effect would yield a similar result as simple randomization. Keep the number of strata used to a minimum for good effect.
I have also seen a trial to require the equal number of subjects for each strata and with each strata, then equal number of subjects assigned to two treatment groups. In a trial to study the IBS (irritable bowel Syndrome), the protocol required the equal number of subjects in two type of IBSs (IBS-C vs. IBS-M). Within IBS-C or IBS-M group, there should be equal number of subjects assigned to treatment A or treatment B. The things turned out not nice because there were a lot of more subjects with IBS-C than IBS-M. During the study, while enrollment target for IBS-C was achieved, there was still a lot of IBS-M subjects to be enrolled.
IBS-C=Irritable Bowel Syndrome (constipation dominant)
IBS-M=Irritable Bowel Syndrome (mixed - constipation and diarrria)
Saturday, March 28, 2009
Too good to be true?
Typically, the regulatory authority requires two pivotal studied to demonstrate the efficacy. If the results from two studies show the conflicting or inconsistent results, the evidence for efficacy may be considered as not convincing.
On the other side, if two studies show the results almost identical, it could raise the issue with regulatory reviewers for suspicious fraud. In the most recent ASA's biopharmaceutical report, two examples were discussed.
NDA 022145 Merck's Isentress
Nearly identical results were observed in the investigational treatment group in two pivotal phase III trials for the applicant’s primary efficacy endpoint.
As part of the data verification process, the statistical review team requested copies of original source documents (laboratory reports) for HIV RNA data from the four sites that were inspected, from the site with the largest number of patients and from an additional site that had highly statistically significant results in favor of the investigational drug.
Because the applicant used an IVRS, there were no fixed randomization lists available prior to enrollment of the patients in the trial and no treatment codes available in envelopes at the sites that DSI inspected. Therefore the statistical review team also requested that copies of original source documents for treatment randomization schedules be sent directly to the FDA from the external vendors. In addition, the statistical reviewer requested the applicant’s standard operating procedures for randomization schedule generation and certification from the external vendors that the randomization code documents were obtained from the original electronic file sent to the vendors from the applicant prior to study initiation. A sample of treatment codes and laboratory data were compared to corresponding values in the SAS data sets and appeared to match.
Of note, in this NDA, two pivotal studies were allowed to be combined. The final assessment is based on the integrated summary of efficacy (ISE). It appears that the dynamic randomization was used in these two studies even though there was no detail description about the randomizaton procedure (ie, dynamic allocation for baseline covariate or dynamic allocaton for response?)
GSK's Relenza (NDA021036)
Two phase III studies assessed post-exposure prophylaxis in household contacts of an index case of influenza. In the first household study, the index case was treated while the index case was untreated in the second study. The primary efficacy endpoint for the two phase III household prophylaxis studies was the proportion of households with at least one previously uninfected household member who contracted symptomatic, laboratory-confirmed influenza.
Nearly identical rates were observed for the primary efficacy endpoint in the two household studies. Such a high degree of coincidence is rare.
Of note, the biopharmaceutical report is a quarterly report by biopharmaceutical section under american statistical association. Unfortunately, the report was not updated on their website. I have to put the report under a temporary web location.
On the other side, if two studies show the results almost identical, it could raise the issue with regulatory reviewers for suspicious fraud. In the most recent ASA's biopharmaceutical report, two examples were discussed.
NDA 022145 Merck's Isentress
Nearly identical results were observed in the investigational treatment group in two pivotal phase III trials for the applicant’s primary efficacy endpoint.
As part of the data verification process, the statistical review team requested copies of original source documents (laboratory reports) for HIV RNA data from the four sites that were inspected, from the site with the largest number of patients and from an additional site that had highly statistically significant results in favor of the investigational drug.
Because the applicant used an IVRS, there were no fixed randomization lists available prior to enrollment of the patients in the trial and no treatment codes available in envelopes at the sites that DSI inspected. Therefore the statistical review team also requested that copies of original source documents for treatment randomization schedules be sent directly to the FDA from the external vendors. In addition, the statistical reviewer requested the applicant’s standard operating procedures for randomization schedule generation and certification from the external vendors that the randomization code documents were obtained from the original electronic file sent to the vendors from the applicant prior to study initiation. A sample of treatment codes and laboratory data were compared to corresponding values in the SAS data sets and appeared to match.
Of note, in this NDA, two pivotal studies were allowed to be combined. The final assessment is based on the integrated summary of efficacy (ISE). It appears that the dynamic randomization was used in these two studies even though there was no detail description about the randomizaton procedure (ie, dynamic allocation for baseline covariate or dynamic allocaton for response?)
GSK's Relenza (NDA021036)
Two phase III studies assessed post-exposure prophylaxis in household contacts of an index case of influenza. In the first household study, the index case was treated while the index case was untreated in the second study. The primary efficacy endpoint for the two phase III household prophylaxis studies was the proportion of households with at least one previously uninfected household member who contracted symptomatic, laboratory-confirmed influenza.
Nearly identical rates were observed for the primary efficacy endpoint in the two household studies. Such a high degree of coincidence is rare.
Of note, the biopharmaceutical report is a quarterly report by biopharmaceutical section under american statistical association. Unfortunately, the report was not updated on their website. I have to put the report under a temporary web location.
Wednesday, March 18, 2009
Should expected clinical outcomes of the disease under study, which are efficacy endpoints, be reported as AEs/SAEs?
The paragraphs below are from the following website:
http://firstclinical.com/journal/2008/0806_GCP35.pdf
Some protocols instruct investigators to record and report all untoward events that occur
during a study as AEs/SAEs, which could include common symptoms of the disease under
study and/or other expected clinical outcomes. This approach enables frequency
comparisons of all events between treatment groups, but can make event recording in the
CRF burdensome, result in more expedited reports from investigators to sponsors, and fill
safety databases with many untoward events that most likely have no relationship to study
treatment and that could obscure signal identification.
In some clinical trials, disease symptoms and/or other expected clinical outcomes
associated with the disease under study, which might technically meet the ICH definition of
an AE or SAE, are collected and assessed as efficacy parameters rather than safety
parameters. An example might be severity scoring of prospectively defined disease
symptoms at each clinic visit during a rheumatoid arthritis study. The hypothesis underlying
this approach is that the study treatment will have a positive impact on disease symptoms.
If prospectively defined clinical outcomes, such as symptoms of a studied chronic disease or
death due to disease progression in an oncology trial, are to be assessed as efficacy
endpoints and not as AEs/SAEs, the methods for recording and analyzing these data should
be clearly described in the protocol. In addition, sponsors are advised to consult with
applicable regulatory authorities to ensure that safety reporting instructions in protocols are
acceptable, especially if certain clinical outcomes are to be excluded from traditional AE/SAE
reporting.
In high morbidity/mortality trials, independent data monitoring committees (IDMC)
generally monitor all acquired AE/SAE and clinical outcomes data to assess benefit and risk
on an ongoing basis. A reviewing IDMC could halt a trial if there was significant
improvement in pre-specified clinical outcomes in the treatment group compared to the
control group. It is also possible that a study treatment might unexpectedly worsen prespecific
disease symptoms and/or other clinical outcomes that are being assessed as
efficacy parameters.1
Reference
1. “Good Clinical Practice: A Question & Answer Reference Guide”, Barnett International,
2007, #9.10 p. 215
Source
“Good Clinical Practice: A Question & Answer Reference Guide 2007,” is available for $39.95
at http://www.barnettinternational.com/
http://firstclinical.com/journal/2008/0806_GCP35.pdf
Some protocols instruct investigators to record and report all untoward events that occur
during a study as AEs/SAEs, which could include common symptoms of the disease under
study and/or other expected clinical outcomes. This approach enables frequency
comparisons of all events between treatment groups, but can make event recording in the
CRF burdensome, result in more expedited reports from investigators to sponsors, and fill
safety databases with many untoward events that most likely have no relationship to study
treatment and that could obscure signal identification.
In some clinical trials, disease symptoms and/or other expected clinical outcomes
associated with the disease under study, which might technically meet the ICH definition of
an AE or SAE, are collected and assessed as efficacy parameters rather than safety
parameters. An example might be severity scoring of prospectively defined disease
symptoms at each clinic visit during a rheumatoid arthritis study. The hypothesis underlying
this approach is that the study treatment will have a positive impact on disease symptoms.
If prospectively defined clinical outcomes, such as symptoms of a studied chronic disease or
death due to disease progression in an oncology trial, are to be assessed as efficacy
endpoints and not as AEs/SAEs, the methods for recording and analyzing these data should
be clearly described in the protocol. In addition, sponsors are advised to consult with
applicable regulatory authorities to ensure that safety reporting instructions in protocols are
acceptable, especially if certain clinical outcomes are to be excluded from traditional AE/SAE
reporting.
In high morbidity/mortality trials, independent data monitoring committees (IDMC)
generally monitor all acquired AE/SAE and clinical outcomes data to assess benefit and risk
on an ongoing basis. A reviewing IDMC could halt a trial if there was significant
improvement in pre-specified clinical outcomes in the treatment group compared to the
control group. It is also possible that a study treatment might unexpectedly worsen prespecific
disease symptoms and/or other clinical outcomes that are being assessed as
efficacy parameters.1
Reference
1. “Good Clinical Practice: A Question & Answer Reference Guide”, Barnett International,
2007, #9.10 p. 215
Source
“Good Clinical Practice: A Question & Answer Reference Guide 2007,” is available for $39.95
at http://www.barnettinternational.com/
Adverse events (AE), treatment emergent adverse events (TEAE), and adverse drug reaction (ADR)
There is some debate and inconsistencies regarding the definition of Adverse Drug Reactions. If you call it an adverse event, you may not have a culprit drug in mind, whereas calling it an adverse drug reaction is already linking it to a suspected drug. Regardless of whether or not there is a suspected drug, an AE or an ADR is commonly defined as any adverse change in health or un-desired "side-effect" that occurs in a person while on a medical treatment (for example, drug or device) or within a pre-specified period after treatment is complete. Not every adverse event is causally related to the treatment or test being studied. However, regardless of causality, people who experienced adverse reactions, or their doctors, are encouraged to report these events to the FDA or the relevant regulatory authority in the country where the drug or device is registered.
Adverse event (AE) is any untoward medical occurrence including:
Adverse events is typically collected after signing the informed consent form and could be related or unrelated to the study drug.
Adverse drug reaction (ADR) is defined as:
The difference between AE and ADR is that AE event does not imply causality, but for ADR, a causal rule is suspected.
Another confusion is about the term 'treatment-emergent adverse event (TEAE)'. A treatment-emergent adverse event is defined as any event not present prior to the initiation of the treatments or any event already present that worsens in either intensity or frequency following exposure to the treatments. Since the starting point for AE collection is the signing of the informed consent, not the start of the study treatment, there are some adverse events occurred prior to the initiation of the study treatment. These AEs may be called "baseline-emergent adverse event" which defined as any event which occurs or worsens during the staged screening process (after informed consent) including the randomization visit. It is common to have separate summaries for AEs occurred piror to the initiation of the treatment and AEs occurred after the initiation of the treatment (ie, summary of treatment emergent adverse events).
I was asked about a programming practice to define the TEAE used in some companies. For any AE with onset date/time after the first study drug administration date/time,they compare if there is a same AE with the same severity. If yes, AE is not counted as TEAE (even though the onset date/time is after the study drug administration). For example, a subject has a mild headache 30 days after using the study medication and subjects also has a mild headache event before using the study medication,the programming will identify this event as non treatment emergent. However I think this is wrong these are two distinct events and the second one should be counted as treatment emergent AE.
The TEAE is different from the drug-related adverse events. While the treatment emergent AEs refers to adverse events temporally related to the study treatment, the drug-related AEs refers to the causality assessment by the investigator.
Adverse event (AE) is any untoward medical occurrence including:
- undesirable signs & symptoms
- disease or accidents
- abnormal lab finding (leading to dose reduction/discontinuation/intervention)
Adverse events is typically collected after signing the informed consent form and could be related or unrelated to the study drug.
Adverse drug reaction (ADR) is defined as:
- For approved pharmaceutical product: a noxious and unintended response at doses normally used or tested in humans;
- for a new unregistered pharmaceutical product: a noxious and unintended response at any dose.
The difference between AE and ADR is that AE event does not imply causality, but for ADR, a causal rule is suspected.
Another confusion is about the term 'treatment-emergent adverse event (TEAE)'. A treatment-emergent adverse event is defined as any event not present prior to the initiation of the treatments or any event already present that worsens in either intensity or frequency following exposure to the treatments. Since the starting point for AE collection is the signing of the informed consent, not the start of the study treatment, there are some adverse events occurred prior to the initiation of the study treatment. These AEs may be called "baseline-emergent adverse event" which defined as any event which occurs or worsens during the staged screening process (after informed consent) including the randomization visit. It is common to have separate summaries for AEs occurred piror to the initiation of the treatment and AEs occurred after the initiation of the treatment (ie, summary of treatment emergent adverse events).
I was asked about a programming practice to define the TEAE used in some companies. For any AE with onset date/time after the first study drug administration date/time,they compare if there is a same AE with the same severity. If yes, AE is not counted as TEAE (even though the onset date/time is after the study drug administration). For example, a subject has a mild headache 30 days after using the study medication and subjects also has a mild headache event before using the study medication,the programming will identify this event as non treatment emergent. However I think this is wrong these are two distinct events and the second one should be counted as treatment emergent AE.
The TEAE is different from the drug-related adverse events. While the treatment emergent AEs refers to adverse events temporally related to the study treatment, the drug-related AEs refers to the causality assessment by the investigator.
Friday, March 06, 2009
What is the easiest way to start a meta analysis?
I used Revman program to do my meta analysis and generated the nice Forest plot two years ago. It worked for me very well at that time. Revman is a free program developed for Cochrane Review - the most reliable reviews for evidence-based medicine.
http://community.cochrane.org/tools/review-production-tools/revman-5
Read the instruction and tutorial about how to use this program. The algorithm used in this program is described in the attached file (following the weblink below).
http://community.cochrane.org/tools/review-production-tools/revman-5/resources
http://community.cochrane.org/sites/default/files/uploads/inline-files/RevMan_5.3_User_Guide.pdf
One of the authors Julian Higgins, is one of the speakers in last year’s Meta analysis workshop sponsored by SAMSI. His topic then is titled "Practical obstables in Meta Analysis".
Since I am a heavy SAS user, I also try to do meta analysis in SAS. The following references may be useful:
http://community.cochrane.org/tools/review-production-tools/revman-5
Read the instruction and tutorial about how to use this program. The algorithm used in this program is described in the attached file (following the weblink below).
http://community.cochrane.org/tools/review-production-tools/revman-5/resources
http://community.cochrane.org/sites/default/files/uploads/inline-files/RevMan_5.3_User_Guide.pdf
One of the authors Julian Higgins, is one of the speakers in last year’s Meta analysis workshop sponsored by SAMSI. His topic then is titled "Practical obstables in Meta Analysis".
Since I am a heavy SAS user, I also try to do meta analysis in SAS. The following references may be useful:
- The SAS book "Integrating Results through Meta-Analytic Review Using SAS Software"
- A book titled "Conducting Meta-Analysis Using SAS"
- Some Meta Analysis Stuff collected by David Wilson
Biosimilar, Follow-up Biologics, Biogenerics, and Generic Biologics
Biosimilars or Follow-on biologics are terms used to describe officially approved new versions of innovator biopharmaceutical products, following patent expiry.
Unlike the more common "small-molecule" drugs, biologics generally exhibit high molecular complexity, and may be quite sensitive to manufacturing process changes. The follow-on manufacturer does not have access to the originator's molecular clone and original cell bank, nor to the exact fermentation and purification process. Finally, nearly undetectable differences in impurities and/or breakdown products are known to have serious health implications. This has created a concern that copies of biologics might perform differently than the original branded version of the drug. However, similar concerns also apply to any production changes by the maker of the original branded version. So new versions of biologics are not authorized in the US or the European Union through the simplified procedures allowed for small molecule generics.
While the term 'biosimilar' or 'follow-on biologics' are getting popular, other terms may also be used in one way or another. Other terms include 'biogenerics', 'generic biologics',...
The Obama administration supports the use and introduction of generic drugs into the market. In his new budget proposal, Obama calls for generic biotech drugs (see CNBC news or forbes news).
on November 21, 2008, FTC held a Roundtable on Follow-on Biologic Drugs: Framework For Competition and Continued Innovation. This workshop signals continuing interest in the issue. The trascript and the videos are available from the website.
Some other readings:
Unlike the more common "small-molecule" drugs, biologics generally exhibit high molecular complexity, and may be quite sensitive to manufacturing process changes. The follow-on manufacturer does not have access to the originator's molecular clone and original cell bank, nor to the exact fermentation and purification process. Finally, nearly undetectable differences in impurities and/or breakdown products are known to have serious health implications. This has created a concern that copies of biologics might perform differently than the original branded version of the drug. However, similar concerns also apply to any production changes by the maker of the original branded version. So new versions of biologics are not authorized in the US or the European Union through the simplified procedures allowed for small molecule generics.
While the term 'biosimilar' or 'follow-on biologics' are getting popular, other terms may also be used in one way or another. Other terms include 'biogenerics', 'generic biologics',...
The Obama administration supports the use and introduction of generic drugs into the market. In his new budget proposal, Obama calls for generic biotech drugs (see CNBC news or forbes news).
on November 21, 2008, FTC held a Roundtable on Follow-on Biologic Drugs: Framework For Competition and Continued Innovation. This workshop signals continuing interest in the issue. The trascript and the videos are available from the website.
Some other readings:
- Schellekens's paper titled "When biotech proteins go off-patent" (trend in biotechnology 2006)
- Amgen CEO assess the generic threat
- Two house members introduce bill to allow FDA to approve generic biotech drugs
- Future of biotechnology (foxbusiness.com)
Sunday, March 01, 2009
iDMC, iSTAT, iDM, and more
I guess that the iDMC (stands for independent Data Monitoring Committee) has been used for a while, however, I first time heard the terms iSTAT and iDM in the recent data monitoring committee conference. iSTAT stands for independent Statistician and iDM stands for independent Data Management.
To support the iDMC who could review the interim data during the study, an independent statistical programming team is typically needed. Within the same organization (sponsor or CRO), there could be two teams: one is the study team and is always blinded to the study treatment (prior to the study unblinding) and one is the independent team that could have access to the randomization codes and prepare the unblinded interim information for iDMC.
Currently there are many different structures in arranging the iDMC operation with statistical support. The iSTAT could be with the sponsor, with CROs (contract research organization), or ARO (academic research organization). Each modol has its own pros and cons.
IS (independent statistician). See the talk about Pat O'meara
IDC (independent data center)
To support the iDMC who could review the interim data during the study, an independent statistical programming team is typically needed. Within the same organization (sponsor or CRO), there could be two teams: one is the study team and is always blinded to the study treatment (prior to the study unblinding) and one is the independent team that could have access to the randomization codes and prepare the unblinded interim information for iDMC.
Currently there are many different structures in arranging the iDMC operation with statistical support. The iSTAT could be with the sponsor, with CROs (contract research organization), or ARO (academic research organization). Each modol has its own pros and cons.
IS (independent statistician). See the talk about Pat O'meara
IDC (independent data center)
Saturday, February 28, 2009
Liability and Indemnification of data monitoring committee members
As stated in Demets' paper "Liability issues for data monitoring committee members" (Clinical
Trials 2004; 1: 525–531): In randomized clinical trials, a data monitoring committee (DMC) is often appointed to review interim data to determine whether there is early convincing evidence of intervention benefit, lack of benefit or harm to study participants. Because DMCs
bear serious responsibility for participant safety, their members may be legally liable
for their actions.
With increasing DMC monitoring in clinical trials, the liability and indemnification issues are the topic of the recent data monitoring committee conference. In the situation where a study was terminated based on DMC's suggestion, the study participants could file lawsuit on either the DMC members or the sponsor for not doing the diligent work to stop the trial or stop the trial sooner enough. For example, Pfizer was sued for its Torcetrapib trial even though Pfizer is cleared of any wrongdoing. Recent events (eg Cox-IIs, Vioxx) have raised the potential for litigation and DMC members have been gotten a subpoena. For protection, DMC charters for industry trials now often cover indemnification clauses.
However, there is no indemnification yet for government-sponsored trials. For example, in NCI's guidance, it is specified "The government is prohibited by statute from indemnifying any party without specific legislative authority and consultation with the United States Department of Justice. Government liability for its own actions is usually limited by the Federal Tort Claims Act."
So what is 'indemnification'?
According to Wikipedia, "An indemnity is a sum paid by A to B by way of compensation for a particular loss suffered by B. The indemnifying party (A) may or may not be responsible for the loss suffered by the indemnified party (B). Forms of indemnity include cash payments, repairs, replacement, and reinstatement."
In the United States, Indemnification is a legal document laying down the legal protection or exemption from liability for compensation or damages from a third party, investigator and/or hospital or institution from claims made by the study subject (or relatives) that harm
was caused to the subject as a result of participation in the clinical trial.
Trials 2004; 1: 525–531): In randomized clinical trials, a data monitoring committee (DMC) is often appointed to review interim data to determine whether there is early convincing evidence of intervention benefit, lack of benefit or harm to study participants. Because DMCs
bear serious responsibility for participant safety, their members may be legally liable
for their actions.
With increasing DMC monitoring in clinical trials, the liability and indemnification issues are the topic of the recent data monitoring committee conference. In the situation where a study was terminated based on DMC's suggestion, the study participants could file lawsuit on either the DMC members or the sponsor for not doing the diligent work to stop the trial or stop the trial sooner enough. For example, Pfizer was sued for its Torcetrapib trial even though Pfizer is cleared of any wrongdoing. Recent events (eg Cox-IIs, Vioxx) have raised the potential for litigation and DMC members have been gotten a subpoena. For protection, DMC charters for industry trials now often cover indemnification clauses.
However, there is no indemnification yet for government-sponsored trials. For example, in NCI's guidance, it is specified "The government is prohibited by statute from indemnifying any party without specific legislative authority and consultation with the United States Department of Justice. Government liability for its own actions is usually limited by the Federal Tort Claims Act."
So what is 'indemnification'?
According to Wikipedia, "An indemnity is a sum paid by A to B by way of compensation for a particular loss suffered by B. The indemnifying party (A) may or may not be responsible for the loss suffered by the indemnified party (B). Forms of indemnity include cash payments, repairs, replacement, and reinstatement."
In the United States, Indemnification is a legal document laying down the legal protection or exemption from liability for compensation or damages from a third party, investigator and/or hospital or institution from claims made by the study subject (or relatives) that harm
was caused to the subject as a result of participation in the clinical trial.
Friday, February 27, 2009
Confidence interval for correlation coefficient
When we perform the correlation analysis, we typically calculate the correlation coefficient and then test if this correlation coefficient is statistically significant or not. We will then judge the degree of the correlation based on the numerical value of the correlation coefficient. Sometimes, we may want to calculate the confidence interval for correlation coefficient to see if the correlation coefficient has reasonable precision.
The easy way to calculate the confidence interval for correlation coefficient is to use FISHER option in SAS procedure. FISHER option is available after SAS version 9. FISHER option specifies the Fisher's z transformation to estimate 95% confidence intervals for a correlation.
When we use the confidence interval to make a judgment about the procision, we need to be aware that this is largely related to the sample size used in the calculation of the correlation coefficient. The larger the sample size, the narrower the confidence interval.
The easy way to calculate the confidence interval for correlation coefficient is to use FISHER option in SAS procedure. FISHER option is available after SAS version 9. FISHER option specifies the Fisher's z transformation to estimate 95% confidence intervals for a correlation.
- Stan Brown has a nice description of the Fisher's z transformation
- SAS Provided an explanation for Applications of Fisher’s z Transformation
- A SUGI paper by David Shen provide the SAS codes
When we use the confidence interval to make a judgment about the procision, we need to be aware that this is largely related to the sample size used in the calculation of the correlation coefficient. The larger the sample size, the narrower the confidence interval.
Thursday, February 19, 2009
Most testing for US drug industry's late-stage human trials done outside the country, study indicates
The Wall Street Journal (2/19, Wang) reports, "Most testing for the US drug industry's late-stage human trials is now done at sites outside the country, where results often can be obtained cheaper and faster, according to a study" published in the New England Journal of Medicine. What "make overseas trials cheaper and faster, [is that] patients in developing countries are often more willing to enroll in studies because of lack of alternative treatment options, and often they aren't taking other medicines. Such 'drug-naïve' patients can be sought after because it is easier to show that experimental treatments are better than placebos, rather than trying to show an improvement over currently available drugs."
According to the New York Times (2/19, B7, Singer), the study "raises questions about the ethics and the science of increasingly conducting studies outside the United States -- when the studies are meant to gather evidence for new drugs to gain approval in this country." The study conducted "by several Duke University researchers, suggests an ethical quagmire when drugs intended for wealthy nations are tested on people in developing countries." The researchers "suggest that human volunteers in foreign countries may be unduly influenced with the promise of financial compensation or free medical care to participate in clinical trials. The report, 'Ethical and Scientific Implications of the Globalization of Clinical Research,' also asks whether drug research conducted in developing countries is relevant to the treatment of American patients." Individuals of East Asian origin, for example, have a genetic variance that may reduce the effects of nitroglycerin treatment.
The researchers' "review of a US government clinical trials registry and of 300 published reports in major medical journals revealed this: A third (157 of 509) of Phase III trials -- typically the largest and most significant trial in the development of a drug -- led by major US pharmaceutical companies were being conducted entirely outside the United States," HealthDay (2/18, Gardner) reported. "In addition, half of the study sites (13,521 of 24,206) used in these trials were located overseas, with many in Eastern Europe and Asia."
On its website, CNN (2/19, Watkins) adds that the researchers "reported one study that found only 56 percent of 670 researchers surveyed in developing countries said their work had been reviewed by a local institutional review board or a health ministry. Another study reported that 18 percent of published trials carried out in China in 2004 adequately discussed informed consent for subjects considering participating in research."
According to the New York Times (2/19, B7, Singer), the study "raises questions about the ethics and the science of increasingly conducting studies outside the United States -- when the studies are meant to gather evidence for new drugs to gain approval in this country." The study conducted "by several Duke University researchers, suggests an ethical quagmire when drugs intended for wealthy nations are tested on people in developing countries." The researchers "suggest that human volunteers in foreign countries may be unduly influenced with the promise of financial compensation or free medical care to participate in clinical trials. The report, 'Ethical and Scientific Implications of the Globalization of Clinical Research,' also asks whether drug research conducted in developing countries is relevant to the treatment of American patients." Individuals of East Asian origin, for example, have a genetic variance that may reduce the effects of nitroglycerin treatment.
The researchers' "review of a US government clinical trials registry and of 300 published reports in major medical journals revealed this: A third (157 of 509) of Phase III trials -- typically the largest and most significant trial in the development of a drug -- led by major US pharmaceutical companies were being conducted entirely outside the United States," HealthDay (2/18, Gardner) reported. "In addition, half of the study sites (13,521 of 24,206) used in these trials were located overseas, with many in Eastern Europe and Asia."
On its website, CNN (2/19, Watkins) adds that the researchers "reported one study that found only 56 percent of 670 researchers surveyed in developing countries said their work had been reviewed by a local institutional review board or a health ministry. Another study reported that 18 percent of published trials carried out in China in 2004 adequately discussed informed consent for subjects considering participating in research."
Saturday, February 14, 2009
Evidence-based medicine - the Evidence Gap
New York Times had a series of articles to explore medical treatments used despite scant proof they work and examining steps toward medicine based on evidence.
Evidence-based medicine (EBM) aims to apply evidence gained from the scientific method to certain parts of medical practice. It seeks to assess the quality of evidence relevant to the risks and benefits of treatment (including lack of treatment). According to the Centre for Evidence-Based Medicine, "Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients."
The key for evidence-based medicine is the quality of evidence. Obviously the regulatory such as FDA applied very strict efficacy standard. According to a slide on FDA's website, FDA does not permit Sponsors To Promote Off-Label Uses because such behaviour
Evidence-based medicine (EBM) aims to apply evidence gained from the scientific method to certain parts of medical practice. It seeks to assess the quality of evidence relevant to the risks and benefits of treatment (including lack of treatment). According to the Centre for Evidence-Based Medicine, "Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients."
The key for evidence-based medicine is the quality of evidence. Obviously the regulatory such as FDA applied very strict efficacy standard. According to a slide on FDA's website, FDA does not permit Sponsors To Promote Off-Label Uses because such behaviour
- Would diminish or eliminate incentive to study the use and obtain definitive data.
- Could result in harm to patients from unstudied uses that actually lead to bad results, or that are merely ineffective.
- Would diminish the use of evidence-based medicine.
- Could ultimately erode the efficacy standard.
However, there are also different voices.
Cronbach's alpha - reliability coefficient
Cronbach's Alpha is a tool for assessing the reliability of scales (for example a quality of life instrument). Cronbach's alpha can be easily calculated from SAS Proc Corr.
To compute Cronbach's alpha for a set of variables, use the ALPHA option in PROC CORR as follows:
PROC CORR DATA=dataset ALPHA;
VAR item1-item10;
RUN;
SAS website provides an example about calculating the Cronbach's alpha.
Very often, 95% confidence interval may be required, the calculation is not straightforward, but there are SAS macros available from the SAS web site.
Some references about Cronbach's alpha can be found below:
To assess the reliability of an instrument, the good reliability features include:
In one of comments on FDA's guidance on PROM (patient reported outcome measures), Cronbach's alpha was cited to measure Internal Consistency and construct validity (with scale analysis) - Cronbach's alpha > 0.70
http://www.fda.gov/ohrms/dockets/dockets/06d0044/06d-0044-EC13-Attach-1.pdf
To compute Cronbach's alpha for a set of variables, use the ALPHA option in PROC CORR as follows:
PROC CORR DATA=dataset ALPHA;
VAR item1-item10;
RUN;
SAS website provides an example about calculating the Cronbach's alpha.
Very often, 95% confidence interval may be required, the calculation is not straightforward, but there are SAS macros available from the SAS web site.
Some references about Cronbach's alpha can be found below:
- http://en.wikipedia.org/wiki/Cronbach
- http://support.sas.com/documentation/cdl/en/procstat/59629/HTML/default/procstat_corr_sect019.htm
- http://www.creative-wisdom.com/teaching/assessment/alpha.html
- http://www2.sas.com/proceedings/sugi26/p246-26.pdf
To assess the reliability of an instrument, the good reliability features include:
- Internal consistency = Cronbach's alpha >= 0.70 for new measures
- Stability = reliability coefficient >= 0.70
- Equivalence = Kappa statistic >= 0.61
In one of comments on FDA's guidance on PROM (patient reported outcome measures), Cronbach's alpha was cited to measure Internal Consistency and construct validity (with scale analysis) - Cronbach's alpha > 0.70
http://www.fda.gov/ohrms/dockets/dockets/06d0044/06d-0044-EC13-Attach-1.pdf
Thursday, February 12, 2009
Blood plasma and serum
Blood plasma, or plasma, is prepared by obtaining a sample of blood and removing the blood cells. The red blood cells and white blood cells are removed by spinning with a centrifuge. Chemicals are added to prevent the blood's natural tendency to clot. If these chemicals include sodium, than a false measurement of plasma sodium content will result. Serum is prepared by obtaining a blood sample, allowing formation of the blood clot, and removing the clot using a centrifuge. Both plasma and serum are light yellow in color.
Plasma is the liquid portion of the blood that is separated from the blood cells by centrifugation. One of the characteristics of plasma is that it clots easily which is important for hemophiliacs needing a transfusion but is a nuisance in most other applications. By agitating the plasma, one can precipitate the clotting factors as a large clot, and the leftover fluid is called serum. So, serum plus clotting factors is plasma, and clotted plasma yields serum (as an interesting aside, "serum" is Latin for whey, the liquid portion of clotted milk removed in making cheese).
The following course note describes the contents of the blood, plasma, and serum.
Plasma is the liquid portion of the blood that is separated from the blood cells by centrifugation. One of the characteristics of plasma is that it clots easily which is important for hemophiliacs needing a transfusion but is a nuisance in most other applications. By agitating the plasma, one can precipitate the clotting factors as a large clot, and the leftover fluid is called serum. So, serum plus clotting factors is plasma, and clotted plasma yields serum (as an interesting aside, "serum" is Latin for whey, the liquid portion of clotted milk removed in making cheese).
The following course note describes the contents of the blood, plasma, and serum.
Thursday, February 05, 2009
DMC (Data Monitoring Committee) vs. DSMB (Data Safety Monitoring Board)
DMC (Data Monitoring Committee) vs. DSMB (Data Safety Monitoring Board) are the same thing. The term DMC is used more now because it is the term used in FDA's guidance "Establishment and Operation of Clinical Trial Data Monitoring Committees" and EMEA's Guidance on Data Monitoring Committees. However, the World Health Organization used DSMB in its guidance titled "Operational Guidelines for the Establishment and Functioning of Data Safety Monitoring Boards". When searching for articles, it is recommended to try both terms "DMC" and "DSMB".
Some further discussions prior to FDA's issurance of DMC guidance are worth to read. These include:
EMEA guidance said "In case of a submission the working procedures of a DMC as well as all DMC reports (open and closed sessions) should form part of the submission."
The internal discussion notes said "A special circumstance is the case in which the sponsor wishes to use interim data in support of a regulatory submission, with the intent to continue the trial to its conclusion. Because of the risks to the trial’s credibility, analysis and use of interim data for this purpose is often ill advised. Exceptional circumstances may arise, however, in which such use could be appropriate. Before accessing and using interim data for this purpose, sponsors should confer with FDA and the DMC (or DMC chair) and consider all potential implications of such actions. "
According to FDA guidance "The agency recommends in the guidance that the DMC or the group preparing the interim reports to the DMC maintain all meeting records. This information should be submitted to FDA with the clinical study report (Sec. 314.50(d)(5)(ii) (21 CFR 314.50(d)(5)(ii)))."
Post-analysis DMC meeting: what are the pros and cons of having the DMC convene post-analyiss so they can make an assessment on complete and clean data?
The principal role of DMC is to ensure the safety of patients, which they do by analyzing adverse events and by performing interim analyses of the clinical outcome data. Due to the time constraints, the DMC analyses are typically based on the data that is incomplete or not totally cleaned. Analyses post DMC meeting are typically not needed unless there are serious issues with the data.
One interesting question is the role of the DMC after the study has been completed. My understanding is that the DMC plays the big role during the study. After the study has been completed, DMC would hand the responsibilities back to the sponsor and investigator since all subjects have been off the study. If there is any DMC meeting after the study completion, it is mainly for the courtesy or information purpose.
If DMC made the suggestion to stop the trial after reviewing the interim analysis data, after their suggestion, it is up to the sponsor and investigators (or steering committees or executive committees) to handle the rest (close out the study, disclose the study results, write manuscript,…). In this situation, no post-DMC meeting is needed. The final analyses will be performed by the sponsor or investigators. Investigators will publish the study results. Some examples are: Novartis ACCOMPLISH trial - stopped for efficacy; Pfizer’s ILLUMINATE trial - stopped for futility.
Some further discussions prior to FDA's issurance of DMC guidance are worth to read. These include:
- An internal discussion note.
- FDA internal assessment of annual report burden
- Notes from CBER open public meeting on DMC
- Comments on draft DMC guidance
- NIH policy on data and safety monitoring
- NIH policy on data and safety monitoring of Phase I and II trials
- NCI policy on data and safety monitoring
- NCI's essential elements on data and safety monitoring
- Template from NHLBI
- Template from NIA
- DMC policy from ECOG
- http://www.ctu.mrc.ac.uk/files/DMCcharter_general.pdf
- Template from Applied Clinical Trials
- Slutsky et al (2004) Data Safety and Monitoring Board. NEJM 350:1143-1147
- Freidlin, B., Korn, E. L. (2009). Monitoring for Lack of Benefit: A Critical Component of a Randomized Clinical Trial. JCO 27: 629-633
- Miller and Wendler (2008). Is it ethical to keep interim findings of randomised controlled trials confidential?. J. Med. Ethics 34: 198-201
- Borer et al (2008) When should data and safety monitoring committees share interim results in cardiovascular trials? JAMA Apr 9;299(14):1710-2
- Mueller et al (2007) Ethical Issues in Stopping Randomized Trials Early Because of Apparent Benefit. ANN INTERN MED 146: 878-881
- Goodman (2007) Stopping at Nothing? Some Dilemmas of Data Monitoring in Clinical Trials. ANN INTERN MED 146: 882-887
- Silverman (2007) Ethical Issues during the Conduct of Clinical Trials. Proc Am Thorac Soc 4: 180-184
- Chen-Mok et al (2006) Experiences and challenges in data monitoring for clinical trials within an international tropical disease research network. Clin Trials 3: 469-477
- Ellenberg, Fleming, Demets (2002) Data Monitoring Committees in clinical trials: a practical perspective
- Demets, Friedman, Furberg (2006) Data Monitoring in clinical trials: a case studies approach
- Moffett (2006) Statistical monitoring of clinical trials: a unified approach
EMEA guidance said "In case of a submission the working procedures of a DMC as well as all DMC reports (open and closed sessions) should form part of the submission."
The internal discussion notes said "A special circumstance is the case in which the sponsor wishes to use interim data in support of a regulatory submission, with the intent to continue the trial to its conclusion. Because of the risks to the trial’s credibility, analysis and use of interim data for this purpose is often ill advised. Exceptional circumstances may arise, however, in which such use could be appropriate. Before accessing and using interim data for this purpose, sponsors should confer with FDA and the DMC (or DMC chair) and consider all potential implications of such actions. "
According to FDA guidance "The agency recommends in the guidance that the DMC or the group preparing the interim reports to the DMC maintain all meeting records. This information should be submitted to FDA with the clinical study report (Sec. 314.50(d)(5)(ii) (21 CFR 314.50(d)(5)(ii)))."
Post-analysis DMC meeting: what are the pros and cons of having the DMC convene post-analyiss so they can make an assessment on complete and clean data?
The principal role of DMC is to ensure the safety of patients, which they do by analyzing adverse events and by performing interim analyses of the clinical outcome data. Due to the time constraints, the DMC analyses are typically based on the data that is incomplete or not totally cleaned. Analyses post DMC meeting are typically not needed unless there are serious issues with the data.
One interesting question is the role of the DMC after the study has been completed. My understanding is that the DMC plays the big role during the study. After the study has been completed, DMC would hand the responsibilities back to the sponsor and investigator since all subjects have been off the study. If there is any DMC meeting after the study completion, it is mainly for the courtesy or information purpose.
If DMC made the suggestion to stop the trial after reviewing the interim analysis data, after their suggestion, it is up to the sponsor and investigators (or steering committees or executive committees) to handle the rest (close out the study, disclose the study results, write manuscript,…). In this situation, no post-DMC meeting is needed. The final analyses will be performed by the sponsor or investigators. Investigators will publish the study results. Some examples are: Novartis ACCOMPLISH trial - stopped for efficacy; Pfizer’s ILLUMINATE trial - stopped for futility.
Tuesday, February 03, 2009
Standard Error of Mean vs. Standard Error of Measurement
Everybody with basic statistical knowledge should understand the differences between the standard deviation (SD) and the standard error of mean (SE or SEM). However, people may be confused with the terms of Standard Error of Mean (SEM) vs. Standard Error of Measurement (SEM). While both shares the same acronym, the meaning and the calculation are quite different. At least, this is the situation when I saw the term 'standard error of measurement'.
I first saw this term in a literature discussing various approaches to identify the minimal clinically important difference (MCID). In an article by Copay et al, SEM (standard error of measurement) was quoted as one of the many approaches in evaluating the MCID. This method was also discussed in a paper by Wyrwich et al. Initially, I mistakenly thought that SEM was for standard error of mean. After further exploration, I realized that this SEM is quite different from that SEM.
The standard error of the mean (SEM) is the standard deviation of the sample mean estimate of a population mean. (It can also be viewed as the standard deviation of the error in the sample mean relative to the true mean, since the sample mean is an unbiased estimator.) SEM is usually estimated by the sample estimate of the population standard deviation (sample standard deviation) divided by the square root of the sample size (assuming statistical independence of the values in the sample).
The standard error of measurement (SEM) estimates how repeated measures of a person on the same instrument tend to be distributed around his or her "true" score. The true score is always an unknown because no measure can be constructed that provides a perfect reflection of the true score. SEM is directly related to the reliability of a test; that is, the larger the SEm, the lower the reliability of the test and the less precision there is in the measures taken and scores obtained. Since all measurement contains some error, it is highly unlikely that any test will yield the same scores for a given person each time they are retested.
Ar article by Dr. James Brown at University of Hawai'i at Manoa gave an good comparison of these two concepts. Also, an free paper by Harvill LM from East Tennessee State University explained in detail how the standard error of measurement is calculated.
I first saw this term in a literature discussing various approaches to identify the minimal clinically important difference (MCID). In an article by Copay et al, SEM (standard error of measurement) was quoted as one of the many approaches in evaluating the MCID. This method was also discussed in a paper by Wyrwich et al. Initially, I mistakenly thought that SEM was for standard error of mean. After further exploration, I realized that this SEM is quite different from that SEM.
The standard error of the mean (SEM) is the standard deviation of the sample mean estimate of a population mean. (It can also be viewed as the standard deviation of the error in the sample mean relative to the true mean, since the sample mean is an unbiased estimator.) SEM is usually estimated by the sample estimate of the population standard deviation (sample standard deviation) divided by the square root of the sample size (assuming statistical independence of the values in the sample).
The standard error of measurement (SEM) estimates how repeated measures of a person on the same instrument tend to be distributed around his or her "true" score. The true score is always an unknown because no measure can be constructed that provides a perfect reflection of the true score. SEM is directly related to the reliability of a test; that is, the larger the SEm, the lower the reliability of the test and the less precision there is in the measures taken and scores obtained. Since all measurement contains some error, it is highly unlikely that any test will yield the same scores for a given person each time they are retested.
Ar article by Dr. James Brown at University of Hawai'i at Manoa gave an good comparison of these two concepts. Also, an free paper by Harvill LM from East Tennessee State University explained in detail how the standard error of measurement is calculated.
Tuesday, January 20, 2009
Multiple Comparisons
In statistics or biostatistics, the multiple comparisons problem occurs when one considers a set, or family, of statistical inferences simultaneously. Errors in inference, including confidence intervals that fail to include their corresponding population parameters, or hypothesis tests that incorrectly reject the null hypothesis, are more likely when one considers the family as a whole.
Multiple comparison issues were nicely summarized in EMEA's guidance titled "Points to consider on multiplicity issues in clinical trials". This guidance also discussed the situations where the adjustment for multiplicity is not needed.
Adjustment for multiplicity is also mentioned in many regulatory guidance, for example, FDA guidance on ISE and its importance has been recognized in may medical journal review process.
SAMSI held a workshop in 2005 to discuss teh multiplicity issues which included the issue in Multiple Testing, Reproducibility, and Subgroup analysis.
For an introduction about multiple comparisons, refer to Wikipedia "http://en.wikipedia.org/wiki/Multiple_comparisons"
SAS Proc Multitest can be an easy tool to compute the adjusted p-values (with different methods) if the raw p-values from multiple tests are provided. For example, with the following program, we would be able to obtain a set of adjusted p-values.
data integrated;
input Method$ Raw_P;
datalines;
method1 .331
method2 .090
method3 .105
method4 .xxx
;
proc multtest pdata=integrated holm hoc fdr bon;
run;
Multiple comparison issues were nicely summarized in EMEA's guidance titled "Points to consider on multiplicity issues in clinical trials". This guidance also discussed the situations where the adjustment for multiplicity is not needed.
Adjustment for multiplicity is also mentioned in many regulatory guidance, for example, FDA guidance on ISE and its importance has been recognized in may medical journal review process.
SAMSI held a workshop in 2005 to discuss teh multiplicity issues which included the issue in Multiple Testing, Reproducibility, and Subgroup analysis.
For an introduction about multiple comparisons, refer to Wikipedia "http://en.wikipedia.org/wiki/Multiple_comparisons"
SAS Proc Multitest can be an easy tool to compute the adjusted p-values (with different methods) if the raw p-values from multiple tests are provided. For example, with the following program, we would be able to obtain a set of adjusted p-values.
data integrated;
input Method$ Raw_P;
datalines;
method1 .331
method2 .090
method3 .105
method4 .xxx
;
proc multtest pdata=integrated holm hoc fdr bon;
run;
Monday, January 19, 2009
Trial Biomarker Analysis More Than Data Dredging
members of the FDA's oncology Drugs Advisory Committee cautioned sponsors against treating retrospective clinical trial biomarker analysis like an exercise in data dredging.
The committee met last month go consider the adequacy of retrospectivly mined data in determining whether a biomarker is truly predictive of patient response. The discussion stemmed from a retrospective data analysis conducted to show that the KRAS biomarker status of patient tumors helps predict responses to Amgen's Vectibix (panitumumab) and ImClone and Bristol-Myers Squibb's Erbitux (cebuximab) cancer drugs.
See meeting transcribts here or the slides.
In other news, the US FDA encourage the integration of biomarkers in drug development and their appropriate use in clinical practice.
The committee met last month go consider the adequacy of retrospectivly mined data in determining whether a biomarker is truly predictive of patient response. The discussion stemmed from a retrospective data analysis conducted to show that the KRAS biomarker status of patient tumors helps predict responses to Amgen's Vectibix (panitumumab) and ImClone and Bristol-Myers Squibb's Erbitux (cebuximab) cancer drugs.
See meeting transcribts here or the slides.
In other news, the US FDA encourage the integration of biomarkers in drug development and their appropriate use in clinical practice.
Data dredging vs. Data mining; Post-hoc vs. Ad-hoc
Data dredging (data fishing, data snooping) is the inappropriate (sometimes deliberately so) search for 'statistically significant' relationships in large quantities of data. This activity was formerly known in the statistical community as data mining, but that term is now in widespread use with an essentially positive meaning, so the pejorative term data dredging is now used instead.
Data mining is the process of extracting hidden patterns from data. As more data is gathered, with the amount of data doubling every three years, data mining is becoming an increasingly important tool to transform this data into information. It is commonly used in a wide range of applications, such as marketing, fraud detection and scientific discovery. Data mining can be applied to data sets of any size. However, while it can be used to uncover hidden patterns in data that has been collected, obviously it can neither uncover patterns which are not already present in the data, nor can it uncover patterns in data that has not been collected.
Post-hoc:
In or of the form of an argument in which one event is asserted to be the cause of a later event simply by virtue of having happened earlier: coming to conclusions post hoc; post hoc reasoning.
[Latin, short for post hoc, ergō propter hoc, after this, therefore because of this : post, after + hoc, neuter of hic, this.]
AD-Hoc:
adv.
For the specific purpose, case, or situation at hand and for no other: a committee formed ad hoc to address the issue of salaries.adj.
Formed for or concerned with one specific purpose: an ad hoc compensation committee.
Improvised and often impromptu: “On an ad hoc basis, Congress has . . . placed . . . ceilings on military aid to specific countries” (New York Times).
[Latin : ad, to + hoc, neuter accusative of hic, this.]
While both post-hoc and ad-hoc analysis may be performed based on the data or results we have seen, the ad-hoc analysis typically occurred alongside the project while the post-hoc analysis occurred absolutely after the project or after the unblinding of the study or after the pre-specified analyses results have been reviewed. In this sense, the ad-hoc analysis is better than post-hoc analysis.
Data mining is the process of extracting hidden patterns from data. As more data is gathered, with the amount of data doubling every three years, data mining is becoming an increasingly important tool to transform this data into information. It is commonly used in a wide range of applications, such as marketing, fraud detection and scientific discovery. Data mining can be applied to data sets of any size. However, while it can be used to uncover hidden patterns in data that has been collected, obviously it can neither uncover patterns which are not already present in the data, nor can it uncover patterns in data that has not been collected.
Post-hoc:
In or of the form of an argument in which one event is asserted to be the cause of a later event simply by virtue of having happened earlier: coming to conclusions post hoc; post hoc reasoning.
[Latin, short for post hoc, ergō propter hoc, after this, therefore because of this : post, after + hoc, neuter of hic, this.]
AD-Hoc:
adv.
For the specific purpose, case, or situation at hand and for no other: a committee formed ad hoc to address the issue of salaries.adj.
Formed for or concerned with one specific purpose: an ad hoc compensation committee.
Improvised and often impromptu: “On an ad hoc basis, Congress has . . . placed . . . ceilings on military aid to specific countries” (New York Times).
[Latin : ad, to + hoc, neuter accusative of hic, this.]
While both post-hoc and ad-hoc analysis may be performed based on the data or results we have seen, the ad-hoc analysis typically occurred alongside the project while the post-hoc analysis occurred absolutely after the project or after the unblinding of the study or after the pre-specified analyses results have been reviewed. In this sense, the ad-hoc analysis is better than post-hoc analysis.
Subscribe to:
Posts (Atom)