Friday, July 03, 2015

Protocol Deviation versus Protocol Violation and its Classifications (minor, major, critical, important)

Every clinical trial will have a study protocol. The investigator is required to follow the study protocol to conduct the study. However, during the clinical trial, there will always be planned or unplanned deviations from the protocol. ICH GCP requires that these protocol deviations are documented. ICH E6 (section 4.5.3) states “the investigator, or person designated by the investigator, should document and explain any deviation from the approved protocol.” At the end of the study, statistical analysis will include a listing for all protocol deviations and a summary table for protocol deviations by category.

Across various regulatory guidelines, both terms ‘protocol deviations’ and ‘protocol violations’ are used. What is the difference between a protocol deviation and a protocol violation?

For a while, there seems to be a thinking that the protocol deviation is less serious non-compliance and the protocol violation is more serious non-compliance of the protocol. However, the recent documents from the regulatory bodies suggest that both terms are the same and can be used interchangeably. In practice, it will not be wrong if we stick to the term ‘protocol deviation’ and avoid using the term ‘protocol violation’.

In FDA’s “Compliance Program Guidance Manual For FDA Staff - Compliance Program 7348.811 Bioresearch Monitoring: Clinical Investigators” in 2008. It provided a definition for ‘protocol deviation’, however, the term ‘protocol deviation/violation’ was lumped together and did not draw a clear distinction between protocol deviation and protocol violation.


“Protocol deviations. A protocol deviation/violation is generally an unplanned excursion from the protocol that is not implemented or intended as a systematic change. A protocol deviation could be a limited prospective exception to the protocol (e.g. agreement between sponsor and investigator to enroll a single subject who does not meet all inclusion/exclusion criteria). Like protocol amendments, deviations initiated by the clinical investigator must be reviewed and approved by the IRB and the sponsor prior to implementation, unless the change is necessary to eliminate apparent immediate hazards to the human subjects (21 CFR 312.66), or to protect the life or physical wellbeing of the subject (21 CFR 812.35(a)(2)), and generally communicated to FDA. “Protocol deviation” is also used to refer to any other, unplanned, instance(s) of protocol noncompliance. For example, situations in which the investigator failed to perform tests or examinations as required by the protocol or failures on the part of study subjects to complete scheduled visits as required by the protocol, would be considered protocol deviations.”

In ICH E3 Guideline: Structure and Content of Clinical Study Reports Questions & Answers in 2012, both ‘protocol deviation’ and ‘protocol violation’ were used. The document suggested protocol violation is equivalent to important protocol deviation. In other words, the protocol violation is a subset of all protocol deviations.

A protocol deviation is any change, divergence, or departure from the study design or procedures defined in the protocol. Important protocol deviations are a subset of protocol deviations that may significantly impact the completeness, accuracy, and/or reliability of the study data or that may significantly affect a subject's rights, safety, or well-being. For example, important protocol deviations may include enrolling subjects in violation of key eligibility criteria designed to ensure a specific subject population or failing to collect data necessary to interpret primary endpoints, as this may compromise the scientific value of the trial. Protocol violation and important protocol deviation are sometimes used interchangeably to refer to a significant departure from protocol requirements. The word “violation” may also have other meanings in a regulatory context. However, in Annex IVa, Subject Disposition of the ICH E3 Guideline, the term protocol violation was intended to mean only a change, divergence, or departure from the study requirements, whether by the subject or investigator, that resulted in a subject’s withdrawal from study participation. (Whether such subjects should be included in the study analysis is a separate question.) To avoid confusion over terminology, sponsors are encouraged to replace the phrase “protocol violation” in Annex IVa with “protocol deviation”, as shown in the example flowchart below. Sponsors may also choose to use another descriptor, provided that that the information presented is generally consistent with the definition of protocol violation provided above. The E3 Guideline provides examples of the types of deviations that are generally considered important protocol deviations and that should be described in Section 10.2 and included in the listing in Appendix 16.2.2. The definition of important protocol deviations for a particular trial is determined in part by study design, the critical procedures, study data, subject protections described in the protocol, and the planned analyses of study data. In keeping with the flexibility of the Guideline, sponsors may amend or add to the examples of important deviations provided in E3 in consideration of a trial’s requirements. Substantial additions or changes should be clearly described for the reviewer.

When protocol deviations are documented, they are also classified into categories according to the severity and their effect on the subject’s rights, safety, or welfare, or on the integrity of the resultant data.  
ICH E3 “STRUCTURE AND CONTENT OF CLINICAL STUDY REPORTS” requires the important protocol deviation to be described. It does not use the categories of critical, major, or minor. However the descriptions In Section 10.2 suggests the important protocol deviations are those with major or critical categories. Section 10.2 stated:

All important deviations related to study inclusion or exclusion criteria, conduct of the trial, patient management or patient assessment should be described. In the body of the text, protocol deviations should be appropriately summarised by centre and grouped into different categories, such as: 

− those who entered the study even though they did not satisfy the entry criteria;

− those who developed withdrawal criteria during the study but were not withdrawn;

− those who received the wrong treatment or incorrect dose;

− those who received an excluded concomitant treatment. 

In appendix 16.2.2, individual patients with these protocol deviations should be listed, broken down by centre for multicentre studies.


In US, while there is no formal guidance, the protocol deviations are usually classified as major or minor categories. For example, in a FDA presentation about “Avoiding Common Mistakes in Clinical Research”, the protocol deviation spectrum contains minor (a missed lab test, a missed visit) and major (ineligible subject enrolled, safety or efficacy assessments not done, did not report SAE to IRB • • • •).

In EU, EMA guidance “Classification and analysis of the GCP inspection findings of GCP inspections conducted at the request of the CHMP”, the protocol deviations are classified as Critical, Major, and Minor categories.

 
Critical: - Conditions, practices or processes that adversely affect the rights, safety or well-being of the subjects and/or the quality and integrity of data.
- Critical observations are considered totally unacceptable.
- Possible consequences: rejection of data and/or legal action required.
- Remarks: observations classified as critical may include a pattern of deviations classified as major, bad quality of the data and/or absence of source documents. Manipulation and intentional misrepresentation of data belong to this group.
 
Major: - Conditions, practices or processes that might adversely affect the rights, safety or well-being of the subjects and/or the quality and integrity of data.
- Major observations are serious findings and are direct violations of GCP principles.
             - Possible consequences: data may be rejected and/or legal action required.
             - Remarks: observations classified as major, may include a pattern of deviations
                    and/or numerous   minor observations.
 
Minor: - Conditions, practices or processes that would not be expected to adversely affect the right, safety or well-being of the subjects and/or the quality and integrity of data.
- Possible consequences: observations classified as minor, indicate the need for improvement of conditions, practices and processes.
            - Remarks: many minor observations might indicate a bad quality and the sum might
                be equal to a major finding with its consequences.
 

In practice, the critical and major protocol deviations may be grouped together. At least this is how it is done in our of NIH studies. See Protocol Deviations CRF Module Instructions

Protocol Deviation Discussion at Firstclinical.com:

Sunday, June 14, 2015

Sample Size and Power Calculation Using SAS Proc Power and twosamplesurvival Statement

For sample size and power calculations, several commercially available software can be used. The commonly used ones are EAST, PASS, and NQuery Advisor. SAS has a procedure (PROC POWER) that can be used for sample size and power calculations for many types of the study designs / study endpoints. One of the statements (twosamplesurvival) in Proc Power is for comparing two survival curves and calculating the sample size/power for time to event variable.

The syntax and descriptions for Twosamplesurvival statement in PROC POWER can be found on SAS website. It can be used to calculate:

  • the total number of events needed (EVENTSTOTAL = .  Option)
  • the total number of subjects needed (NTOTAL = . Option)
  • the number of subjects needed per treatment group (NPERGROUP=. Option)
  • the statistical power (POWER=. Option)

Notice that only one option designated as the result is allowed. If we want to get both the total number of events and the total number of subjects, we would need to run the program two times: one for solving the total number of events and one for solving the total number of subjects.

Here are some of the example applications of using twosamplesurvival statement.

EXAMPLE #1:

In a SUGI paper "Proc Power in SAS 9.1" by Bauer, Lavery, and Ford, an example was provided to calculate the sample size for log-rank test with 2:1 randomization ratio and with drop out.

The example assumes 30% of placebo patients are sustained responders (exponential hazard =0.3567) compared to 45 or 50% for the treatment group (exp. hazard = 0.5978 or 0.6931). Twice as many patients are on treatment as placebo, and all patients are enrolled at the beginning of the study with a 30% drop-out rate.

Prior to the sample size calculation, the event rates were converted to hazards. Exponential hazard in Placebo group = - ln(1 - event rate) = -ln(1-0.3) = 0.3567.  Similarly, Exponential hazards corresponding to 45% or 50% event rate were 0.5978 and 0.6931.

The dropout rate were also converted to group loss hazards in the same way. Therefore, the 30% dropout rate was corresponding to the group loss hazard of -ln(1-dropout rate)=-ln(1-0.3)=0.3567.

Groupweights statement was used to indicate the 2:1 randomization ratio. 

proc power;
       twosamplesurvival test=logrank
       gexphs= 0.3567 | 0.5978 .6931
       grouplossexphazards=(0.3567 0.3567)
       accrualtime = 1
       followuptime = 1
       groupweights = (1 2)
       power = .
       ntotal=225;
run;


EXAMPLE #2:

Dr Hudgens from UNC had a nice posting about the power and sample size calculations for log-rank test. He gave an example as following:


Clinical trial to assess new treatment for patients with chronic active hepatitis. Under standard treatment, 41% of patients survive beyond 5 years. Expect new treatment to increase survival beyond 5 years to 60%.


In order to calculate the sample size, we will need to calculate some parameters.
Event rate for standard treatment (Ec) = 1-0.41 = 0.59
Event rate for new treatment (Et) = 1-0.60 = 0.4
Since event rate E = 1 - exp(-t*HAZARD), we have HAZARD = -ln((1-E)/t   
The Hazard for standard treatment is HAZARDc=-ln(1-Ec)/t = -ln(1-0.59)/t = -ln(0.41)/t
The Hazard for new treatment is t* HAZARDt = -ln(1-Et)/t = -ln(1-0.40)/t = -ln(0.60)/t
The hazard ratio  = HAZARDt/HAZARDc = ln((0.6)/ln(0.41)=0.5729
T=5, the hazard for standard treatment is HAZARDc = -ln(0.41)/5 = 0.178

After these calculation, the following SAS codes can be used to calculate the sample size:

proc power;
    twosamplesurvival test=logrank
    hazardratio = 0.57
    refsurvexphazard=0.178
    followuptime = 5
    totalTIME = 5
    power = 0.90
    ntotal = . ;
run;

EXAMPLE #3: Sample Size Calculation with piecewise linear survival curve

SAS has a GUI desktop application PSS (the Power and Sample Size Application) that provides easy access to power analysis and sample size determination techniques. Anything implemented in PSS desktop application can also be realized using Proc Power. Here is a link to an example from using PSS desktop application. The calculation can be realized using Proc Power Twosamplesurvival.


Suppose you want to compare survival rates for an existing cancer treatment and a new treatment. You intend to use a log-rank test to compare the overall survival curves for the two treatments. You want to determine a sample size to achieve a power of 0.8 for a two-sided test using a balanced design, with a significance level of 0.05.

The survival curve of patients for the existing treatment is known to be approximately exponential with a median survival time of five years. You think that the proposed treatment will yield a survival curve described by the times and probabilities listed in Table 69.9. Patients are to be accrued uniformly over two years and followed for three years.

Table 69.9 Survival Probabilities for Proposed Treatment
Time
Probability
1
0.95
2
0.90
3
0.75
4
0.70
5
0.60


The descriptions for using PSS desktop application for this example can be found on SAS website. The following program will do exactly the same.

proc power;
      twosamplesurvival test=logrank
       curve("Existing Treatment") = 5 : 0.5
      curve("Proposed Treatment") = 1 : 0.95 2 : 0.90 3:0.75  4:0.70 5:0.60
      groupsurvival = "Existing Treatment" | "Proposed Treatment"
      accrualtime = 2
      FOLLOWUPTIME = 3
      power = 0.80
      alpha=0.05
      npergroup = . ;
run;


EXAMPLE #4:
twosamplesurvival statement embedded in PROC SEQDESIGN can be used to estimate the sample size for group sequential design with interim analyses.

EXAMPLE #5:
In the following SAS program to calculate the sample size, the survival probability at 12 months are for standard and proposed groups are specified and the statement of grouplossexphazards is used to account for the dropout rate.

proc power;
      twosamplesurvival test=logrank
      curve(“Standard”) = 12 : 0.8781
      curve(“Proposed”) = 12 : 0.9012
      groupsurvival = “Standard” | “Proposed”
      accrualtime = 18
      Totaltime = 24
      GROUPLOSSEXPHAZARDS = (0.0012 0.0012)
      NSUBINTERVAL = 1
      power = 0.85
      ntotal = . ;
run;

Sunday, June 07, 2015

Some Explanations about Survival Analysis or Time to Event Analysis

The concept of Survival analysis came from the old days for cancer study where the measurement is the length of the survival or time to death. The concept of survival analysis has since been generalized to ‘time to event’ analysis. ‘Time to death’ is just one type of time to event variables. There are a lot of events that can be analyzed in the same way. For time to event measures, the event can be positive and negative. For example, in a clinical trial for recurrent herpes labialis (or cold sores), the event (healing of the primary lesion complex) is positive. The time to event is measured as time to healing of the primary lesion complex. Perhaps, more clinical trials have the negative event (event is bad). In oncology studies, we measure survival (or time to death). In COPD studies, we may measure time to exacerbation. In a PAH study, we may measure time to clinical worsening.

In short, in Time to Event analysis, the analysis relates not just to whether an event occurs but also when.
The planning (for example sample size estimation) and analysis of the time to event study, several important concepts and their relationship are important. These concepts and their relationships are explained below:

Time to Event is just the measure from the start of an intervention to the time when an event occurs. The start of an intervention could be the randomization, start of the treatment, date of surgery,…

Event Rate is the proportion of subjects or patients in a group in whom an event is observed.  Event rate is usually measured for a period of the time from t to t + Dt. For example, if the Dt = 12 months, the event rate will be for one year. Event rate is also given as the event rate for the entire study period.

Hazard Rate is the probability of an event occurring given that it hasn’t occurred up to the current point in time. Hazard rate is the instantaneous risk of a patient experiencing a particular event at each specified time. The instantaneous rate with which an event occurs at a single point in time. It is the probability that the event occurs between time t and time t+delta given that it has not yet occurred by time t, divided by delta, as delta becomes vanishingly small. Note that rates, unlike probabilities, can exceed 1.0 because they are quotients.

Hazard Ratio is a measure of effect produced by a survival analysis. This represents the increased risk with which one group is likely to experience the outcome of interest.  For example, if the hazard ratio for death for a treatment is 0.5, then we can say that treated patients are likely to die at half the rate of untreated patients.

Hazard ratio is calculated as the ratio of hazard rates at a single time t, for two groups of subjects (treatment versus control group). Hazard ratios are in the interval [0, infinity), and they are frequently good ways to summarize the relative effects of two treatments at a specific time t. Like odds ratios, hazard ratios can apply to any level of outcome probability for the reference group. Note that a hazard ratio is distinct from a risk ratio, the latter being the ratio of two simple probabilities and not the ratio of two rates.

The Median Event Time is calculated as the smallest even time for which the event function is less or equal to 0.5.

When the event is death, the median event time is called the median survival time. The median survival time is calculated as the smallest survival time for which the survivor function is less than or equal to 0.5. In oncology study, median survival time the time from either diagnosis or treatment at which half of the patients with a given disease are found to be, or expected to be, still alive. In a clinical trial, median survival time is one way to measure the effectiveness of a treatment to see how well a new treatment works. Median survival time may be called median overall survival or simply median survival.  
Censoring is a form of missing data problem which is common in survival analysis and time to event analysis. In clinical trials, we usually have to deal with the right censoring. In the situation of the right censoring, the event did not occur when subjects are lost to follow-up or when the study ends. A patient might be known not to have had the event only up to a particular point in time, so ‘time to event’ or ‘survival time’ is censored at this point.

Lost to Follow-up refers to patients who at one point in time were actively participating in a clinical trial, but have become lost (either by error in a computer tracking system or by being unreachable) at the point of follow-up in the trial. These patients can become lost for many reasons. Without properly informing the investigator associated with the clinical trial, they may have opted to withdraw from the clinical trial, moved away from the particular study site during the clinical trial, or become ill and unable to communicate or are deceased.

Attrition: The loss of participants during the course of a study. Participants that are lost during the study are often call dropouts.

Accrual time or accrual period is recruitment period during which subjects are being enrolled (recruited) into a study.
Follow-up time or follow-up period is the period after the last subject entered the study until the end of the study. The follow-up defines the phase of a study during which subjects are under observation and no new subjects enter the study.

 
If T is the total duration of a study, and R is the accrual period of the study, then follow-up period f is equal to T – R.
Event Rate =  1 -  Non Event Rate
Mortality Rate =  1 - Survival Rate

Given the MET (median event time), we can calculate the hazard rate and the event rate, and hazard ratio.

If METc is the Median event time for control group and METt is the Median event time for treatment group, HAZARDc and HAZARDt are Hazard rates for control group and treatment groups, we will have:
        HAZARDc = log(2)/METc       
        HAZARDt = log(2)/METt
        METc  = log(2)/HAZARDc       
        METt = log(2)/HAZARDt

    Event rate at month 12 for control group is
      Ec = 1 - exp(-12*HAZARDc) 

    Event rate at month 12 for treatment group is  
      Et = 1 - exp(-12*HAZARDt);   

The hazard ratio is:
      HAZARDt/HAZARDc

If the given parameter is event rate over the entire course of the study (for example, 5 years), the event rate for one year can be calculated using the following formula:

        1 - (1 - event rate)^(1/t)
 
        where t=5
 
 
If the given parameter is event rate over one year period, the event rate for five year study period can be calculated using the formula below:


       1 - exp(t*ln(1 - annual event rate))
 
        where t=5
 

The formula above can also be used to convert the loss follow up rate from the entire treatment period to one year or vice versa.

References:

Friday, May 01, 2015

Study Day in Clinical Study Protocol

Any clinical trial should always have a study protocol. The study protocol will contain a schedule of events table to specify the study visits and the study procedures. For some protocols, study visits will just be the study days (Day 1, Day 7, Day 21,…).

A while ago, I wrote an article “counting the study day” to describe two ways in counting the study day: starting with zero or starting 1. At that time, I said that both ways were acceptable.

It turns out that starting with study day 0 can cause a lot of trouble for programming, especially in SDTM (study data tabulation model).  According to SDTM Implementation Guidelines, there should not be a study day 0 in SDTM data set. If a study protocol uses a study day 0 as the first day of study drug administration, there will be an inconsistency when comparing the SDTM data sets (therefore, the data listings) with the protocol. Here is what it is said about the study day in SDTM implementation guidelines (SDTMIG).






To avoid the inconsistency issue, it is advisable that in study protocols, study day 0 should not be used. In clinical study protocols, the first day of the study drug administration (or randomization day) should always be counted as day 1 (instead of day 0).  With study day 1 being the first day of the study drug administration, the 7 days after treatment will be study day 8, and so on.

The schedule of events table below is advisable. It correctly uses Day 1 as the first day of dosing.



The schedule of events table below is not advisable since it uses Day 0. To avoid the trouble for downstream SDTM programming, it is better to have study visits listed as Day 1, Day 2, Day 8, Day 15,  Day 29.

Additional References:

1. Studyday calculation ( --DY Variable in SDTM)
2. SAS Programming in the Pharmaceutical Industry             

Sunday, April 19, 2015

Hockey stick effect, yo-yo effect, honeymoon effect, halo effect, Hawthorne effect, John Henry effect, and Pygmalion effect

In clinical trials, people need to pay attention to the various effects in designing the clinical trials. A while ago, I mentioned the hockey stick effect that described an initial rebound in a treatment measure followed by the true deterioration of the treatment measures, which occurred in diseases such as Alzheimer’s disease, alpha-1 antitrypsin deficiency, where the purpose of the treatment is to prevent the further deterioration of the disease progression instead of improving.

Other these kinds of effects were described below.

In a clinical trial design discussion, the KOL (key opinion leader) used the yo-yo effect to describe a outcome measure that could fluctuate during the screening period – before the treatment intervention started. 

According to Wikepedia, yo-yo effect, also known as weight cycling, is a term "yo-yo dieting" coined by Kelly D. Brownell at Yale University, in reference to the cyclical loss and gain of weight, resembling the up-down motion of a yo-yo. In this process, the dieter is initially successful in the pursuit of weight loss but is unsuccessful in maintaining the loss long-term and begins to gain the weight back. The dieter then seeks to lose the regained weight, and the cycle begins again.

In a FDA ADCOM meeting minutes, the yo-yo effect is used to describe the hemoglobin level “And hemoglobin levels often plummet to less than 10 before therapy is resumed. This causes a yo-yo effect and is not in the best interests of patients in treating their anemia. There is also no safety net to prevent patients' hemoglobin levels from falling so low that a blood transfusion is required. “


Honeymoon Effect

Honeymoon effect is used to describe the temporary rebound in insulin level right after the initial treatment. After starting treatment with insulin a person's own insulin levels may temporarily improve. This is believed to be due to altered immunity and is known as the "honeymoon phase". The clinical trial for type 1 diabetes must consider honeymoon effect in the study design.

The Honeymoon Phase (or Honeymoon Period) amongst people with type 1 diabetes refers to the period of time shortly following diabetes diagnosis when the pancreas is still able to produce a significant enough amount of insulin to reduce insulin needs and aid blood glucose control.

This does not, unfortunately, indicate that the diabetes is in remission or can be cured. 

There is no hard and fast rule for how long the honeymoon phase lasts amongst people with type 1 diabetes. The diabetes honeymoon phase can last for weeks, months or in some cases years.

In an ADCOM meeting, the honeymoon effect was also used to describe the transient initial effect in epilepsy trial “Beneficial effect of stimulation was lasting. We did not see the familiar and transient honeymoon effect of adding medication after medication to the regimens of these refractory patients. By three years of stimulation, seizures were at a median 68 percent improvement relative to baseline, that is to say a third of their initial level”


The halo effect
is a cognitive bias in which an observer's overall impression of a person, company, brand, or product influences the observer's feelings and thoughts about that entity's character or properties. It was named by psychologist Edward Thorndike in reference to a person being perceived as having a halo. Subsequent researchers have studied it in relation to attractiveness and its bearing on the judicial and educational systems. The halo effect is a specific type of confirmation bias, wherein positive feelings in one area cause ambiguous or neutral traits to be viewed positively. Edward Thorndike originally coined the term referring only to people; however, its use has been greatly expanded especially in the area of brand marketing.

The term "halo" is used in analogy with the religious concept: a glowing circle that can be seen floating above the heads of saints in countless medieval and Renaissance paintings. The saint's face seems bathed in heavenly light from his or her halo. Thus, by seeing that somebody was painted with a halo, the observer can tell that this must have been a good and worthy person. In other words, the observer is transferring their judgment from one easily observed characteristic of the person (painted with a halo) to a judgment of that person's character.

The halo effect works both in both positive and negative directions (the horns effect): If the observer likes one aspect of something, they will have a positive predisposition toward everything about it. If the observer dislikes one aspect of something, they will have a negative predisposition toward everything about it.

In a FDA memo, the halo effect was used to described as a bias in subjects’ assessment of the device effectiveness. “In general, subject data have not been employed as a primary endpoint in device trials because it is difficult to determine whether adequate training of patients has been achieved. Such subjects are particularly prone to the “halo effect” in which subjects judge a return to a “baseline” appearance (after resolution of the swelling, redness and tenderness associated with implantation) to be a significant improvement. “


Hawthorne Effect

When people are under study, observation or investigation, this very fact can have an effect on them and on the results of the study. This is known as the Hawthorne effect.

In a FDA presentation, the Hawthorne Effect is described as one of the main contributor to the placebo effect.

In clinical trial auditing/inspection, the observer effect needs to be minimized. “When you observe the inspection, you change the dynamics of the situation. When people are aware they are being observed, they act differently than they would if they were not being observed. The problem is not unique to this program. It has sometimes been referred to as the experimenter effect, Hawthorne effect, restaurant manners effect, and job interview effect.”
Reference: De Amici D, Klersy C, Ramajoli F, Brustia L and Politi P (2000) impact of the Hawthorne effect in a longitudinal clinical study. The case of anesthesia. Control Clin Trials. 21: 103-14

Hello-goodbye effect

This is part of the psychodynamics of some people, whereby they initially present themselves in the worst possible light in order to become eligible for treatment, and then after treatment present themselves in the best possible light in an attempt to signal substantial gain in their condition. If uncorrected, the health gain of the patient will be overestimated. The hello-goodbye effect is a serious threat to the validity of clinical studies, meta-analyses and policy making.


John Henry effect

The John Henry effect is an experimental bias introduced into social experiments by reactive behavior by the control group.

In a controlled social experiment if a control is aware of their status as members of the control group and is able to compare their performance with that of the treatment group, members of the control group may actively work harder to overcome the "disadvantage" of being in the control group.

For example, if in an educational trial where the school classes who are in the treatment receive an extra support teacher, students who are in the control group may be induced to work harder to overcome that disadvantage.

The term was first used by Gary Saretsky (1972) to describe the behavior of a legendary American steel driver in the 1870s who, when he heard his output was being compared with that of a steam drill, worked so hard to outperform the machine he died in the process.

Pygmalion effect

The Pygmalion effect, or Rosenthal effect, is the phenomenon whereby the greater the expectation placed upon people, the better they perform. The effect is named after the Greek myth of Pygmalion, a sculptor who fell in love with a statue he had carved.