Win Ratio and its application in clinical trials

The win ratio is a method for examining composite endpoints in clinical trials. It was introduced in 2012 by Dr Pocock and has since been widely adopted in cardiovascular trials. The win ratio accounts for relative priorities of the components and allows the components to be different types of outcomes. The win ratio is calculated by dividing the total number of winners by the total number of losers.

The win ratio was motivated by the Finkelstein–Schoenfeld (FS) test, with the aim of providing an estimate of the treatment effect (the win ratio) and confidence interval, in addition to a P-value. The general principle behind both the Finkelstein-Schoenfeld test and the win ratio is that they are both methods for examining composite endpoints in clinical trials. Win ratio can be considered as a popular name for the Finkelstein-Schoenfeld test. See my previous post "Finkelstein-Schoenfeld Method, Win Ratio, and Hodges-Lehman Estimates - Statistical Methods Based on All Paired Comparisons". In practice, we may see the terms "Win Ratio test" and "Finkelstein-Schoenfeld test" interchangeably. 

Win Ratio is defined differently than hazard ratio in survival analysis, however, both Win Ratio and Hazard Ratio are important statistical measures used in clinical trials to assess treatment outcomes and compare different treatments. However, they have distinct interpretations and applications, making them suitable for different types of clinical trial data and research questions. Here are comparison table for Win Ratio and Hazard Ratio:

Metric	Win Ratio	Hazard Ratio
Definition	Ratio of treatment success in the experimental group to the control group	Compares the risk of an event occurring in the treatment group to the control group over time
Clinical trial design	Usually fixed-duration studies	Both fixed-duration studies and event-driven studies
Endpoint	Composite endpoint, all events are considered	Composite endpoint, usually time to the first event
Interpretation	Great than 1: Experimental treatment has higher success rate than control group Equal to 1: Both groups have the same success rate<br> Less than 1: Experimental treatment has lower success rate than control group	Greater than 1: Higher risk of event in treatment group<br> Equal to 1: Equal risk of event in both groups<br> Less than 1: Lower risk of event in treatment group
Application	Non-inferiority or superiority trials to assess treatment efficacy	Survival analysis with time-to-event outcomes (e.g., overall survival, progression-free survival)
Focus	Treatment success rates	Risk of an event over time
Type of Data	Binary or categorical data	Time-to-event data

For win-ratio and hazard ratio comparison, please see the article by Ferreira et al "Use of the Win Ratio in Cardiovascular Trials".

Win Ratio approach has been used to re-analyze the clinical trials with a composite endpoint (either the primary efficacy or secondary efficacy endpoint). There is a table in Redfors et al 2020 paper that summarized the re-analysis results using the Win Ratio approach. We recently published a paper in Annals of American Thoracic Society "A Novel Approach to Clinical Change Endpoints: A Win Ratio Analysis of the INCREASE Trial" where the Win Ratio approach was used to analyze the secondary efficacy endpoint of clinical worsening events.

Win Ratio approach has also been prespecified as the primary analysis method to analyze the clinical trials with composite endpoints. A table in Redfors et al 2020 paper summarized seven different trials including ATTR-ACT, Chart-1, and TAVR-UNLOAD trials where the Win Ratio approach was listed as the primary analysis method.

This past week, Biotech company, Bridgebio Pharma announced their phase III study of acoramidis in transthyretin amyloid cardiomyopathy, or ATTR-CM. One of the primary efficacy endpoint is a composite endpoint - a hierarchical combination of 1) all-cause mortality, 2) cumulative frequency of cardiovascular-related hospitalization, 3) change from baseline in NT-proBNP, and 4) change from baseline in 6MWT over a 30-month fixed treatment duration. According to clinicaltrials.gov, the composite endpoint is analyzed using the following approach: 

Each subject will be compared to every other subject within a stratum over outcomes of all-cause mortality (death due to any cause), cumulative frequency of cardiovascular-related hospitalizations (number of times a subject is hospitalized for cardiovascular-related causes), change from baseline in NT-proBNP, and change from baseline in the total distance walked in 6 minutes (distance in meters).

The hierarchical approach with the Finkelstein-Schoenfeld test will be applied and the test recognizes the greater importance of the mortality endpoint. Scores are transformed to -1, 0, +1. The alternative hypothesis is a subject in the acoramidis treatment group will have a greater score than a subject in the placebo group.

In the company's presentation slides, results for the primary endpoint and secondary endpoints are listed. The win ratio is listed as 1.8, which indicates 80% more wins in acoramidis treatment group than in the placebo group.

The win ratio test requires a pairwise comparison (each subject in the treatment group is compared with each subject in the placebo (control) group). The number of pairs is the sample size (n) for the treatment group times the sample size (m) for the placebo group, i.e., n x m pairs. The win ratio test results are largely dependent on the rules or algorithms in determining the win/loss/tie for each individual pair. Therefore, the rules and algorithm need to be pre-specified. For the prospective clinical trials, these rules and algorithms need to be pre-specified in the statistical analysis plan (SAP) and the SAP needs to be submitted to FDA for review to avoid the potential biases or potential impression of biases. 

 

References:

• Pocock et al (2012) The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities.
• Redfors et al, 2020 The win ratio approach for composite endpoints: practical guidance based on previous experience
  
• Ferreira et al (2020) Use of the Win Ratio in Cardiovascular Trials
• Maurer et al (2018) Tafamidis Treatment for Patients with Transthyretin Amyloid Cardiomyopathy

GCP violation and tampering with evidence

Last week, biotech company BioXcel reported positive efficacy results of their investigational product Igalmi in the treatment of Alzheimer's Agitation, but the supposed good news was shadowed by data integrity issues. On the same day, the company revealed allegations that an investigator had failed to adhere to trial protocol and was alleged to have fabricated emails to cover their tracks.

See the article "Missed trial protocols, fabricated emails and failed endpoint mar BioXcel's Alzheimer's agitation readout".

The company filed Form 8-K and revealed the GCP violation issues. Here are the excerpts from the SEC filing: 

          Important Information Regarding TRANQUILITY II Phase 3 Clinical Trial 

In December 2022, the U.S. Food and Drug Administration (“FDA”) conducted an inspection of one of the clinical trial sites in the Phase 3 TRANQUILITY II clinical trial, where the principal investigator enrolled approximately 40% of the subjects participating in the trial. At the conclusion of this inspection, the FDA issued an FDA Form 483 identifying three inspectional observations. These observations related to the principal investigator’s failure to adhere to the informed consent form approved by the Institutional Review Board for a limited number of subjects whose records the FDA reviewed, maintain adequate case histories for certain patients whose records the FDA reviewed, and adhere to the investigational plan in certain instances. For example, the FDA cited the principal investigator’s delay in informing the sponsor’s medical monitor or pharmacovigilance safety vendor of a serious adverse event (“SAE”) for one of the subjects, which report was made to the Company’s vendor outside of the 24 hour time period prescribed by the clinical trial protocol. The principal investigator for this clinical site responded to the FDA observations within the time period requested. The FDA inspection remains open, however, as the FDA has not issued an Establishment Inspection Report. 

In May 2023, it came to the Company’s attention that this same principal investigator in the TRANQUILITY II clinical trial may have fabricated email correspondence purporting to demonstrate that the investigator timely submitted to the Company’s pharmacovigilance safety vendor a report of an SAE from a different subject than the one cited in the FDA Form 483, and purporting to show that the vendor had confirmed receipt. Upon receipt of this information, the Company promptly initiated an investigation and recently received confirmation that the principal investigator fabricated the email correspondence related to the timing of the reporting of this SAE to the Company’s pharmacovigilance vendor to make it appear as though this SAE had been timely reported to the pharmacovigilance vendor as required by the clinical trial protocol. The Company also confirmed that this SAE had been timely entered into the electronic data capture system, even though the SAE had not been separately reported to the Company’s pharmacovigilance safety vendor within the 24 hour timeframe required under the protocol. 

In connection with this ongoing investigation, the Company was made aware that the fabricated email correspondence was provided to the FDA by the principal investigator’s employer during the on-site inspection in December 2022. After unblinding of the data, the Company determined that the SAE that was the subject of this fabricated correspondence between the principal investigator and the Company’s pharmacovigilance vendor occurred in a subject in the placebo arm. This principal investigator has not participated in any other clinical trial sponsored or conducted by the Company. Moreover, the study was designed such that trained study staff other than principal investigators were to conduct assessments of the primary efficacy measure.

In this case, the fabrication of evidence to hide the late reporting of the SAE to the sponsor is far more serious than the late reporting of the SAE itself. The situation is similar to tampering with evidence and obstruction of justice in the criminal law. 

ICH E6 Good Clinical Practice requires the investigational site to report the serious adverse events (SAEs) to the sponsor in a timely manner usually within 24 hours when the site staff become aware of the occurrence of an SAE. Failure to do so will be recorded as a major protocol deviation. 

However, fabricating the evidence to conceal the protocol deviation (late reporting of the SAE) is no longer a GCP compliance issue, it is a scientific misconduct issue.