Data Query on Diary and Patient Reported Outcome?

We keep hearing that for patient report outcome (PRO) including patient diary data, there is no query or data clarification required and the diary data is source data and can not be queried. For example, in a paper titled “How to clean up dirty data in Patient reported outcomes”, it said “The investigator is not allowed to query any of the patient's answers which leads in general to a lot of dirty data.”. Is it true that the diary or patient reported outcome can not be queried at any circumstance no matter how horrible the data quality is? It is not true and this popular perception is just not true.

Data clarification or data query is an essential process in clinical data management to ensure that the questionable data are corrected. According to EMA reflection paper ON EXPECTATIONS FOR ELECTRONIC SOURCE DOCUMENTS USED IN CLINICAL TRIALS, “Data clarification is part of the process to ensure complete data and the clarification process is one of the processes underlining the need for the maintenance of the audit trail.”

In a clinical study with paper-based case report form, the data clarification is typically issued by the data managers to the investigation sites. The investigator then reviews the issues to provide the responses to the data query. Data managers or clinical monitors can not directly make the changes to the data without issuing query and getting approval from the investigation site. In a clinical study with electronic data capture (EDC), the data clarification/query process is built into the EDC system. The data is entered at the investigation site. The query is issued by data managers or study monitors within EDC system. The investigator will then provide the responses to the query also within EDC system and make the data corrections.

The process becomes vague for patient reported outcome (PRO) or patient diary (no matter it is on the paper or electronic). The key difference for PRO data is that the data is directly recorded or entered by the subject or patient. There is a perception that no matter how poorly the data is, there is no data clarification or query process for PRO data or diary data.

In many clinical trials, the study endpoints rely on the collection of the information provided by the patient (daily symptoms, daily activities, quality of life,…). For example, for clinical studies on urinary incontinence, the primary efficacy endpoint may be the frequency of urinary incontinence episodes (UIE) per week as determined from patient daily diary. For clinical studies on female sexual dysfunction, the clinical endpoint may be the sexual events or encounters recorded daily by the study subjects using diaries. .

I used to work on a clinical trial with irritable bowel syndrome indication where the study endpoints were collected by a touch-tone telephone system (IVR – interactive voice response system). Efficacy parameters were symptom relief, abdominal discomfort or pain, bloating, stool frequency, stool consistency, straining and urgency recorded through IVR on daily basis by the study subjects. For stool frequency, the subject were asked “how many bowel movements did you have today?” As a statistician, I had to check the outliers before analyzing the data. I found some entries with the number of bowel movements being extremely high (55, 66). When I discussed these impossible numbers with the study manager, I was told that patient diary could not be queried. I pointed out that if these obvious data errors could not be corrected, the study data would be severely compromised. Later, we identified many more entries with duplicate numbers (such as 11, 22, 33, 44, 55, 66). After inquiring to the study subjects, it was found that the subject pressed the telephone number key twice for 1, 2, 3, 4, 5, and 6. This is just an example showing that the diary data could be easily recorded wrongly in the database and the query for accuracy is necessary.

FDA’s guidance on “Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims” touched on ensuring the quality of PRO data. In discussion about the ePRO, FDA guidance indicated the concern about clinical investigator inability to maintain and confirm electronic PRO data accuracy. It said “the data maintained by the clinical investigator should include an audit trail to capture any changes made to the electronic PRO data at any point in time after it leaves the patient’s electronic device.” Furthermore, the guidance indicated the concern about ability of any entity other than the investigator (and/or site staff designated by the investigator) to modify the source data”. These statements inexplicitly imply that the diary data can be queried and modified by the clinical investigator (and/or site staff designated by the investigator) and the clinical investigator has responsibility to ensure the completeness and accuracy of the diary data.

Adaptive Licensing for Drug Approval

From this year’s Drug Information Association (DIA) annual conference in Chicago, I learned a new concept of “Adaptive Licensing”. According to http://www.bnid.org/node/6921, “in the past five years, a wave of proposals for reform of drug licensing has emerged in the EU, US, and Canada under the labels of staggered approval, adaptive licensing, managed entry and progressive authorization. Through iterative phases of information gathering followed by regulatory evaluation and correction, these approaches seek to align licensing decisions on market access of drugs with emerging information on benefits and harms of drugs as actual used. It is hoped that this approach will provide patients with earlier access to innovative drugs to address unmet medical needs, better management of known risks, and improved detection of unanticipated adverse effects that emerge in use. “

The current drug licensing process is called ‘phased approach’ which requires the sponsors to conduct a series of clinical trials from phase I to phase III to establish the safety/tolerability and to confirm the efficacy before the regulatory agencies can consider the approval for marketing authorization of a product. This phased approach and the purpose of each phase of the clinical trial are discussed in this free web article. Recently, with the adaptive design concept, we are trying to break the traditional phased approach. The seamless phase II/III studies or seamless phase I/II studies have been much discussed and debated. Even with adaptive design, before a drug can be approved, a series of clinical trials are still required. Let’s now call “learning and confirming”: from early trial for learning to late stage trial for confirmation of the safety and efficacy with Adequate and well-controlled clinical study(ies) (A&WC).

With the phased drug approval approach, there will be a magic moment during the drug approval process. This magic moment is the regulatory reviewer’s action date (or decision date) - The date tells when a regulatory action, such as an original or supplemental approval, takes place. The regulatory agencies will be based on the review of data from pre-marketing clinical studies to make a decision of approving or not approving a production for market authorization. If the efficacy and safety have been demonstrated, the product is approved; if the efficacy and safety have not been sufficiently demonstrated, additional clinical studies may be requested; if the efficacy and safety are not demonstrated, the application of market authorization will be denied.

The adaptive licensing is trying to remove this magic moment from the drug approval process and instead considers the drug licensing as a continuous process. Whether or not a product should be authorized for marketing depends on the risk-benefit ratio. When the benefit outweighs the risk, the product should be approved; when the risk outweighs the benefit, the product should not be approved. For an approved product, if the benefit/risk ratio becomes unfavorable, the product should be withdrawn from the market. I liken this continuous adaptive drug licensing process to the p-value assessment in statistics. The magic moment is like the magic number of p=0.05 (p-value should be a continuous number and p=0.051 (not significant) may not be different from p=0.049 (significant)).

It looks like EU is pioneering in implementing the adaptive licensing. EMA has started to work on Adaptive Licensing to Reach Roadmap Goals (Roadmap to 2015) . However, in US, there is a similar program called “Accelerated Approval of New Drugs for Serious or Life-Threatening Illnesses”. With this program, “FDA may grant marketing approval for a new drug product on the basis of adequate and well-controlled clinical trials establishing that the drug product has an effect on a surrogate endpoint that is reasonably likely, based on epidemiologic, therapeutic, pathophysiologic, or other evidence, to predict clinical benefit or on the basis of an effect on a clinical endpoint other than survival or irreversible morbidity. Approval under this section will be subject to the requirement that the applicant study the drug further, to verify and describe its clinical benefit, where there is uncertainty as to the relation of the surrogate endpoint to clinical benefit, or of the observed clinical benefit to ultimate outcome. Postmarketing studies would usually be studies already underway. When required to be conducted, such studies must also be adequate and well-controlled. The applicant shall carry out any such studies with due diligence.”

However, the recent debates on whether or not Avastin (Bevicuzmab) should be withdrawn from the market for breast cancer demonstrated how difficult to implement this program. Based on the established rule, if a drug is approved through ‘accelerated approval’, the approval is conditional and can be withdrawn from the market if the drug is later showed to be ineffective or with unfavorable benefit/risk ratio. Avastin in breast cancer case just showed how difficult to withdraw a product due to ineffectiveness. In even worse situation, the drug companies may not fulfill the commitment to finish the follow-up studies. There have been several cases of marketing withdrawal due to safety concerns (for example, Vioxx), but it seems to be more difficult to withdraw a product from the market due to the efficacy concern.

Adaptive licensing may be the future direction for drug approval process; however, many issues need to be considered and resolved before this new process can be implemented.

• What is the impact of adaptive licensing on patent expiration?
• What is the tipping point the conditional approval can be granted?
• How to assess the benefit/risk ratio with sound scientific / statistical approaches (ie avoiding subjective assessment in benefit/risk ratio)?
• What if the committed follow-up studies never completed?
• How to deal with the difficulties to withdrew a product when other factors (for example, emotional effect in Avastin case) are kicked in?
• Adaptive Licensing: Taking the Next Step in the Evolution of Drug Approval
• Commentary by Jane Woodcock