Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing—Why, What, and How Recommendations and a Roadmap from the Real-World Evidence Transparency Initiative

Authors: Berger M, Crown W, Daniel G, Eichler H, Goettsch W, Graff J, Guerino J, Jonsson P, Lederer NM, Monz B, Mullins CD, Schneeweiss S, Van Brunt D, Wang SV,
Willke RJ
Publication: Value in Health and Pharmacoepidemiology and Drug Safety
Date: September 16, 2020

A position paper jointly published in Value in Health and Pharmacoepidemiology and Drug Safety describes a plan for improving the transparency of the research process and making registration of real-world evidence (RWE) study methods easier and more routine. The paper, developed by the RWE Transparency Initiative, a partnership between the International Society for Pharmacoeconomics and Outcomes Research, the International Society for Pharmacoepidemiology, the National Pharmaceutical Council, and the Duke-Margolis Center for Health Policy and international experts, outlines why greater research transparency, including study registration, is needed, describes what information to register and when, and provides a roadmap for how incentives can encourage registration. 

Why this Matters:

Interest is growing in the use of data from clinical practice, referred to as real-world data, to generate RWE and inform regulatory, coverage and reimbursement decisions. High-quality RWE studies — when done with high-quality data and good research methods — can fill gaps in knowledge and complement clinical trials. However, the lack of transparency in the study methods used can make it difficult for decision-makers to assess the quality and credibility of RWE studies. Study registration — particularly for RWE studies evaluating medical treatment effectiveness and safety using existing or secondary data — has been proposed to improve research transparency and build trust in the study findings.

Summary Recommendations from the RWE Transparency Initiative

Why Greater Methods Transparency and Study Registration is Needed: Barriers to the routine acceptance and use of RWE exist. Many RWE studies originate from existing or secondary data sources such as electronic health records or administrative claims, where there is a risk that the results reported may reflect the most positive or impressive results rather than pre-specified analyses. Decision-makers such as the Food and Drug Administration and health plans seek greater transparency on study design and analyses to evaluate the study methods.

Some researchers currently submit RWE study synopses and protocols to existing research registries such as the Clinicaltrials.gov or the European Union Register of Post-Authorisation Studies (EU PAS). However, study registration is neither easy nor routine.

What Registration Should Involve and When to Register Studies: Any RWE study can be registered; however, RWE studies vary in the type of data used and the study purpose. Studies which use existing or secondary data sources and evaluate treatment effects using hypothesis-based methods to guide population decision-making are most likely to face scrutiny and should be prioritized for registration.

Study registration begins with the submission of a limited number of study characteristics. To avoid duplication and create synergies, the recommendations build upon other collaborations and efforts underway at the Food and Drug Administration. Concerns regarding intellectual property must balance the desire to register study hypotheses and analytic plans prior to study analysis with the public release of this information. Time-stamps of submission will enable an audit trail of changes as needed.

How to Encourage Transparency and Study Registration:

1. Identify a site to register RWE studies that evaluate treatment effects and use secondary data. Existing sites are “good enough” for some registration even if originally designed for other purposes. Focus on RWE studies evaluating hypotheses to support decision-making.
2. Determine the characteristics of a “good” registration process. A good registration should consider the researcher and reviewer workload and key elements, while balancing transparency and confidentiality, and allowing for an audit trail of changes.
3. Provide incentives for routine registration of RWE hypothesis evaluating studies. Registration alone will not be successful without incentives from research funders, peer-review journals, regulators, payers, and those who assess health technologies.

A full description of the considerations and recommended next steps are included in the full paper. As the potential use of RWE to support decision making for regulatory decisions, clinical guidelines and coverage and reimbursement grows, the need to trust that evidence grows correspondingly. Improving the culture of transparency can help shed light on RWE study practices so that users of the results can better determine study quality for themselves.

Background:

In 2019, a group of thirty international experts met to explore the structural and practical challenges to improve the transparency of RWE research methods. The meeting led to the creation of the RWE Transparency Initiative, a partnership between the International Society for Pharmacoeconomics and Outcomes Research, the International Society for Pharmacoepidemiology, the National Pharmaceutical Council, and the Duke-Margolis Center for Health Policy, which seeks to establish a culture of transparency for study analysis and reporting. The position paper also received endorsement from the International Society of Pharmacoepidemiology.