Six Ways to Make Real-world Evidence Methods More Transparent

Thanks to technology, health researchers now have enormous amounts of data available at their fingertips. Data from electronic health records, clinical data from laboratories, diagnostic testing, claims data and pharmacy dispensing data all can yield important information about how treatments are working in patients going about their everyday lives. Yet acceptance of studies based on this “real-world data” remains limited, partly because the research methods used to generate usable evidence are not always transparent. That perception could change thanks to new efforts by the National Pharmaceutical Council (NPC) and AcademyHealth aimed at improving the credibility and reliability of real-world evidence (RWE).

Both NPC and AcademyHealth recognized that by making RWE research methods more transparent, they could improve efforts to reproduce studies, disclose key study methods and reduce skepticism about the ultimate results. However, there is little agreement in the scientific community about how to achieve these goals. Through individual interviews and a panel discussion with a broad group of researchers, they developed six actionable recommendations to enhance research methods transparency and credibility. These include:

• The hypothesis should be specified in advance of conducting the research and logged in a centralized database or repository.
• Similarly, the real-world evidence analysis plan should be specified in advance and logged in a central place. Any changes from the plan, along with the rationale for making those changes, should be documented.
• Steps should be documented to assure the available data being used for the study are feasible, appropriate and of high quality for use given the research question and the hypothesis.
• Sensitivity analyses should be performed on key definitions and outcomes.
• Data coding should be made available upon request, along with a natural language description of the coding logic, so that other research teams could replicate the study in an appropriate secondary dataset.
• Access to summary tables of data should be provided.

NPC and AcademyHealth view these recommendations as a guide to help end-users of RWE assess whether they can trust and rely on the studies for making various health care decisions. Ensuring these recommendations are more broadly accepted, however, could require additional administrative processes and human resources, along with considerations of privacy concerns, protection of patient information, and mitigation of misuse of data where applicable.

NPC and AcademyHealth are not alone in their efforts. They also are part of the RWE Transparency Initiative Partnership coordinated by International Society for Pharmacoeconomics and Outcomes Research (ISPOR), which is examining how to foster more transparent research methods. The steering committee, which includes organizations such as the Duke-Margolis Center, the European Medicines Agency and the UK’s National Institute for Health and Care Excellence, will be hosting a panel session during ISPOR’s annual meeting on Monday, May 20, 2019.