NPC Comments on Request for Information: ClinicalTrials.gov Modernization (NOT-LM-20-003)

March 14, 2020

Dr. Rebecca Williams, PharmD, MPH
Acting Director, ClinicalTrials.gov
National Library of Medicine

Submitted electronically via https://nlmenterprise.co1.qualtrics.com/jfe/form/SV_e2rLEUAx99myump

Request for Information: ClinicalTrials.gov Modernization (NOT-LM-20-003) 

Dear Dr. Rebecca Williams,

We, the undersigned, applaud ClinicalTrials.gov as it celebrates its 20th anniversary and seeks input on how it may best serve the research community in the years to come as part of the National Library of Medicine roadmap for modernization. Over the past two decades, there has been a growing interest in using data from clinical practice, referred to as real-world data (RWD), as well as transforming these data into real-world evidence (RWE) to inform decision-making. Several federal agency initiatives in the United States provide funding to improve the generation and quality of RWD [1] or to evaluate the potential use of RWE for regulatory or reimbursement decision-making.[2,3]

In parallel, the types of studies registered on ClinicalTrials.gov have diversified. As of March 5, 2020, over 69,000 studies (21%) are observational studies—a type of RWE.[4] We commend the inclusion of these studies, however, some of these studies have unique characteristics due to the use of secondary data, which could make it harder to use the current version of the study registry. For the reasons we enumerate here would benefit from registration. For example, concerns exist that study investigators could change study outcomes, modify inclusion and exclusion criteria, or alter the analytic approaches, ultimately cherry-pick findings rather than publishing results from pre-specified analyses.[5] Recognizing these challenges, several groups have developed recommendations regarding the transparency of observational studies and specifically hypothesis-evaluating treatment effectiveness (HETE) studies.[6,7,8] Over the past year, the Real-World Evidence Transparency Partnership, a joint collaboration to establish a culture of transparency for study analysis and reporting of RWE HETE studies, was formed and outlined recommendations.[9]

One of the challenges is the ease of registering HETE studies on existing study registration sites. Study sponsors and investigators of HETE studies must modify study details when submitting information to the Protocol Registration and Results System, increasing the potential for errors and inconsistencies. For example, researchers must determine if the “Study Start Date” field should report the start of the pre-index observation period or the study index date. In other cases, for RWE studies, researchers often have good reasons to evaluate RWD sources prior to study initiation (e.g., to assess the data quality or completeness) or to inform the study design (e.g., feasibility counts, patterns of care, or treatment switching). The ability to describe the degree to which data exploration and analyses were conducted before the pre-specified analysis plan or the rationale for modifications are decided could improve the credibility of the reported findings. Finally, disincentives to register HETE studies exist due to 1) the broad availability of secondary data sources and 2) the ease with which researchers may be able to quickly start a parallel research effort on similar data to get results in a similar timeframe if given enough information. Enabling researchers to select when information is released (e.g., immediately, after the study analysis, or after a specified time period) and which users (e.g., regulators vs. other researchers) have access at various time points could promote the need for confidentiality and intellectual property protection while also ensuring transparency. A more fit-for-purpose registration for RWE HETE studies would improve the consistency and accuracy of the information submitted. 

The goals of ClinicalTrials.gov modernization and a more fit-for-purpose registration site for HETE studies are not mutually exclusive. While ClinicalTrials.gov was originally designed as a repository for patients to find information regarding clinical trials, the goals outlined for ClinicalTrials.gov apply to all studies. 

The availability of RWD and the use of RWE to guide decision-making are likely to grow in the next two decades. Broader flexibility to include evolving data sources and study designs such as RWE could streamline the site for investigators and improve the value of the information found on ClinicalTrials.gov for the people who rely on it. The Real-World Evidence Transparency Partnership shares the goals of increasing public trust in research and hopes to continue this critical dialogue. Thank you for the opportunity to provide comments on the Modernization of ClinicalTrials.gov. 

Sincerely, 

Jennifer S. Graff, PharmD 
Vice President, Comparative Effectiveness Research, 
National Pharmaceutical Council

Richard J. Willke, PhD 
Chief Science Officer
ISPOR

------------------------

[1] The Office of the National Coordinator for Health Information Technology. Draft 2020-2025 Federal Health IT Strategic Plan. Available at: https://www.healthit.gov/sites/default/files/page/2020-01/2020-2025FederalHealthIT%20StrategicPlan_0.pdf. Accessed March 1, 2020.

[2] 114th Congress. H.R.34 - 21st Century Cures Act. Published 2016. Available at: https://www.congress.gov/bill/114th-congress/house-bill/34. Accessed March 1, 2020.

[3] 84 FR 42044. Medicare Program; Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Policy Changes and Fiscal Year 2020 Rates; Quality Reporting Requirements for Specific Providers; Medicare and Medicaid Promoting Interoperability Programs Requirements for Eligible Hospitals and Critical Access Hospitals

[4] Trends, Charts, and Maps. Available at https://clinicaltrials.gov/ct2/resources/trends. Accessed March 4, 2020.

[5] National Academies of Medicine. Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. February 6, 2019. Available at: https://www.nap.edu/download/25352#. Accessed March 1, 2020.

[6] Berger ML, Sox H, Willke RJ, et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol Drug Saf. 2017;26(9):1033-1039

[7] National Pharmaceutical Council and Academy Health. Six Ways to Make Real-World Evidence Methods More Transparent. May 20, 2019. Available at: https://www.npcnow.org/blog/six-ways-make-real-world-evidence-methods-more-transparent. Accessed March 5, 2020.

[8] Duke Margolis Center for Health Policy. Understanding the Need for Non-Interventional Studies Using Secondary Data to Generate Real-World Evidence for Regulatory Decision-Making, and Demonstrating Their Credibility. November 25, 2019. Available at: https://healthpolicy.duke.edu/sites/default/files/u31/non-interventional_study_credibility_final.pdf. Accessed March 5, 2020.

[9] ISPOR. Improving Transparency in Non-Interventional Research for Hypothesis Testing—WHY, WHAT, and HOW: Considerations from The Real-World Evidence Transparency Initiative. Draft White Paper. September 18, 2019. Available at: https://www.ispor.org/docs/default-source/strategic-initiatives/improving-transparency-in-non-interventional-research-for-hypothesis-testing_final.pdf?sfvrsn=77fb4e97_6. Accessed March 5, 2020.

[10] National Library of Medicine. Request for Information (RFI): ClinicalTrials.gov Modernization (NOT-LM-20-003). December 30, 2019. Available at https://grants.nih.gov/grants/guide/notice-files/NOT-LM-20-003.html. Accessed March 4, 2020.