Online Tools to Synthesize Real-World Evidence of Comparative Effectiveness Research to Enhance Formulary Decision Making

The authors describe and compare the features and characteristics of five tools to evaluate real-world evidence studies and add clarity on what the tools provide.

NPC Logo Placeholder Image

High-quality real-world evidence (RWE) studies — when done with high-quality data and good research methods — can fill gaps in knowledge and inform routine care and coverage decisions. However, many health plans and pharmacy and medical directors report feeling ill-prepared to evaluate and use these studies in decision-making. To aid health plans and other decision-makers several research organizations, group collaborations and professional societies have developed tools to help decision-makers evaluate and RWE studies. However, it’s not always clear which tools may be most useful.

A new Best Practices article published in the Journal of Managed Care & Specialty Pharmacy compares five tools to evaluate RWE studies. These tools can help decision-makers better identify high-quality studies to help inform broader health decisions. The authors describe and compare the features and characteristics of the tools and add clarity on what the tools provide.

Findings:

Several tools are available to assist health care decision-makers in assessing the quality and usefulness of the RWE. These include:

Several tools are available to assist health care decision-makers in assessing the quality and usefulness of the RWE. These include:

  1. Good ReseArch for Comparative Effectiveness (GRACE) Checklist
    Derived from a set of principles that define the elements of good practice for the design, conduct, analysis and reporting of observational RWE studies. This PDF Checklist aids stakeholders in evaluating the adequacy of RWE data quality and methods.
     
  2. IMI GetReal RWE Navigator (RWE Navigator)
    Designed to increase the quality of RWE generation in medicines development and the regulatory/health technology assessment (HTA) processes. The RWE Navigator is a collection of online educational resources to guide demonstration of effectiveness, a guidance tool to support development of medicines, and a directory of authoritative resources for RWE data and methods.
     
  3. Center for Medical Technology Policy (CMTP) RWE Decoder
    Developed to help decision-makers assess RWE. The Excel® spreadsheet facilitates user assessment of both the relevance and rigor of existing evidence from randomized controlled trials and RWE for coverage decision-making.
     
  4. CER Collaborative tool
    Helps users assess a study’s relevance and credibility and subsequently grade a body of evidence including RWE. The online CER Collaborative Initiative aims to provide greater uniformity and transparency in the evaluation of evidence to inform coverage decisions.
     
  5. Real World Evidence Assessments and Needs Guidance (REAdi)
    Provides guidance on the use of RWE for HTA decision making for drug and diagnostic interventions and leads users through the decision-making process in five phases. The READi package aids in refinement of the research question, study retrieval, quality assessment, grading the body of evidence, and questions to facilitate coverage decisions.

All five tools specify a framework, were designed with stakeholder input, assess internal validity, are available online and are easy to use. However, the tools vary in their objective (e.g., to aid health care coverage decisions vs. increase the awareness of RWE), complexity (e.g., number of questions and knowledge required) and comprehensiveness (e.g., ability to evaluate a single RWE study vs. grading a body of evidence which may include RWE and other studies).

  • The simplest to use, the GRACE checklist, provides a simple quality rating, while the Navigator provides education, guidance and resources.
     
  • The three remaining tools – RWE Decoder, CER Collaborative tool and REAdi tool – are like each other in that they integrate quality ratings, education and guidance resources. They are therefore more complex and are intended to evaluate a body of RWE to enhance formulary decision making.

These tools make the evaluation of RWE studies less challenging, allowing users to determine which tool best fits their purpose. A more complete and consistent use of the evidence can help improve decision-making and patient health.