Peer-reviewed Journal Editors' Views on Real-world Evidence

Authors: Oehrlein EM, Graff JS, Perfetto EM, Mullins CD, Dubois RW, Onukwugha CAE
Publication: International Journal of Technology Assessment in Health Care, published online February 8, 2018
 

Peer-reviewed publication is a critical step in the path of translating research results into action. However, with the increased availability and focus on real-world evidence (RWE), it is unclear how these types of data are perceived by peer-reviewed journals and editors. A study published in the International Journal of Technology Assessment in Health Care reveals that RWE is considered valuable by the editors of peer-reviewed journals—if it meets certain criteria for quality. Researchers from the Department of Pharmaceutical Health Services Research at the University of Maryland School of Pharmacy and the National Pharmaceutical Council assessed attitudes of editors of peer-reviewed journals to answer these questions:

• What is the value of RWE and how does it compare to other types of studies, such as randomized controlled trials (RCT)?
• What education and/or resources do journal editors provide to their peer reviewers or perceive as needed?

What is the perceived value of RWE in peer-reviewed journals? Fifteen journal editors who participated in telephone interviews reported that they receive more RWE than RCT manuscript submissions—in some cases, nearly twice as many. They do not differentiate in the way they review the manuscripts, but rather follow the same editorial process to assess the quality of submissions regardless of study design.

The editors find RWE, in general, to be valuable, offering these advantages:

• Ability to complement RCT evidence
• Ability to assess the impact of interventions in the real world
• Ability to understand treatment effects among more diverse, representative populations.

Some editors felt that RWE also has its disadvantages, including a lack of high-quality data and less established methodological standards for RWE compared to other study designs.

How do journal editors perceive RWE studies compare to other study types such as RCTs? Journal editors noted that the characteristics of high-quality RWE studies are similar to other studies such as RCTs. For example, editors reported that high-quality RWE manuscripts are based on whether the study asks important questions, fills a gap in knowledge, can be generalized to other care settings or populations, clearly describes the methods, and utilizes a data source that is aligned with the research question. However, while editors highlighted features associated with high-quality RCTs (e.g., investigator blinding and recruitment, protocol violations, and study follow-up periods), editors did not mention features associated with high-quality RWE (e.g., methods to ensure similar patient populations, confirm treatment exposure, or address missing data).

What education and/or resources do journal editors provide to their peer reviewers? Editors reported difficulty in finding qualified reviewers. They do not frequently use training, tools, or checklists to assist peer reviewers, relying on existing peer reviewer knowledge. They identified a need for tools to help them make decisions on manuscripts more quickly and provide constructive feedback to authors and opportunities to improve the rigor and transparency of RWE studies.

About the research: A mixed methods approach was used to gather peer-reviewed journal editors’ perceptions. Editors of 79 peer-reviewed journals in general medicine, specialty medicine, and health policy/services research were contacted in late 2015 to participate in the study. Fifteen journal editors were interviewed by telephone and surveyed online, while two additional editors took the online survey. Eight editors attended an in-person roundtable discussion in March of 2016.