Questions Abound on the Adoption of Evidence

Medical decision making should rely on evidence (when available) to ensure that treatments are appropriate and effective for patients. Given the substantial amount of funding being committed to comparative effectiveness research (CER) it is important to understand the characteristics of research and “milieu” that may determine whether or not the evidence is adopted into practice.  Research -- even very rigorous research -- that is not adopted into practice will be an unsatisfying exercise. One approach to developing this understanding is to examine how and why previous studies may or may not have had an impact. As the Patient-Centered Outcomes Research Institute (PCORI) considers funding of future projects, a look back at the types of studies that have proven influential could be informative. Ultimately, the success of CER funding may be judged by the impact of the research on improving clinical practice and patient care.

Historically, some studies have had immediate impact. The Women’s Health Initiative (WHI), which examined hormone use in menopausal women, yielded findings that quickly and dramatically changed practice. While additional research and analyses of the WHI results challenge some of the initial findings, there is little disagreement that practice changed (at least for a while) after the WHI publication. Other studies, such as those demonstrating the benefits of statins in cardiovascular disease, were slow in adoption.

There are many questions that must be wrestled with to understand why or why not a research finding was adopted into practice. For example, as new CER reports become available, potentially with diverse findings for a particular therapy, the question is was there enough evidence to make a decision? When does a payer have adequate evidence to approve, deny, or grant conditional coverage? When is evidence sufficient for providers to incorporate the evidence into decisions about therapy for individual patients? How will the evidence affect patients’ willingness to accept the treatment recommended and have confidence in the expected therapeutic benefits? All of these questions factor into evidence adoption.

Additionally, a hard look at experience with studies already completed suggests a number of questions and issues that are likely to influence the ways in which payers, providers, and patients may be affected by future comparative studies. Naturally, several are related to the convincing nature of the evidence. Does the evidence consist of one study or a number of studies, and if the latter, are the results consistent? Does the evidence provide a direct answer to the important clinical question or does the answer rely on connecting the research results to other clinical knowledge? How precise are the findings? Are there other factors, measured or unmeasured, in the research that may have biased and thus affected the reliability of the results?

Less often considered are several other issues that form the contextual background for the research and may have important implications for its acceptance and the adoption of its findings. These include:

• Is the population being studied likely to adopt CER findings?
• How much better is the new therapy compared to existing ones?
• What is the “value” of the new therapy and how does it compare to other therapies?
• What is the downside of being “wrong” in interpreting the evidence -- i.e., the consequences of turning down an effective therapy or incorrectly choosing one that is not best?  
• What would it cost for more studies that could provide clearer guidance on therapeutic choices?
• How long would it take to get that additional information?

These are issues that we will need to consider in understanding the linkage between research and adoption.  There will not be clear answers for all questions, and the importance of the issues will vary across payers, providers, and patients.  But as unanswered clinical questions are considered for new research, the structure and prioritization and dissemination for this research is likely to benefit by consideration of these issues and the effects they have had on adoption of prior research.