Why Value Framework Assessments Arrive at Different Conclusions: A Multiple Myeloma Case Study

Authors: Kimberly Westrich, MA; Lisabeth Buelt, MPH; and Robert W. Dubois, MD, PhD.
Publication: Journal of Managed Care & Specialty Pharmacy, May 25, 2017

As the United States transitions from a volume-based health care system to one that rewards value, new value assessment frameworks are emerging to support various types of decision-making. These assessments have the potential to substantially affect patients, whether as tools for shared decision-making with their doctors, as an input to care pathways used by providers, or through payer use of the frameworks to make coverage or reimbursement decisions.

Building on a prior case study analysis by the Lewin Group, National Pharmaceutical Council (NPC) researchers conducted cross-framework comparisons of multiple myeloma assessments using value assessment frameworks from the American Society of Clinical Oncology (ASCO), the Institute for Clinical and Economic Review (ICER), the Memorial Sloan Kettering Cancer Center (DrugAbacus), and the National Comprehensive Cancer Network (NCCN), and examined the consistency of findings across three case studies.

Although one might expect the various multiple myeloma value assessments to arrive at generally consistent results, this study indicates that consistency varied across our case studies. Only a third of the assessment results were clearly consistent. Areas of inconsistency included conflicting efficacy and toxicity assessments, as well as differing interpretations of the quality of the clinical evidence. For some areas, consistency could not be determined due to a lack of transparency around methods and evidence, or due to incomparable measures of cost.

At best, lack of consistency can lead to confusion among end users. At worst, it could lead to misinformed or conflicting health care decisions. Both scenarios suggest that these tools are not ready for widespread adoption and use for the purposes of policy development and health care decision-making.

Researchers suggested two key opportunities to improve the quality and utility of value assessment frameworks:

Transparency impacts the reproducibility and interpretation of assessment results. Full transparency on the specific body of evidence underlying an assessment, how various pieces of that evidence drove an assessment’s results, the process by which the evidence was transformed into components of the assessment, and the actual values of those components would enhance the accessibility and credibility of value framework assessments.

Usability of the frameworks can be improved by ensuring assessment results are fit-for-purpose and providing more information on how to interpret and use the results. Guidance on using the results should also extend to who the appropriate users are, as payer-focused assessments may be misleading for patients and vice versa.

1. Improve transparency
   
   Transparency impacts the reproducibility and interpretation of assessment results. Full transparency on the specific body of evidence underlying an assessment, how various pieces of that evidence drove an assessment’s results, the process by which the evidence was transformed into components of the assessment, and the actual values of those components would enhance the accessibility and credibility of value framework assessments.
    
2. Enhance usability
   
   Usability of the frameworks can be improved by ensuring assessment results are fit-for-purpose and providing more information on how to interpret and use the results. Guidance on using the results should also extend to who the appropriate users are, as payer-focused assessments may be misleading for patients and vice versa.

By addressing these points, framework developers can deliver assessments that are more transparent, reproducible, and relevant for their respective target audiences.