NPC Comments on ASCO Framework

August 21, 2015

Richard L. Schilsky, MD, FASCO Chief Medical Officer American Society of Clinical Oncology 2318 Mill Road, Suite 800 Alexandria, VA 22314

Lowell E. Schnipper, MD, FASCO Chair, ASCO’s Value in Cancer Care Task Force Chief, Hematology/Oncology Beth Israel Deaconess Medical Center

RE: ASCO Conceptual Framework to Assess the Value of Cancer Treatment Options

Submitted electronically via: [email protected] and [email protected]

Dear Dr. Schilsky and Dr. Schnipper:

The National Pharmaceutical Council (NPC) appreciates the opportunity to offer feedback to the American Society of Clinical Oncology (ASCO) and the Value in Cancer Care Task Force (VCCTF) on the Conceptual Framework to Assess the Value of Cancer Treatment Options.1

NPC is a health policy research organization dedicated to the advancement of good evidence and science, and to fostering an environment in the United States that supports medical innovation. NPC is supported by major U.S. research-based biopharmaceutical companies. NPC focuses on research development, information dissemination, education and communication of the critical issues of evidence, innovation and the value of medicines for patients. Our research helps inform critical healthcare policy debates and supports the achievement of the best patient outcomes in the most efficient way possible. 

The ASCO/VCCTF value framework was designed with the laudable goal of supporting shared decision-making between patients and physicians. NPC supports the spirit of this goal and applauds the concept of helping individual patients understand the evidence, innovation and value of medicines to them. We commend ASCO and VCCTF for seeking broad stakeholder input and utilizing an inclusive process in the development of the draft framework. We are pleased to see that some of the input NPC shared with ASCO in August 2014 was incorporated into the current draft of the framework.

NPC offers the following comments as suggestions to create a more patient-centric framework that is better equipped to inform shared decision-making among patients and their physicians. We would be pleased to work directly with ASCO and VCCTF as you continue to refine the framework.

I.        Ensure Framework Is Used as Intended

NPC recognizes the considerable work that has gone into the framework and the care with which the purpose has been identified. It is important to incorporate safeguards to ensure the framework is understood and used appropriately by patients and their physicians; it is similarly important to protect against (mis)use of the framework by anyone other than the intended users.

Minimize the Risk of Erroneous Comparisons

Health technology assessment (HTA) is a very complex and sophisticated undertaking. Decades of work have shaped the methods and levels of sophistication; accordingly HTA has not been boiled down into simple algorithms and web-based tools. Such over-simplification risks creating confusion as well as inappropriate comparisons and conclusions.

The framework calculates “Net Health Benefit” (NHB) for a treatment relative to a comparator from a clinical trial. This enables comparisons between the treatment and the comparator, but the NHB value is meaningless if it is taken out of the comparator context. Although the framework is not a ranking system that can compare any two treatment regimens to one another, great potential exists for it to be treated as such and for comparisons to be carelessly or incorrectly made. Following the draft framework release, media reported the NHB scores as if they were stand-alone scores, creating the misleading impression that comparing the scores was possible.2 Basing treatment decisions on incomplete and indirect comparisons may actually lead to more confusion in the shared-decision making process.

While the exchange of patient-physician information is a critical prerequisite for shared decision-making.3 it will be necessary to include detailed educational wrap-around materials for both patients and physicians to make explicitly clear how the framework should (and should not) be used. Both patients and physicians may have difficulty interpreting the incremental “net health benefit” score, and providing guidance on interpretation and use will be paramount.

Minimize the Risk of Framework Misuse

Not only is there great potential for the physician and patient to erroneously compare across treatments, there is potential for others to do so as well. Although ASCO clearly states the goal of the framework is to provide a standardized approach for physicians and patients to assess the value of cancer treatments, payers could misappropriate the information, misinterpret it, and use it to support policies that limit access to life-extending cancer treatment regimens. Such access limitations could be potentially harmful to patients’ health, contrary to the intended purpose of the framework. We encourage ASCO to incorporate safeguards against misuse such as creating a guidance statement that is explicit about how the framework scores should (and should not) be interpreted, how to incorporate these scores into shared decision-making, and that they are not appropriate for or intended for any purpose other than informing shared decision-making between the physician and the patient.

II.      Broaden and Sharpen Framework Components

The components of the framework – benefit, toxicity, and cost – are important factors to patients. We believe the framework can serve patients even further by (1) broadening these factors to include others of importance to patients, and (2) sharpening the methods used to assess these factors.

Include More Factors That Patients Value

Patients value a wide variety of factors and outcomes, and expanding the framework to include more of these items can enhance the overall value of the framework. We recommend considering the following items for inclusion:

• Patient reported outcomes such as quality of life.
• Potential for long-term survival.
  • A focus on median survival masks the potential for long-term survival. A drug may offer only a modest increase in median survival (e.g., 1-2 months) but a considerable increase in the number of patients alive at 5 years (e.g., 12% vs. 4% for the comparator). Some patients will strongly prefer the latter treatment regimen with greater potential for long-term survival, yet median survival will blur this distinction.
• Patient preferences for factors that may enhance adherence.
  • Route of administration: patients may prefer an oral medication over an infused medication.
  • Complexity of regimen: A regimen that consists of 4 pills once a day with or without food is much simpler to follow than one that includes 1 pill twice a day without food taken concomitantly with a second pill 3 times a day with food.
  • Amelioration of side effects: patients may be more adherent to a regimen with fewer side effects.

Strengthen Assessment of Clinical Benefit

Clinical benefit is currently measured using overall survival (OS), or progression-free survival (PFS) if OS is not available, or response rate (RR), if neither OS nor PFS is available. The maximum clinical benefit score is 80 points. There are several limitations to this approach, which are presented below. Strengthening the clinical benefit assessment methodology to overcome these limitations will strengthen the usefulness of the framework for patients.

• Relying on median survival ignores those patients who experience long-term survival (e.g., those patients in the tail of the distribution).
• The structure of the framework is skewed toward lower scores (e.g., NHB<50); a broader distribution across the entire scale would be expected.
• The multiplier values—16 for OS, 11 for PFS, and 8 for RR—are arbitrary. These multipliers drive much of the NHB score, so their impact is considerable, but perhaps not meaningful. The relatively higher weight for OS puts the framework at risk of being misaligned with FDA when they use accelerated approval to bring a product to market without OS data; the other outcomes should not be de-valued in this situation. A broader exercise to derive appropriate multipliers is strongly recommended.
  • Trials that are stopped early due to overwhelming benefit to the patient may not reach median OS and the assessment of benefit would be underestimated.
  • When there is treatment cross-over or open label after progression in a trial, OS is not an appropriate endpoint and can lead to an underestimate of benefit.
  • There are entire disease areas that do not have OS endpoints in their clinical studies (e.g., hematology), limiting the possible score for these treatments.
  • This approach also could disadvantage newer treatments due to the lack of OS data at launch, discouraging adoption of new treatments that could be demonstrated as superior with more mature data.
• The hierarchy of OS > PFS > RR is limiting and may not accurately represent NHB.
  • If both OS and PFS are available as primary outcomes, the NHB score may vary depending on which one is used, but the score would only be calculated using OS. Sensitivity analyses to explore a range of outcomes and possible NHB scores may be appropriate. Including the various outcomes would allow flexibility for physicians and patients to vary the multiplier values in consideration of what is most important to the patient.
• Well-established biomarkers should be incorporated into the benefit assessment when scientifically and medically appropriate, such as those in the ASCO perspectives on defined clinically meaningful outcomes.4
• Measurement of clinical benefit needs to be taken in the context of the treated population. In many cases, trials are conducted with the patients who are hardest to treat, which results in smaller benefit gains than would otherwise be seen in a broader patient population.

Strengthen Assessment of Toxicity

Toxicity is currently scored based on the percentage increase in the number of Grade 3-5 toxicities, and is worth a maximum of 20 points. Some patients might choose to give toxicity a higher weight; the ability to incorporate weights to reflect patient preferences is advisable. Greater granularity in the toxicity scale is advisable as well. Incorporating weights and greater granularity can increase the value of the framework for patients.

• In the framework, a 49% increase in Grade 3-5 toxicities for a new product over an existing product is considered the equivalent of a 49% decrease in Grade 3-5 toxicities for the new product versus an existing product (both receive a score of “0”). These ranges are very wide and a patient may not view these scenarios as equivalent.
• A Grade 5 toxicity (death related to adverse event) is quite different from a Grade 3 toxicity (a severe but not life-threatening event limiting a patient’s ability to care for himself, requiring hospitalization). A more granular scoring system could pick up these differences.
• There are many different ways to calculate adverse events, and including a range of options (e.g., absolute number, length, frequency) can help take the full toxicity picture into account.
• Including a mechanism for patients to weight toxicities based on severity or seriousness can help capture what matters most to the patient.

Clarify Palliation

Treatments for advanced disease are eligible for a palliation bonus of 10 points. It is not clear what qualifies for the palliation bonus, and it is questionable whether specific cancer-related symptoms should be weighted or scored equally. Additional clarity would be helpful to make this measure most meaningful for patients.

Consider Unintended Consequences of Treatment-Free Interval

A statistically significant improvement in treatment-free interval is eligible for a bonus of up to 20 points. Assigning bonus points for treatment-free intervals can inadvertently give incentives to toxic therapies where patients are forced to slow down or stop therapy altogether in order to manage their side effects/toxicities, and we encourage you to consider whether an alternative approach could mitigate this incentive.

III.    Ensure Cost Information is Relevant to Patients

Cost is an important consideration for patients. It is critical that the framework incorporate cost in a way that is clear, complete, and meaningful to the patient.

Reconsider Inclusion of Drug Acquisition Cost

Drug acquisition cost (DAC) is a poor proxy for cost and is not specifically relevant to the patient. There are many shortcoming associated with relying on DAC:

• The patient cannot consider the cost of additional treatment-related medical services (e.g., diagnostics, laboratory procedures, physician visits, facility charges).
• The patient cannot consider the variance of costs depending on site of care (e.g., treatment costs in hospitals or hospital clinics are significantly more expensive than in oncology clinics in the community setting).
• The patient cannot consider direct medical cost offsets (e.g., lower toxicity can result in less emergency room use or hospitalizations); the ASCO Patient-Centered Oncology Payment identifies decreases in hospitalizations and emergency room visits due to complications as a primary driver of value, so capturing this value would be optimal.5

Additionally, DAC does not represent actual drugs costs as it does not include markups by hospitals and providers or discounts from manufacturers. This measure is not relevant to individual patients and we recommend removing it from the framework.

Ensure Patients Receive Accurate and Relevant Cost Information

It is important that patients have access to accurate cost-sharing data for various treatment options. Safeguards should be put into place to ensure patients are receiving accurate and individualized cost information for their specific situation. Clear guidance on the following questions can help build those safeguards:

• Who is responsible for collecting the data for an individual patient?
• How will accuracy be confirmed?
• Many oncology patients have met their maximum out-of-pocket thresholds and may actually have no cost-sharing responsibility—how will this be assessed and included?
• Some oncology patients qualify for patient assistance programs—how will this be assessed and included?
• How can patients receive information about all the costs they will incur related to their care (not simply drug costs)?

Patients may be further confused by the presentation of an “incremental” NHB measure alongside an “absolute” cost. Consistency in presentation (e.g., both incremental or both absolute) can help mitigate confusion.

 IV.   Broaden and Strengthen the Evidence Base

The evidence that goes into the framework will drive the results and shape its usefulness. The framework could be made even more useful by expanding the current evidence.

Extend Beyond Randomized Clinical Trials

The framework relies on randomized clinical trials (RCTs). In some instances, real-world evidence from observational studies may be the best available evidence and it should be considered for inclusion. Observational studies also can allow for comparisons across multiple treatments.

A drawback to RCT study populations is that they may not be generalizable or relevant to an individual patient. Using an incompatible population can alter the framework score, which can adversely impact the discussion between patient and physician. For example, when ASCO recalculated the NHB for cisplatin/pemetrexed relative to cisplatin/gemcitabine using the subset of patients that represent the labeled population, the results were much more favorable than in the original comparison, which relied on the original study population rather than the relevant subset. The original comparison suggested no relative value for cisplatin/pemetrexed, whereas the comparison with the labeled population did indeed show the relative value.[6] As this was the only example in the paper that was provided outside the labeled indication, we encourage ASCO to correct this in the final version of the framework.

Incorporate Patient Differences

Patients have differences in treatment response and accommodating these differences would make the framework more meaningful to the individual patient. One example of this concept is highlighted in the cisplatin/pemetrexed example above, which underscores the importance of evaluating a population that is relevant to the patient. The framework assumes all patients will respond to a treatment in the same way. Patients are different, and their responses will be different. We recommend considering ways to build this flexibility into the model.

Clarify the Selection Criteria

It is unclear how individual trials will be selected for inclusion in the yet-to-be-developed framework tool. As the cisplatin/pemetrexed example above shows, the score from the tool will be heavily impacted by the data that underlie it. Adjusting for the quality of a trial, its statistical significance, the strength of other supporting trials, and the relevance of the comparator (e.g., an outdated comparator that is no longer standard of care can result in a misleading NHB score) may be needed.

Clarifying the selection criteria that will be used, whether multiple trial results can be incorporated into a sensitivity analysis to produce a range of scores or confidence intervals, and what the mechanisms are for identifying and correcting possible errors will strengthen the evidence base and lead to a stronger framework with greater value to the patient.

Clarify the Update Process

We recommend clarifying what the process will be for submitting new data for consideration, and how often the tool will be updated. Regular updates will be needed to keep information current. Without regular updates, the framework risks underestimating the value of the newest innovations. Evidence continues to evolve after launch and incorporating this evolving evidence into the framework in a timely manner will best meet the needs of patients.

V.     Conclusion

NPC supports ASCO’s goal of helping individual patients understand the evidence, innovation and value of medicines so they can engage in meaningful shared decision-making with their physicians. Meeting that goal will require a flexible framework based on sound evidence that reflects what matters most to patients. ASCO has put in considerable work towards this goal, and NPC would be pleased to work with you directly as you continue to refine the framework. Thank you for the opportunity to provide these comments.

Respectfully submitted,

Robert W. Dubois, MD, PhD Chief Science Officer National Pharmaceutical Council

----------------------------------------------------------

[1]  Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology Statement: A Conceptual Framework to Assess the Value of Cancer Treatment Options. J Clin Oncol 2015 Aug 10;33(23)2563-77.

[2] Langreth R and Koons C. “Drug Costing $9,200 Scores Zero on Cancer Doctor Value Scale.” Bloomberg Business 22 Jun 2015: http://www.bloomberg.com/news/articles/2015-06-22/cancer-doctors-rate-drugs-as-prices-surge-past-10-000-a-month. Accessed 7/21/2015.

[3]  Charles C, Gafni A, Whelan T.: Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med. 1997 Mar;44(5):681-92.

[4]  Ellis Lm, Bernstein DS, Voest EE, et al: American Society of Clinical Oncology Perspective: Raising the Bar for Clinical Trials by Defining Clinically Meaningful Outcomes. J Clin Oncol 2014 Apr 20;32(12):1277-1280.

[5] American Society of Clinical Oncology. “Patient-Centered Oncology Payment” May 2015: http://www.asco.org/sites/www.asco.org/files/asco_patient-centered_oncology_payment_final_2.pdf. Accessed August 7, 2015.

[6]  Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology Statement: A Conceptual Framework to Assess the Value of Cancer Treatment Options. J Clin Oncol 2015 Aug 10;33(23)2563-77.