Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13
BMJ 2017; 359 doi: https://doi.org/10.1136/bmj.j4530 (Published 04 October 2017) Cite this as: BMJ 2017;359:j4530
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
I wish to respond to the person from the EMA.
Dr. Pignatti states it is “well known” that in many situations showing survival or quality of life benefit is not feasible. Unfortunately, the arguments he gives are not universally agreed upon, they are merely oft repeated.
Dr Pignatti states that survival benefits may be difficult to detect because subsequent lines of therapy “dilute” them. Step back and consider this argument. Even though median survival with metastatic cancer is short, and life limiting, and even though a new drug approved is a novel, costly therapy, that drug cannot improve survival because all of the prior drugs are good enough to result in the same survival. An analogy would be: you were running a marathon, and purchased a 100,000 pound energy drink and drank it at mile 2. Unfortunately, you complete the marathon in the exact same time you otherwise would. A critic points out that the drink had no value. It didn’t change the outcome. You then argue that is only because the effect of the energy drink was diluted over all the other miles. But that is precisely an argument why the energy drink was a waste. Its cost and side effects are not justified unless it improves an outcome that matters.
Dr. Pignatti also argues prolonging progression free survival (PFS) may delay the onset of symptoms. Here, he uses PFS as a surrogate for symptoms, and fails to provide data—in the form of surrogate validation studies—that proves PFS gains are accompanied by reduction in symptoms.
It is important to remember that PFS and response rate are typically based on the arbitrary change in tumor size on a CT scan. Either an increase of 20% (progression) or decrease of 30% (response). Almost no patient says “I feel bad” the moment their tumor grows from 19% to 21%. Nor do they sudden exclaim "I feel better" if the tumor is 31% rather than 29% smaller. Moreover, studies show variability in the measurement of tumors on scans by different readers [1]. For this reason, it is impossible to contend either of these metrics is something inherently valuable to patients. They are radiographic surrogates.
Dr. Pignatti argues that “dramatic activity” in the form of response rates may make further randomized trials non-feasible. This argument is unsound for a few reasons. First, the response rate in early clinical trials is almost always inflated over later trials [2]. The response rate tends to be smaller if you test the drug again. Second, there have been drugs with impressive response rates (>60%) that have failed in every single randomized trial they have latter been tested in, resulting in market withdrawal [3]. Most cancer drugs offer marginal benefits, and can and should be tested in randomized trials. In fact, gains in PFS are always measured in randomized trials, only response rate can be single armed.
Dr. Pignatti states quality of life is hard to measure. I agree, and support better and fairer ways to measure it. Moreover we have to acknowledge that good drugs improve both survival and quality of life, and this is the goal. Simply because quality of life is hard to measure however, does not justify allowing drugs to remain on the market for years with proof they only change imaging results.
Dr. Pignatti says that drugs that offer incremental benefits are still worthwhile. The major finding of the paper by Davis is that far too often we don’t know if drugs offer benefits at all. I personally support the use of provisional approval based on surrogates in certain situations, but that must come with a responsibility to show survival benefit post approval. You cannot have both lax approval and lax post-marketing commitments, which is what Davis and colleagues show.
Finally, Dr. Pignatti engages in speculation. The world is better off by having surrogate approvals. The truth is we do not know if allowing surrogate approvals benefits or harms patients because we do not know the counterfactual. While some may believe a world without surrogate approvals will result in delays for drug approval, the unanticipated consequence may be the industry will no longer pursue targets that merely result in changes to imaging scans, and instead focus on truly promising compounds. We may have fewer drugs, but the ones we have might be better. I do not know know this is true, and neither does anyone else. But just as we can speculate the world will be worse, we can also speculate it might be better. Both are just speculation.
The major point of Davis and colleagues is that while it may be ok to allow cancer drugs on the market based on a surrogate, it seems illogical to allow them to remain on the market 3, 5, or 7 years without proof of survival or quality of life benefit. At some point in the lifecycle of these drugs, this must be shown. The EMA has the obligation to make that happen. These comments raises concern that the EMA views their role differently.
[1] https://academic.oup.com/jnci/article/106/12/dju331/922126/Use-and-Misus...
[2] https://www.ncbi.nlm.nih.gov/pubmed/26259493
[3] https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/1...
Competing interests: I am the editorialist
In their recent article, Davis et al1 show that nearly three quarters of the anticancer drugs approved by EMA in the years 2009-2013 do not attain a meaningful survival benefit or a quality of life improvement in spite of very high costs. The article gives voice to many medical oncologists and hematologists around the world who are alarmed by the escalating cost system without significant demonstration of efficacy and cost effectiveness.
The authors rightly point to the responsibility of regulatory authorities such as EMA in approving expensive drugs with little if any survival benefit. The situation is not different in the United States, where between 2008 and 2012 the US Food and Drug Administration approved most uses of cancer drugs without evidence of survival or improved quality of life (67%, 36/54)2. In her commentary, Cohen3 underlines the potential conflict of interest of EMA who receives 89% of its entire budget from the drug industry fees, thus creating doubts over the independence of the agency. Here we would like to add to these issues our responsibility as cancer clinicians in perpetrating this vicious circle of unsustainable costs in spite of limited efficacy and increasing financial burden to the health care systems.
In medical oncology, financial relationships have increased through the years and have influenced clinical research, scientific visibility and career development.4 The issue is particularly important in our discipline given the increasing volume of investments made by the pharmaceutical industry in cancer treatment.5 In this escalating prize system6, pharmaceutical companies tend to orient a significant part of their investments towards marketing and promotional activities (at least 20-30% of the final drug price)7,8, rather than to research and development, a notion which is rarely publicly available. In a recent analysis on 10 approved cancer drug in the US9, the median cost to develop a drug was $648.0 million, a figure significantly lower than prior estimates, and the revenue since approval was substantial (median, $1658.4 million; range, $204.1 million to $22 275.0 million).
Direct financial relationships between industry and physicians and/or researchers, consisting of stock options, advisory fees, honoraria, speaking fees, travel and lodging expenses, have become common practice in modern medicine but represent an important source of conflict of interest and avoidable expenditures. In addition, they violate the first principle of the Hyppocratic oath to make the patient’s interest first without undue external influences, thus blemishing the concept of medical professionalism.
What can be done to stop this ill system? First of all, we propose a stringent policy of conflict of interest by the governments which should ban direct financial relationships between pharmaceutical industry and individual doctors as well as regulatory authorities, scientific journals, academic societies and patient advocacy groups. This may be a concrete approach to select better treatments and reduce drug prices.
References
1. Davis C, Naci H, Gurpinar E, Poplavska E, Pinto A, Aggarwal A. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13. BMJ. 2017 Oct 4;359:j4530.
2. Kim C, Prasad V. Cancer drugs approved on the basis of a surrogate end point and subsequent overall survival: An analysis of 5 years of US Food and Drug Administration approvals. JAMA Intern Med 2015;359:1992-4. doi:10.1001/jamainternmed.2015. 5868 pmid:26502403.
3. Cohen D. Cancer drugs: high price, uncertain value. BMJ. 2017 Oct 4;359:j4543.
4. Moy B, Bradbury AR, Helft PR et al. Correlation between financial relationships with commercial interests and research prominence at an oncology meeting. J Clin Oncol. 2013 Jul 20;31(21):2678-84.
5. Global Oncology Trend Report a Review of 2015 and Outlook to 2020. IMS Institute for healthcare informations. http://www.imshealth.com/en/thought-leadership/quintilesims-institute/re....
6. Saltz LB. Perspectives on Cost and Value in Cancer Care. JAMA Oncol. 2016;2(1):19-21.
7. Socolar D, Sager A. Pharmaceutical marketing and research spending: The evidence does not support PhRMA’s claims. Presented at the American Public Health Association Annual Meeting (session 2018.0), Atlanta, GA. 2001.
8. Anderson R. Pharmaceutical industry gets high on fat profits. By Business reporter, BBC News 6 November 2014. http://www.bbc.com/news/business-28212223
9. Prasad V, Mailankody S. Research and Development Spending to Bring a Single Cancer Drug to Market and Revenues After Approval. JAMA Intern Med. 2017 Sep 11. doi: 10.1001/jamainternmed.2017.3601. [Epub ahead of print] PubMed PMID: 28892524.
Competing interests: No competing interests
This paper provides further evidence that most new cancer drugs have not been shown to improve survival or quality of life, and that when survival gains are shown they are not always clinically meaningful.(1)
Wise previously drew attention to over permissive new drug approval systems internationally, and the almost routine offer of chemotherapy without adequate guidance on benefits and risks that deprives patients of deserved empowerment: their potential and right to choose only palliative (best supportive) care rather than drugs.(2)
This paper is a further wake-up call to oncologists, general practitioners and patients for an early palliative care approach integrated with disease-modifying care - that is giving personalised support to patients to help them socially and emotionally as well as whatever cancer medication may be required. This approach should be started at diagnosis of a life-threatening illness, when people start to worry, rather than in the final weeks when it is too late to prevent and treat most problems. This may spearhead the realisation of realistic medicine,(3) and it represents a major contribution to health promotion towards the end of life. Our recent analysis and embedded video gives a rationale for this approach to be triggered earlier than later.(4) https://www.youtube.com/watch?v=vS7ueV0ui5U
If palliative care were a drug, the manufacturers would be busy due to the great demand for such a beneficial treatment with no toxic side-effects. The general public would be calling for it, and patient support groups would be approaching politicians.
However “palliative care” has a bad press as it’s associated wrongly with imminently dying. Promoting early palliative care is to everyone’s advantage: patients, carers and the health service at large. Being able to offer compassionate support that covers all dimensions of need, and communicating and planning openly would prevent many treatments that are likely to cause more harm than good. The expense and toxicity of cancer drugs mean we have an obligation to expose patients to such medication only when they can reasonably expect an improvement in survival or quality of life.(5)
We must offer palliative care or “palliatosin” as an active choice, an alternative therapeutic option to enable people to live well in such difficult situations.
Scott A Murray, Sebastien Moine
1. Davis C, Naci H, Gurpinar E, Poplavska E, Pinto A, Aggarwal A. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13. BMJ. 2017;359.
2. Wise PH. Cancer drugs, survival, and ethics. BMJ. 2016;355:i15792.
3. The Scottish Government. Realising Realistic Medicine: Chief Medical Officer's Annual Report 2015-16. http://www.gov.scot/Resource/0051/00514513.pdf.
4. Murray SA, Kendall M, Mitchell G, Moine S, Amblàs-Novellas J, Boyd K. Palliative care from diagnosis to death. BMJ. 2017;356.
5. Prasad V. Do cancer drugs improve survival or quality of life? BMJ. 2017;359 j4528.
Competing interests: No competing interests
The high price of new cancer drugs has fuelled a debate on their added value and the evidentiary standards for regulatory approval. The findings by Davis et al. (BMJ 2017;359:j4530) that several cancer drugs entered the European market without clear evidence of extending duration of survival or improving quality of life (QoL) are not surprising to anyone familiar with cancer drug development. It is well known that in many situations demonstrating a clear effect on survival or QoL is not feasible and a benefit can be shown on the basis of other endpoints. There are several reasons for this.
First, there are good reasons why an effect on overall survival may be difficult to detect, such as when control-group patients in a randomized clinical trial (RCT) switch to the experimental treatment after progression, or when multiple subsequent lines of effective treatments “dilute” the effect of a drug used in earlier lines. In these and other situations, progression-free survival (PFS, the time during which treatment can induce and maintain a response or at least delay the growth of cancer) has been the efficacy outcome on which many cancer drugs have been approved; the justification being that (if of sufficient duration, with acceptable toxicity and no detriment in overall survival) prolonging PFS will delay onset and worsening of symptoms. Importantly, if a sufficient effect is observed based on this endpoint, it is generally considered to reflect an intrinsic clinical benefit and is not a “surrogate” for survival requiring subsequent confirmation, as Davis et al. seem to imply.(1,2,3)
Second, when “dramatic activity” is observed in terms of objective response (tumour shrinkage) and response duration on the basis of single-arm trials in a well-defined patient population with high unmet medical need, RCTs are considered unethical or unfeasible and “early approval” mechanisms can be used. This applies to a minority of cases and requires that the course of the disease is highly predictable, that there are no good therapeutic alternatives and that there are sufficient supportive clinical and non-clinical data to show a positive benefit-risk balance. Confirmatory data on long-term endpoints are generally requested in these situations, although the evidence may sometimes be extrapolated from related indications when direct confirmation in the same indication is not possible (e.g., when RCTs are not well underway at the time of approval).
Third, QoL is often difficult to assess for a number of reasons, including when double-blind trials cannot be conducted because the distinct toxicity profiles of drugs make them difficult to mask, or there is poor compliance with questionnaire completion. Drawing conclusions on the basis of single items or domains of a QoL instrument is also generally challenging due to the risk of chance findings. Therefore, QoL is rarely used as the primary efficacy endpoint in cancer clinical trials and convincing clinical benefits in terms of QoL are only rarely shown. This, however, does not mean that EMA does not value such studies, which are encouraged even if often they do not lead to robust conclusions. (1,4)
Lastly, there are indeed situations when drugs are approved on the basis of small incremental benefits instead of giant leaps, which are relatively rare. The European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) used by Davies et al. to identify drugs that bring a clinically meaningful benefit has its limitations including failure to recognise small incremental benefits. This scale was never designed for the purpose of clinical decision-making and was mainly constructed on the basis of oncologists’ views rather than a systematic evaluation of patient preferences. Patients play a key role in informing value judgments of benefits and risks of treatments, and EMA has been active over the years in implementing its framework for patient involvement in benefit-risk assessment.
A detailed discussion of the examples mentioned in the article, as well as a response to the claim that “EMA has either failed to identify or overlooked” methodological problems made in the accompanying Feature (doi:10.1136/bmj.j4543), is not possible here. It suffices to say that approval is not a ruling on drug development or whether the ideal trial designs, comparators and methods of analysis have been used, but a scientific conclusion on the balance of the benefits and risks based on available data. Thus, the conduct of trials according to the most rigorous methodology is strongly encouraged whenever possible but in general is not a pre-requisite for approval provided that existing uncertainties can be addressed on the basis of the totality of the data, including future studies, where appropriate.(5) For every assessment, EMA publishes on its website all the details about the data submitted, benefit-risk assessment evaluations, remaining uncertainties and how they were handled, and justifications of any deviations from standard methodology in the context of the available evidence and therapeutic context.
We all strive to help patients live better and longer. However, restricting approvals of cancer drugs only to situations where there is indisputable evidence of improvement in survival or QoL will not improve the lives of cancer patients. On the contrary, such an approach may deprive patients in urgent need of early access to effective medicines. In such situations, early access to new drugs that fulfil an unmet medical need has to take precedence over generating evidence according to the highest standards for health economic evaluation. Science and standards are here to serve patients and not the other way round.
References
1. European Medicines Agency. Guideline on the evaluation of anticancer medicinal products in man. 2012. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guidelin....
2. Pazdur R. Endpoints for assessing drug activity in clinical trials. Oncologist 2008;13 Suppl 2:19-21.
3. European Medicines Agency. Appendix 1 to the guideline on the evaluation of anticancer medicinal products in man - methodological consideration for using progression-free survival (PFS) or disease-free survival (DFS) in confirmatory trials. 2012. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guidelin....
4. European Medicines Agency. Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man: The use of patient-reported outcome (PRO) measures in oncology studies. 2016. http://www.ema.europa.eu/docs/en_GB/document_library/Other/2016/04/WC500....
5. Jonsson B, Bergh J. Hurdles in anticancer drug development from a regulatory perspective. Nat Rev Clin Oncol 2012;9(4):236-43.
Competing interests: No competing interests
“We demand rigidly defined areas of doubt and uncertainty!” This quote was lifted from Douglas Adams’ Hitchhiker’s Guide to the Galaxy and is perceived by most to be absurd (those who thought it was extracted from NICE’s Guide to the methods of technology appraisal, go to the bottom of the class).
But you might be forgiven for expressing exactly that sentiment after reading this article. The researchers found that most oncology drugs entering the market between 2009 and 2013 did so without clear evidence of a marked improvement in the quality or quantity of patients’ lives.
However, in order to approve a treatment for use, the European Medicines Agency must conclude that the benefits of introducing a new treatment outweigh the risks. So, how can we be sure that only cancer treatments that are clinically and cost effective become available in the NHS? How do we balance the demand for early access to innovative treatments with the concern that public funds could be wasted? It’s a tough job but one that NICE relishes.
In 2016, there were some important changes to the way we appraised cancer medicines, and NICE and NHS England starting working together much more closely. The Cancer Drugs Fund was reformed to become a managed access fund for promising treatments with significant clinical uncertainty. This means that before a medicine is made available via the Cancer Drugs Fund, a managed access agreement is drawn up. It specifies the data that will be collected to address the uncertainty. The price the NHS pays during the managed access period reflects the level of uncertainty in the evidence. Afterwards, NICE reviews its guidance in light of the new evidence to decide if the treatment can be made routinely available in the NHS.
Perhaps in the context of managed access, ‘rigidly defined areas of doubt and uncertainty’ isn’t such an absurd concept after all!
Competing interests: No competing interests
Sir,
The legitimate concerns raised by Davis and colleagues (BMJ 2017;359:j4530) about the availability of data on overall survival and quality of life benefits of cancer drugs approved recently in Europe, had been addressed, albeit from a different angle, as early as 2011, for one of the diseases included in Davis’ survey [1].
Davis and colleagues conclude that their systematic evaluation of oncology approvals by the European Medicines Agency, in the years 2009-13, shows that most drugs, entered the market without evidence of benefit on survival or quality of life.
To the reasonable discussion presented by these authors, I would like to add another general observation about scientific articles which I find perplexing.
Why is it that highly educated, highly intelligent, competent investigators, presenting their results in highly respected medical journals, now require a professional writer, invariably payed by the sponsoring drug company, to articulate their findings? Can the authors of such practices really claim these papers as their own? Or should bibliometric organisations now devise systems marking with an asterisk those papers authored by someone but written by another?
Spyros Retsas MD FRCP
Medical Oncologist (Retired)
Reference
Retsas S. Latest developments in the treatment of melanoma: ‘a penicillin moment for cancer’? J R Soc Med 2011: 104: 269–272. DOI 10.1258/jrsm.2011.100405
Competing interests: No competing interests
What is preposterous is how Homeopathic Medicine, available in the NHS since 1951, has a better evidence base than this[1,2]!
[1]http://facultyofhomeopathy.org/wp-content/uploads/2016/03/2-page-evidenc...
[2]https://www.hri-research.org/resources/homeopathy-faqs/conventional-medi...
Competing interests: No competing interests
An interesting study would be to compare any or all of these drugs against a mandatory inclusion of fresh fruit and vegetables in the patients' diets.
Competing interests: No competing interests
Correcting a common misunderstanding regarding ESM0-MCBS grading
In their recent paper in the BMJ (1) Davis et al. have used the ESMO-MCBS to evaluate the level of clinical benefit of anticancer agents licensed between 2009-2013. While we laud them on this project and agree with the general conclusion that many treatments with very limited evidence for benefit are licensed, part of their analysis is based on a flawed understanding of the ESMO-MCBS scale.
The statement in the methods section: “Only scores of A or B (for treatments of curative intent), or 5 or 4 (for treatments used in the non-curative/palliative setting) are defined as clinically meaningful according to the ESMO framework.” is not an accurate representation. Indeed, it perpetuates a common misunderstanding of the ESMO-MCBS grading. While the ESM0-MCBS does distinguish high benefit from low benefit studies, it does not set a threshold for “clinical meaningfulness” (2-4).
While scores of A or B (for treatments of curative intent), or 5 or 4 (for treatments used in the non-curative/palliative setting) indicate “a high level of proven clinical benefit” or “substantial benefit”, this does not preclude that studies achieving slightly lower scores, such as a grade of 3 in the non-curative/palliative setting, may also provide clinically meaningful benefit. Indeed, a study correlating the decisions of the Israeli HTA body to ESMO-MCBS scoring found that in the non-curative setting, most medications with a score of >3 were approved for reimbursement whereas those with a score of <3 were very rarely approved (5). This experience would indicate that in a high income country, scores of >3 were usually judged by an experienced HTA body to be clinically meaningful, and worthy of reimbursement in a resource limited environment (5).
Accurate interpretation of the results generated by the ESMO-MCBS require methodological diligence in data acquisition, careful application of the tool, and a clear understanding of the meaning (and acknowledged limitations) of the grades generated (2-4).
References
1. Davis C, Naci H, Gurpinar E, Poplavska E, Pinto A, Aggarwal A. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13. BMJ 2017;359:j4530 http://www.bmj.com/content/359/bmj.j.
2. Cherny N, Dafni U, Bogaerts J, Latino N, Piccart M, Pentheroudakis G, et al. ESMO-Magnitude of Clinical Benefit Scale Version 1.1. Ann Oncol 2017:mdx310, https://doi.org/10.1093/annonc/mdx310.
3. Cherny NI, Sullivan R, Dafni U, Kerst JM, Sobrero A, Zielinski C, et al. A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:1547-73.
4. Dafni U, Karlis D, Pedeli X, Bogaerts J, Pentheroudakis G, Tabernero J, et al. Detailed statistical assessment of the characteristics of the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS) threshold rules. ESMO Open. 2017;2(4). e000216; DOI: 10.1136/esmoopen-2017-000216
5. Hammerman A, Greenberg-Dotan S, Feldhamer I, Birnbaum Y, Cherny NI. The ESMO-Magnitude of Clinical Benefit Scale for novel oncology drugs: correspondence with three years of reimbursement decisions in Israel. Exp Rev Pharmacoecon Outcomes Res 2017:1-4. doi: 10.1080/14737167.2017.1343146
Competing interests: No competing interests