Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis
BMJ 2017; 359 doi: https://doi.org/10.1136/bmj.j5058 (Published 28 November 2017) Cite this as: BMJ 2017;359:j5058
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Authors' response to Owens and Abrams response to Lopez-Lopez et al. 2017 [1]
We thank Owens and Abrams (rapid response dated 29th May 2019) for bringing the important issue of falsified data in the ARISTOTLE trial (Seife 2015 [2]) to our attention. Details of the potentially falsified data are available in the FDA report (pp. 53-59 of the Clinical Review document, which correspond to pp. 180-186 of the .pdf):
www.accessdata.fda.gov/drugsatfda_docs/nda/2012/202155Orig1s000MedR.pdf
The ARISTOTLE trial provides the majority of the evidence (based on 18,104 participants) on the comparative effectiveness of apixaban 5mg twice daily. Based on a systematic review, network meta-analysis and linked cost-effectiveness analysis, we found that apixaban 5mg twice daily ranked highest for most outcomes, and to be cost-effective against warfarin (López-López et al. 2017 [1]). We agree with Owens and Abrams that it is important to assess the sensitivity of our conclusions to exclusion of the falsified data.
In brief, there was evidence that pointed to falsified data at one of the Chinese study sites (site 1200, which recruited 35 ARISTOTLE participants). For some outcomes, the FDA report provides results for: (1) all ARISTOTLE study sites (N=18,210); (2) excluding site 1200 (N=18,166); (3) excluding all 24 sites identified by the FDA’s Office of Scientific Investigations (OSI) where a key researcher was involved (N=17,567); and (4) excluding all Chinese study sites (N=17,358).
For the Stroke or Systemic Embolism (SE) outcome the hazard ratio (95% CI) for apixaban 5mg twice daily compared with warfarin for the different data-sets was: (1) All sites: 0.79 (0.66, 0.95); (2) Excluding site 1200: 0.80 (0.67, 0.96); (3) Excluding 24 sites: 0.78 (0.65, 0.96); and (4) Excluding all Chinese sites: 0.76 (0.63, 0.92).
The corresponding figures for the All-cause mortality outcome were: 1) All sites: 0.89 (0.80,1.00); (2) Excluding site 1200: 0.90 (0.80, 1.00); (3) Excluding 24 sites: 0.89 (0.79, 0.99); and (4) Excluding all Chinese sites: 0.88 (0.79, 0.98).
These results can be seen to be robust to the exclusion of site 1200, and the 24 sites identified by OSI. Excluding all Chinese sites gives very similar results for all cause mortality, and a very slightly more favourable result for Apixaban for the Stroke/SE outcome, but conclusions are still robust.
The FDA report does not provide results for the outcomes myocardial infarction (MI), major bleeding or clinically relevant bleeding, but states “When the analyses for the primary safety endpoint (ISTH major bleeding) and GUSTO severe bleeding are compared in an analogous fashion, sensitivity analyses had no material effect on the overall results (data not shown).”
We constructed sensitivity analyses for these outcomes by making the extreme assumption that all events reported in site 1200 occurred in the warfarin arm, and none in the apixaban arm, but that the real event probability for apixaban is no different to that in the remaining sites. This allows us to obtain an adjustment factor for the overall event probability in the apixaban arm.
Let N1200 be the number of patients on the apixaban arm in site 1200, and Nrest be the number on the apixaban arm in the remaining sites. If we assume that no events were reported on the apixaban arm in site 1200, then the event probability in the sites excluding site 1200 is Prest = R/Nrest, where R is the reported number of results from the study. Assuming that this event probability would have been observed in site 1200, we would have expected R1200 = R*N1200/Nrest events in site 1200. The overall event probability adjusting for the misreporting of outcomes in site 1200 under these assumptions is
Padj = ((R*N1200/Nrest)+R)/(N1200+Nrest)
Rearranging gives:
Padj = ((N1200+Nrest)/Nrest) * (R/(N1200+Nrest)) = Adj * Punadj
where Adj = ((N1200+Nrest)/Nrest) is the adjustment factor and Punadj = (R/(N1200+Nrest)) is the unadjusted event probability.
The adjustment factor Adj can be applied to obtain adjusted odds ratios for the MI, major bleeding and clinically relevant bleeding outcomes, assuming the most extreme distribution of outcomes in site 1200. The same approach can be used to make an extreme adjustment for the 24 sites identified by the OSI, and also for all Chinese sites.
For the Myocardial infarction outcome the odds ratio (95% CI) for apixaban 5mg twice daily compared with warfarin for the adjustments was: (1) Unadjusted: 0.877 (0.660, 1.167); (2) Adjusted for site 1200: 0.879 (0.661, 1.169); (3) Adjusted for 24 sites: 0.909 (0.686, 1.206); and (4) Adjusted for all Chinese sites: 0.920 (0.695, 1.220).
The corresponding figures for the Major bleeding outcome were: (1) Unadjusted: 0.694 (0.600, 0.802); (2) Adjusted for site 1200: 0.695 (0.602, 0.804); (3) Adjusted for 24 sites: 0.720 (0.624, 0.831); and (4) Adjusted for all Chinese sites: 0.729 (0.632, 0.841).
The corresponding figures for the Clinically relevant bleeding outcome were: (1) Unadjusted: 0.674 (0.605, 0.751); (2) Adjusted for site 1200: 0.676 (0.607, 0.752); (3) Adjusted for 24 sites: 0.700 (0.629, 0.779); and (4) Adjusted for all Chinese sites: 0.709 (0.638, 0.789).
Because these estimates are based on an extreme scenario, we expect that the truth lies somewhere between the adjusted and unadjusted estimates. The estimates adjusted for site 1200 are very similar to the unadjusted estimates. The estimates adjusted for 24 sites and all Chinese sites are attenuated slightly compared with the unadjusted estimates, however the effects are small. We therefore expect the impact of any fraud in the ARISTOTLE trial on our network meta-analysis results to be negligible.
In summary, we are confident that the conclusions from our network meta-analysis are robust to the potentially falsified data in the ARISTOTLE trial. There does not appear to be any case, based on the extent of problems reported in the FDA report, for omitting ARISTOTLE entirely from our analyses. This would omit valid information from 17,358 patients and would mean that precise comparison of apixaban with other directly acting oral anticoagulants (DOACs) or with warfarin were not possible. The analysis reported by López-López et al. 2017 [1] is robust to the reported falsified data in ARISTOTLE, and can be used by policy makers, commissioners and treating physicians.
References
1. López-López JA, Sterne JAC, Thom HHZ, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ. 2017 Nov 28;359:j5058. doi: 10.1136/bmj.j5058.
2. Seife C. Research misconduct identified by the US Food and Drug Administration: out of sight, out of mind, out of the peer-reviewed literature. JAMA Intern Med. 2015 Apr;175(4):567-77.
Competing interests: No competing interests
Dear Editors,
ARISTOTLE was a large, randomised controlled Phase 3 trial comparing the direct-acting oral anticoagulant, apixaban, with warfarin in >18,000 patients with atrial fibrillation (Lopes et al 2010). The results showed that apixaban was superior to warfarin in preventing stroke or systemic embolism, caused less bleeding and resulted in lower mortality (Granger et al 2011).
Since publication of the primary manuscript (Granger et al 2011) over 30 secondary manuscripts have been published on the ARISTOTLE study (PubMed search, 18 Mar 2019). Data from the full data set of ARISTOTLE have also been included in 22 meta-analyses (Garmendia et al 2019).
However, the FDA has identified that ARISTOTLE included some falsified data (Seife 2015) and this is an important consideration for subsequent research that relies on these data.
A subsequent analysis conducted recently by Garmendia et al (2019) concluded that among the 22 meta-analyses that included ARISTOTLE data, exclusion of these data would change the results in almost half of cases (10/22; 46%) and the conclusions would change for 21 of the 99 analyses conducted (32%). Of the 32 affected estimates 31 (97%) no longer favoured apixaban for the prevention of serious medical issues, and 1 (3%) favoured the control.
One of these meta-analyses, published in 2017 (Lopez-Lopez et al 2017), was carried out to evaluate the comparative clinical effectiveness and cost-effectiveness of different oral anticoagulants using a network meta-analysis (NMA) in a UK setting. The analysis included 23 studies of >93,000 patients; amongst these, ARISTOTLE was the largest, and thus in a fixed effect network meta-analyses, ARISTOTLE will receive the most weight. Based on the results, the authors concluded that ‘Despite a similar mechanism of action, apixaban at the right dose appears to maximise efficacy and safety among the DOACs, with favourable cost effectiveness.’ Additionally, the authors concluded that apixaban 5mg twice daily was ranked highest for most outcomes. In light of the falsified data reported in ARTISTOTLE, the results of this analysis could have the potential to mislead policy makers, commissioners and treating physicians.
The network geometry in the Lopez-Lopez NMA provides some insights into the central importance of the ARISOTLE study for estimates of the relative clinical effectiveness for apixaban. For example, with regard to the primary outcome of Stroke and Systemic embolism, apixaban 5mg is connected to the network via a direct treatment comparison with warfarin INR 2-3. If ARISTOTLE were removed from the network, indirect treatment comparisons between apixaban 5mg and all other interventions would be informed by a direct treatment comparison between apixaban 5mg and warfarin INR 2-3 which, in the absence of ARISTOTLE, is obtained from a small Phase 2 three-arm trial (n=222, Ogawa et al 2011) in a Japanese patient population with few events (#events=3); and thus, uncertainty in the point estimate for apixaban relative to all other interventions is likely to increase, potentially impacting on the ranking of treatment outcomes. If the falsified data in ARISTOTLE systematically leads to an under or over reporting of the observed effects in ARISTOTLE, then the relative effects of apixaban could also change.
We encourage Lopez-Lopez et al. to acknowledge the data falsification reports associated with the ARISTOTLE trial and suggest caution with respect to interpreting NMA outcomes that rely heavily on the results of this study. In the absence of appropriate caution, policy and treatment decisions may be made that are not supported by the evidence.
References
1. Garmendia CA, Nassar Gorra L, Rodriguez AL, et al. Evaluation of the Inclusion of Studies Identified by the FDA as Having Falsified Data in the Results of Meta-analyses: The Example of the Apixaban Trials. JAMA Intern Med. 2019 Mar 4. doi: 10.1001/jamainternmed.2018.7661. [Epub ahead of print]
2. Granger CB, Alexander JH, McMurray JJ, et al. Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med. 2011 Sep 15;365(11):981-92.
3. Lopes RD, Alexander JH, Al-Khatib SM, et al. Apixaban for reduction in stroke and other ThromboemboLic events in atrial fibrillation (ARISTOTLE) trial: design and rationale. Am Heart J. 2010 Mar;159(3):331-9.
4. López-López JA, Sterne JAC, Thom HHZ, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ. 2017 Nov 28;359:j5058. doi: 10.1136/bmj.j5058.
5. Ogawa S, Shinohara Y, and Kanmuri K. Safety and Efficacy of the Oral Direct Factor Xa Inhibitor Apixaban in Japanese Patients With Non-Valvular Atrial Fibrillation – The ARISTOTLE-J Study. Circ J 2011; 75: 1852–1859.
6. Seife C. Research misconduct identified by the US Food and Drug Administration: out of sight, out of mind, out of the peer-reviewed literature. JAMA Intern Med. 2015 Apr;175(4):567-77.
Competing interests: Rhiannon Owen and Keith Abrams have received consulting fees from Daiichi Sankyo UK Ltd in relation to this work. Keith Abrams is partially supported as a UK National Institute for Health Research (NIHR) Senior Investigator Emeritus (NI-SI-0512-10159). Rhiannon Owen has acted as a paid consultant providing methodological advice to Astellas. Keith Abrams has served as a paid consultant, providing methodological advice, to; Abbvie, Amaris, Allergan, Astellas, AstraZeneca, Boehringer Ingelheim, Bristol-Meyers Squibb, Creativ-Ceutical, GSK, ICON/Oxford Outcomes, Ipsen, Janssen, Eli Lilly, Merck, NICE, Novartis, NovoNordisk, Pfizer, PRMA, Roche and Takeda, and has received research funding from Association of the British Pharmaceutical Industry (ABPI), European Federation of Pharmaceutical Industries & Associations (EFPIA), Pfizer and Sanofi. He is a Partner and Director of Visible Analytics Limited, a healthcare consultancy company.
We thank the authors of both rapid responses for their interest in our paper.(1)
We agree that additional information to that reported in published studies may be available from information in the clinical study reports submitted to regulatory agencies. The US FDA medical review of apixaban,(2) which is cited by O’Sullivan and Tejani, points out that vital status could not be confirmed for around 3% of participants from each arm of the ARISTOTLE trial,(3) which might have affected the hazard ratio for all-cause mortality. We are not aware of evidence that the extent of missing data on vital status differed between trial arms, and it seems unlikely the small amount of missing data could have substantially biased the estimated hazard ratio reported in our network meta-analysis (0.88, 95% CI 0.79 to 0.98). Hence, it is unlikely that access to the vital status of all participants in the ARISTOTLE trial would have a notable impact on the results and conclusions of our review.
The discussion section of our paper addresses the potential need for monitoring of patients treated with dabigatran, which limits the advantages of this drug over warfarin, as well as the faulty device used to monitor INR in ROCKET AF (the largest rivaroxaban trial).(4, 5) The influence of this faulty device was minimal.(6) Whilst we did not discuss the concerns expressed by Cohen(7) about unreported fatal bleeds in the RE-LY trial of dabigatran versus warfarin,(8) we note (as did Cohen) that a correction published in NEJM found that the results for major bleeding were not materially changed.(9)
We agree with the conclusions of the FDA review that superiority of DOACs over warfarin is conditional on how well warfarin is used.(2) It is possible that control of INRs in patients treated with warfarin in trial settings was better than for patients in routine care, in which case our results may underestimate the benefits of DOACs in practice.
The aim of our paper was to provide an objective ranking, balancing benefits and harms, of the different drug options for stroke prevention in atrial fibrillation. We are keen to address the possibility that our results could change based on additional data from the published trials that is contained in clinical study reports. We have contacted the authors of the Cochrane review cited by Boesen,(10) and will rerun our analyses if they find that results from one or more studies are substantially different from those reported in the published article and included in our review.
1. López-López J, Sterne JAC, Thom H, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ. 2017;359:j5058.
2. Food and Drug Administration. Apixaban Medical Review. NDA 202155 2012 [Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2012/202155Orig1s000M....
3. Granger CB, Alexander JH, McMurray JJ, et al. Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med. 2011;365(11):981-92.
4. Cohen D. Manufacturer failed to disclose faulty device in rivaroxaban trial. BMJ. 2016;354:i5131.
5. Patel MR, Mahaffey KW, Garg J, et al. Rivaroxaban versus warfarin in nonvalvular atrial fibrillation. N Engl J Med. 2011;365(10):883-91.
6. Food and Drug Administration. FDA analyses conclude that Xarelto clinical trial results were not affected by faulty monitoring device 2016 [updated 10/11/2016. Available from: http://www.fda.gov/Drugs/DrugSafety/ucm524678.htm.
7. Cohen D. Concerns over data in key dabigatran trial. BMJ. 2014;349:g4747.
8. Connolly SJ, Ezekowitz MD, Yusuf S, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med. 2009;361(12):1139-51.
9. Connolly SJ, Ezekowitz MD, Yusuf S, Reilly PA, Wallentin L. Randomized Evaluation of Long-Term Anticoagulation Therapy Investigators. Newly identified events in the RE-LY trial. N Engl J Med. 2010;363:1875-6.
10. Mahtani KR, Heneghan C. Novel oral anticoagulants for atrial fibrillation. BMJ. 2016;354:i5187.
Competing interests: No competing interests
In a recent network meta-analysis, López-López and colleagues compared direct acting oral anticoagulants (DOACs) with warfarin for atrial fibrillation and concluded that several DOACs are of net benefit compared with warfarin (1). Their results were praised in a linked editorial (2).
Since 2014, the BMJ has led an ongoing investigation of two of these drugs, dabigatran (3, 4) and rivaroxaban (5, 6). Investigations editor of the BMJ, Deborah Cohen, has revealed that; 1) There are concerns and doubts about the actual number of bleedings (3) in the pivotal, randomised, single-blind, warfarin-controlled trial of dabigatran, RE-LY (7), that led to regulatory approval; 2) Patients might benefit from plasma level monitoring of dabigatran contrary to the drug’s branding and claimed advantage over warfarin (4); and 3) The INR device used in the rivaroxaban trial was faulty and later retracted from the market due to false, low readings, which could lead to warfarin overdosing (5, 6). Cohen’s revelations led the directors of the Oxford Centre for Evidence-Based Medicine, Mahtani and Heneghan, to conclude that new, industry-independent assessments of the dabigatran and rivaroxaban trial data are necessary to make reliable conclusions of the benefits and harms (8).
The authors briefly mentioned the concerns about plasma level monitoring and stated that the FDA did a reanalysis concluding that the effects of the faulty device were minimal (1). However, Cohen’s most important finding, the doubt about the number of bleedings in the RE-LY trial (3), was not addressed or accounted for:
The authors referred to RE-LY’s published data only (7, 9) (appendix 3). A “rapid response” by O’Sullivan and Tejani points out that this was the case also for another of the included drugs, apixaban (1). Therefore, the reported numbers of bleedings in the analysis for the dabigatran trial (appendix 4) was the same as that in New England Journal of Medicine (7, 9). The authors did not cite Cohen’s article (3) about the missing bleedings despite her concerns about the published numbers in these particular publications (7, 9), and the authors did not refer to, or use, the most updated (yet still uncertain) number of bleedings from an ensuing correction from 2014 (10).
Another concern about the RE-LY trial, the warfarin group’s high bleeding rate (7, 11), was not mentioned in the analysis. The authors listed each included trial’s total bleeding events but they did not provide annual outcome rates; thus the reader cannot compare possible differences in bleeding rates between the individual trials. The authors also conducted a meta-regression analysis that found no effect modification from “mean time in therapeutic warfarin range” compared to the different DOACs (1). However, an FDA review of the RE-LY trial drew another conclusion in 2010 (12). The FDA review stated that “Dabigatran’s advantage on bleeding, relative to warfarin, was in subjects at centers where mean TTR [time in therapeutic range] was worse than the median” and “Virtually all of the reduction in death was attributable to centers where INR control was worse than the median” (12). This suggests that López-López’ meta-regression analysis is misleading, at least in the case of dabigatran. The FDA review concluded: “Patients whose INRs were well-controlled with warfarin had the equivalent risk of having a stroke or fatal event as those treated with dabigatran 150 mg. Thus, the superiority is really conditional, and depends on how well warfarin is used” (12).
Considering the concerns of dabigatran and the RE-LY trial (the number of bleedings (3); the possible advantages of plasma dabigatran monitoring (4); and the impact of suboptimal warfarin treatment (12)), the network meta-analysis’ conclusion in favour of DOACs seems premature. While we await the upcoming Cochrane review, which should be based on the clinical study reports in order to reduce the amount of reporting bias (8), it remains uncertain whether the benefits outweigh the harms compared to warfarin.
References
1) López-López JA, Sterne JAC, Thom HHZ, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ 2017;359:j5058.
2) Ball J. Which anticoagulant for stroke prevention in atrial fibrillation? BMJ 2017;359:j539.
3) Cohen D. Concerns over data in key dabigatran trial. BMJ 2014;349:g4747.
4) Cohen D. Dabigatran: how the drug company withheld important analyses. BMJ 2014;349:g4670.
5) Cohen D. Rivaroxaban: can we trust the evidence? BMJ 2016;352:i575.
6) Cohen D. Manufacturer failed to disclose faulty device in rivaroxaban trial. BMJ 2016;354:i5131.
7) Connolly SJ, Ezekowitz MD, Yusuf S, et al. Dabigatran versus Warfarin in Patients with Atrial Fibrillation. N Engl J Med 2009;361:1139-51.
8) Mahtani KR, Heneghan C. Novel oral anticoagulants for atrial fibrillation. BMJ 2016;354:i5187.
9) Connolly SJ, Ezekowitz MD, Yusuf S, et al. Randomized Evaluation of Long-Term Anticoagulation Therapy Investigators. Newly identified events in the RE-LY trial. N Engl J Med 2010;363:1875-6.
10) Connolly SJ, Ezekowitz MD, Yusuf S, et al. Additional Events in the RE-LY Trial. N Engl J Med 2014;371:1464-5.
11) Therapeutics Initiative. Dabigatran for atrial fibrillation. Why we cannot rely on RE-LY. Therapeutics Letter 2011;80. Available from: www.ti.ubc.ca/letter80 (accessed 08 Dec 2017).
12) FDA 2010. Center for Drug Evaluation and Research. Application number 22-512. Summary review. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2010/022512Orig1s000S... (accessed 08 Dec 2017).
Competing interests: No competing interests
In 2013 the BMJ published in Research Methods & Reporting “How to access and process FDA drug approval packages for use in research” by Erick Turner [1]. In reviewing the sources of evidence used in the systematic review “Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis” [2], we cannot find evidence that the authors accessed regulatory documents to inform their risk of bias assessment for the contributing trials.
For example, Table 54 “Included studies and risk-of-bias assessment for all-cause mortality (stroke prevention in AF)” in the detailed version of the report [3] lists 11 references for the second largest trial contributing to the review: ARISTOTLE [4]. These references refer to the truncated New England Journal of Medicine summation of the trial and various subsequent subgroup and secondary analyses. Yet the report is without reference to the 393 page U.S. Food and Drug Administration’s 2012 review of apixaban, informed principally by ARISTOTLE [5].
Published summaries of clinical trials are known to contain substantially less information (especially for harm) than the regulatory documents for the same clinical trials [6]. More importantly, the results of meta-analyses can change when unpublished data from regulatory documents are included in a meta-analysis [7]. Turner’s 2013 article indicated that one of the reasons authors of systematic reviews do not access regulatory documents is that they “did not know it was possible to get them” [1]. Now, a tool assists in accessing regulatory documents from the U.S. Food and Drug Administration [8].
How can we be confident with the conclusion of this systematic review if all available information was not used?
Cait O’Sullivan, PharmD
Aaron M. Tejani, PharmD
Therapeutics Initiative
University of British Columbia, Vancouver, BC, Canada
1. Turner EH. How to access and process FDA drug approval packages for use in research. BMJ 2013;347:f5992
2. Lopez-Lopez JA, Sterne JA, Thom HHZ, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ 2017;359;j5058
3. Sterne JA, Bodalia PN, Bryden PA. Oral anticoagulants for primary prevention, treatment and secondary prevention of venous thromboembolic disease, and for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis and cost effectiveness analysis. Health Technol Assess 2017;21:1-386
4. Granger CB, Alexander JH, McMurray JJ, Lopes RD, Hylek EM, Hanna M, et al. Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med 2011;365:981–92
5. US Food and Drug Administration. Apixaban Medical Review. NDA 202155 [Internet]. 2012. Available from: http://www.accessdata.fda.gov/drugsatfda_docs/nda/2012/202155Orig1s000Me...
6. Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: A review of publication and presentation. PLoS Med 2008; 5(11): e217
7. Hart B, Duke D, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ 2012;344:d7202
8. OpenTrialsFDA: Unlocking the trove of clinical trial data in Drugs@FDA. https://opentrials.net/opentrialsfda/
Competing interests: No competing interests
NICE guideline review reveals limitations in 2017 network meta-analysis of oral anticoagulants for the prevention of stroke in atrial fibrillation, and cost effectiveness analysis
Dear Editor,
Further to the recent publication of the NICE guideline ‘Atrial fibrillation: diagnosis and management (NG196)’,[1] we write to inform you of stakeholder comments received as part of the NICE CG180 guideline update which relate to the BMJ paper ‘Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. 2017 Nov 28;359:j5058’. [2]
It is acknowledged that differences exist between the four available DOACs which has generated interest into whether one DOAC provides favourable clinical and economic outcomes in patients with non-valvular atrial fibrillation (NVAF). In the absence of head-to-head trial data comparing the DOACs, research by the team at Bristol University [published by Lopez-Lopez et al. (2017) and by Sterne et al. (2017) in an NIHR report [3], formed the main evidence base for the clinical and cost-effectiveness of DOACs that underpinned NICE’s draft guideline recommendations (September 2020). Based on Bristol University’s findings, the initial draft guidelines supported preferential DOAC use favouring the two twice-daily DOACs (apixaban and dabigatran) over once-daily options (edoxaban and rivaroxaban) using NHS List prices.
As part of the NG196 development process, stakeholder consultation provided an opportunity for clinical and patient experts to comment on the draft DOAC recommendations. Stakeholder comments received during the guideline development process highlighted limitations and errors within the body of evaluated evidence which resulted in NICE losing confidence in the Lopez-Lopez et al./Sterne et al. clinical and cost-effectiveness findings, and thus a reversal from preferential DOAC use (in the draft guideline) back to parity recommendations in the final guideline (NG196, published 27 April 2021).
The comments received specifically highlighted limitations in the network meta-analysis (NMA) and meta-regression approach, which impact the reliability of the clinical effectiveness findings and parameters for the health economic model. In their research, Lopez-Lopez et al. acknowledged that there were no direct comparisons available between DOACs, thus necessitating an NMA approach, and that there was no clear evidence of effect modification where data were available for meta regression analyses and comparisons.
On this subject, and having reviewed the views of stakeholders, NICE “agreed that the NMA by Lopez Lopez was probably not able to adequately adjust for the differences between treatment comparisons in terms of population characteristics that could affect the apparent relative efficacy of the different DOACs, such as TTR [time in therapeutic range] in the warfarin arms. Initially the committee agreed that the meta-regressions used to adjust for TTR were adequate, but after consideration of the numbers of studies involved it does seem unlikely that the meta-regression would have been able to make realistic adjustments to effect that were sufficient to negate inter-comparison differences. Given that the original decision to recommend apixaban and dabigatran over rivaroxaban and edoxaban was based on the results of these meta-regressions, the committee’s subsequent belief that these meta-regressions are not valid has led to an amendment to not recommend any of the 4 DOACs over any other.” [4]
Additionally, errors were identified within the health economic model itself revealing limitations in the accuracy and transparency of the model and its findings. These findings were validated by NICE which stated that “the health economic model has been revised to account for an error in the coding for the annual cost of stroke and an error in the probabilistic sensitivity analysis sampling. As a consequence of these revisions the credible intervals were wider and the results more uncertain regarding which DOAC(s) are the most clinically and cost effective. The committee therefore were no longer confident to recommend a specific DOAC or DOACs” [4] (using NHS List prices).
Beyond clinical and economic aspects, stakeholders also highlighted the benefits to patients that would result from having all four DOACs available as first-line treatment options. These points specifically related to patient (and clinician) choice to determine the optimal treatment option taking into account all risk factors and preferences; supporting patient adherence (having access to once-daily as well as twice-daily DOACs); and patient switching (only when appropriate and not because of a change in DOAC recommendation). We are encouraged that the committee’s recommendation on selection of anticoagulant for stroke prevention now acknowledges the importance of these patient-related and practical factors of treatment. [5]
In light of the CG180 guideline review and NICE points of critique, we would encourage the authors to reflect on the key points raised through the consultation process including the importance of patients’ individual clinical needs and preferences, and adherence, which were not taken into account as part of their clinical and cost-effectiveness analyses.
We are supportive of the rigorous methodology and process that NICE apply in their development of clinical guidelines which includes comprehensive engagement with stakeholders including Patient Advocacy Groups and clinical experts. Since their introduction, DOACs have transformed the AF landscape and been life changing for patients. The recommendations in NICE NG196 to ensure all four licensed DOACs are parity first-line anticoagulation options will continue to enable healthcare professionals to optimise outcomes for their patients whilst considering patient preferences and safety in shared-care treatment decision making.
References
1. National Institute for Health and Care Excellence. Atrial Fibrillation: Management clinical guideline (NG196) published April 2021 [Internet]. 2021. Available from: https://www.nice.org.uk/guidance/ng196
2. López-López JA, Sterne JAC, Thom HHZ, Higgins JPT, Hingorani AD, Okoli GN, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ [Internet]. 2017 Nov 28 [cited 2019 Sep 25];359:j5058. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29183961
3. Sterne JA, Bodalia PN, Bryden PA, Davies PA, López-López JA, Okoli GN, et al. Oral anticoagulants for primary prevention, treatment and secondary prevention of venous thromboembolic disease, and for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis and cost-effectiveness analysis. Health Technol Assess (Rockv) [Internet]. 2017 Mar [cited 2019 Sep 25];21(9):1–386. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28279251
4. National Institute for Health and Care Excellence. Atrial Fibrillation: Management consultation on draft guideline - Stakeholder comments table. 2020.
5. National Institute for Health and Care Excellence (NICE). NG196: Rationale and impact [Internet]. 2021. Available from: https://www.nice.org.uk/guidance/ng196/chapter/Rationale-and-impact
Competing interests: No competing interests in direct relation to this work. AJC: has received institutional grants from Bayer, BMS/Pfizer, Boehringer Ingelheim, Daiichi Sankyo, and personal advisory fees from Bayer, BMS/Pfizer, Boehringer Ingelheim, Daiichi Sankyo. PBN: has received speaking fees from BMS/Pfizer and Daiichi Sankyo ; consulting fees from Bayer and Daiichi Sankyo; and grant support from BMS/Pfizer and Daiichi Sankyo. AF: has attended advisory boards and received speaker fees for Bayer, BMS/Pfizer, Boehringer Ingelheim, and Daichii Sankyo Arrhythmia Alliance & AF Association has received grants/donations from Bayer, BMS/Pfizer and Daichii Sankyo