Effect of adoption of neoadjuvant chemotherapy for advanced ovarian cancer on all cause mortality: quasi-experimental study
BMJ 2018; 360 doi: https://doi.org/10.1136/bmj.j5463 (Published 03 January 2018) Cite this as: BMJ 2018;360:j5463
- Alexander Melamed, fellow1,
- Günther Fink, associated professor2,
- Alexi A Wright, assistant professor3,
- Nancy L Keating, professor4,
- Allison A Gockley, fellow5,
- Marcela G del Carmen, professor1,
- John O Schorge, chief of gynecology1,
- J Alejandro Rauh-Hain, assistant professor6
- 1Division of Gynecologic Oncology, Vincent Department of Obstetrics and Gynecology, Massachusetts General Hospital, Boston, MA 02114, USA
- 2Department of Global Health and Population, Harvard TH Chan School of Public Health, Boston, MA, USA
- 3Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- 4Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
- 5Division of Gynecologic Oncology, Department of Obstetrics and Gynecology, Brigham and Women’s Hospital, Boston, MA, USA
- 6Department of Gynecologic Oncology and Reproductive Medicine, Division of Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
- Correspondence to: A Melamed alexander.melamed{at}mgh.harvard.edu
- Accepted 9 November 2017
Abstract
Objective To estimate the causal effect of increased use of neoadjuvant chemotherapy (NACT) on all cause mortality in advanced epithelial ovarian cancer.
Design Quasi-experimental fuzzy regression discontinuity design and cross sectional analysis.
Setting Cancer programs throughout the United States accredited by the Commission on Cancer.
Participants 6034 women with a diagnosis of stage 3C or 4 epithelial ovarian cancer from regions that rapidly adopted use of NACT from 2011 to 2012 (27% increase in the New England and east south central regions) or remained unchanged (control regions, south Atlantic, west north central, and east north central regions).
Main outcome measure All cause mortality within three years of diagnosis. Kaplan-Meier curves and proportional hazard models were estimated to compare mortality differences between rapidly adopting regions and controls.
Results 1156 women were treated for advanced epithelial ovarian cancer during 2011 and 2012 in the two rapidly adopting regions and 4878 women in the three control regions. In the rapidly adopting regions, patients treated in 2012 compared with 2011 had a mortality hazard ratio of 0.81 (95% confidence interval 0.71 to 0.94) after adjusting for mortality time trends, whereas no difference was observed in control regions (1.02, 0.93 to 1.12). Compared with control regions, larger declines in 90 day surgical mortality (7.0% to 4.0% v 5.0% to 4.3%, P=0.01) and in the proportion of women not receiving surgery and chemotherapy (20.0% to 17.4% v 19.0 to 19.5%, P=0.04) were observed in rapidly adopting regions. Cross sectional analysis confirmed that treatment in regions with greater use of NACT was associated was lower mortality (P=0.001).
Conclusions Adoption of NACT for advanced epithelial ovarian cancer in New England and east south central regions led to a sizable reduction in mortality within three years after diagnosis.
Introduction
Ovarian cancer is the fourth leading cause of cancer related deaths among women in the United States, and is usually diagnosed after it has metastasized within the peritoneal cavity.1 Although two randomized trials found equivalent overall survival and reduced surgical morbidity with neoadjuvant chemotherapy (NACT) followed by surgery compared with primary cytoreductive surgery,23 the use of NACT remains controversial in the United States.4567 Current national guidelines recommend that NACT should be reserved for patients who are not candidates for primary surgery because of unacceptable surgical risk or unresectable disease.8 In the United States only 22% of women with advanced ovarian cancer received NACT followed by interval cytoreductive surgery in 2013.9
The slow uptake of NACT in the United States may be related to the limitations of existing international randomized trials, which had poor overall survival, low rates of optimal cytoreduction, and short operative times, leading many to question whether the surgical techniques were comparable to those used in the United States.56 Indeed, observational studies in the United States and Canada have shown that women selected to undergo primary cytoreductive surgery live longer than those treated with NACT.10111213 These findings raise questions about the extent to which the clinical trial results can be generalized to the broader community of patients with advanced ovarian cancer.
Despite this skepticism, the use of NACT has increased gradually in the United States in recent years.9 Two regions increased the use of NACT by more than 25% from 2011 to 2012, providing an ideal natural experiment to assess the causal effect of the increased utilization of NACT on survival in women with advanced epithelial ovarian cancer using a fuzzy regression discontinuity design.
Methods
Data
We used data from the National Cancer Database, a joint program of the Commission on Cancer of the American College of Surgeons and the American Cancer Society. The National Cancer Database aggregates tumor registry data from more than 1500 hospitals and includes 70% of all incident cancer diagnoses in the United States.14 Certified cancer registrars abstract the information about patient demographics, tumor characteristics, treatment course, and overall survival. The National Cancer Database also includes anonymized hospital identifiers, geographic information, and hospital characteristics.
Sample selection
We identified women who underwent treatment for stage 3C and 4 epithelial ovarian cancer, between 2004 and 2013, in the National Cancer Database participant use file. We excluded women with pre-existing malignancies, without histologic confirmation, or those who did not receive any treatment, obtained treatment at outside facilities, or whose treatment was unknown (see appendix figure 1). All women who received chemotherapy as their first cancer directed treatment were included in the NACT group, even if they never received surgery, and all women who underwent surgery as primary treatment were included in the primary cytoreductive surgery group, irrespective of subsequent receipt of chemotherapy.
Primary outcome
The primary outcome of interest was time from diagnosis to death, or last follow-up, as recorded by the cancer registrar. Vital status was ascertained through to the end of 2015. To avoid bias from differential follow-up, we censored all patients alive three years after diagnosis.
Statistical methods
The primary empirical approach used was a regression discontinuity design.15 As discussed in a growing body of literature, regression discontinuity designs can identify causal effects of interest in observational studies by exploring exogenous shifts in treatment probabilities.161718 The main logic of the regression discontinuity design estimator is based on the insight that when the probability of receiving a treatment changes abruptly at a threshold value of a randomly distributed continuous assignment variable, people on either side of this threshold should be identical with respect to their counterfactual outcomes in the absence of treatment. This exchangeability of people around the threshold essentially mimics a randomized controlled trial, where treatment is randomly assigned. The key factor for a valid regression discontinuity design is a discontinuous increase in treatment probabilities around an underlying observable variable. The adoption of NACT provides an ideal setting for this methodology owing to large shifts in NACT uptake in New England and east south central census divisions shortly after the publication of the first randomized trial of NACT (see appendix table A).2 The application of the regression discontinuity design methodology to survival analysis, as well as utilization of time as the assignment variable, have been well described in the literature.1619
We first used a logistic regression model to verify that, after adjusting for temporal trends, diagnosis after 2012 was associated with a statistically significant increase in the probability of receiving NACT in the rapidly adopting regions. Subsequently, we estimated the causal effect of increased NACT uptake on survival by fitting proportional hazard models of the form:
Ln(hazard)=β0+β1[year]+β2[year2012]+β3[region]where Ln(hazard) is the natural logarithm of the all cause mortality hazard, year is the calendar year of diagnosis, modeled as a continuous variable, Year2012 is a dummy variable coded as 1 for patients with a diagnosis after 2012, and 0 in those with a diagnosis before, region is the census division (New England or east south central), β0 is the baseline hazard function, and β1-3 are regression coefficients. We fit these models over a variety of time ranges, initially restricting to patients with a diagnosis in 2011 and 2012, and progressively increasing the bandwidth to include all available data (2004-12). We estimated the complier average causal effect of NACT using two stage instrumental variables estimation (see appendix).20
Next we identified three control census divisions (south Atlantic, east north central, and west north central) where use of NACT over the same interval (2011-12) increased by less than 2 percentage points (see appendix table A), and replicated the main analysis in these regions. We compared regression discontinuity design estimates between in New England and east south central divisions and control regions using Cox proportional hazards difference-in-differences models (see appendix).
To examine potential mechanisms underlying the effect of NACT on survival, we calculated the frequency of death within 30 and 90 days of surgery, and the proportion of patients who did not receive both surgery and chemotherapy, in 2011 and 2012, and compared trends between New England and east south central and control regions using a logistic difference-in-differences model.
Sensitivity analyses
We conducted several sensitivity analyses to assess the robustness of our findings. To ensure that regression discontinuity results were insensitive to model selection, we performed the analysis using both Cox and exponential survival models, modeled survival trends as linear and quadratic functions, and repeated the main analysis using models adjusting for age, race/ethnicity, stage, histologic type, grade, and comorbidity (see appendix). Additionally, we constructed separate regression discontinuity models in New England and east south central divisions, to confirm that the effect of NACT on survival persisted in both regions (see appendix). Finally, since the South Atlantic and west north central census divisions—two of the three regions that served as controls in our main analysis—increased use of NACT from 2012 to 2013, we conducted a regression discontinuity design in these regions using 2013 as the threshold year, to verify that adoption of NACT resulted in a comparable survival effect to that of our main analysis (see appendix). We compared these results to the east north central census division, the third control region from our main analysis, in which use of NACT remained largely unchanged in 2013 (see appendix).
Cross sectional analysis
To further confirm our findings, we performed a cross sectional analysis of the association between region level prevalence of NACT use and all cause mortality. For each census division, we calculated year specific relative hazard of death from any cause and the year specific relative prevalence of NACT (with national averages serving as the referent values) for 2004 to 2013. We plotted the relative hazard against relative use of NACT, and in Cox proportional hazard models tested whether these were statistically significantly associated (see appendix).
Results
Figure 1 (top panel) illustrates the discontinuity in the frequency of NACT use in the New England and east south central census divisions. From 2011 to 2012, the use of NACT increased by 27.3% in New England (from 36.2% to 46.1%) and by 23.3% in east south central (from 29.9% to 36.9%). Increased use of NACT in adopter regions was entirely the result of a rise in the proportion of patients who received NACT followed by interval debulking surgery (19.5% to 28.7%), rather than by patients who received chemotherapy only (12.6% to 12.0%, see appendix table B). Adjusting for linear time trends in NACT use, patients with a diagnosis in 2012 had 41% greater odds of receiving NACT than those with a diagnosis in previous years (odds ratio 1.41, 95% confidence interval 1.16 to 1.71, P=0.001). In comparison, the bottom panel in figure 1 illustrates that in control regions (South Atlantic, west north central, and east north central census divisions) patients with a diagnosis in 2012 were no more likely to receive NACT than those with a diagnosis in prior years (0.98, 0.86 to 1.12, P=0.78).
Annual frequency of neoadjuvant chemotherapy for advanced epithelial ovarian cancer (blue circles) in New England and east south central census division (A), and south Atlantic, west north central, and east north central census divisions (B). Red lines represent linear trends in use of neoadjuvant chemotherapy estimated from 2007 to 2011 and extrapolated for 2012. Shade areas are 95% prediction intervals. After adjustment for secular trends, there was a significant increase in frequency of neoadjuvant chemotherapy in 2012 in the New England and east south central division (odds ratio 1.41, 95% confidence interval 1.25 to 1.72, P<0.001). In south Atlantic, west north central, and east north central divisions, treatment in 2012 was not associated with any deviation from secular trends (odds ratio 0.98, 95% confidence interval 0.86 to 1.12; P=0.78)
Table 1 shows patient level and hospital level characteristics of women treated for advanced epithelial ovarian cancer in 2011 and 2012, in regions that rapidly adopted NACT (n=1157) and control regions (n=4878). No differences in age, race/ethnicity, stage, histologic type, grade, or comorbidities were observed between patients with a diagnosis in 2011 versus 2012, in either New England and south central divisions, or control regions. Hospital attributes, such as annual volume of ovarian cancer cases, and academic affiliation, also remained unchanged. There was no significant difference in the number of hospitals that reported one or more cases of advanced ovarian cancer in 2011 compared with 2012 in rapidly adopting (92 v 95 hospitals, P=0.44) or control (374 v 378 hospitals, P=0.46) regions. Furthermore, only 1 of 95 (1.1%) hospitals in rapidly adopting regions and 7 of 378 (1.9%) hospitals in control regions that reported one or more cases of advanced ovarian cancer in 2012 had not reported a case in previous years.
Characteristics of patients and hospitals in New England and east south central census divisions, and south Atlantic, west north central, and east north central census divisions. Values are numbers (percentages) unless stated otherwise
Table 2 summarizes the results of the primary regression discontinuity analysis (tabulated in detail in appendix table B). Among patients treated in New England and east south central census divisions, treatment after increased utilization of NACT in 2012 led to a reduction in hazard of all cause mortality after accounting for mortality time trends (hazard ratio 0.81, 95% confidence interval 0.71 to 0.94). Conversely, in control regions, after adjustment for mortality trends, those treated in 2012 had similar hazard of death as patients treated in prior years (1.02, 0.93 to 1.12). The hazard reduction observed in New England and east south central divisions, was significantly larger than that of control regions (difference-in-differences P=0.001).
Regression discontinuity design analysis of the effect of increased utilization of chemotherapy on all cause mortality in women with advance ovarian cancer
Figure 2 plots the annual mortality rates for the New England and east south central division (A) and control regions (B). Blue circles are crude all cause mortality hazard estimates within three years of diagnosis. Predicted hazards are displayed as solid red lines, and dashed lines show the extrapolated predictions if 2007-11 hazard trends continued. Kaplan-Meier survival curves for patients with a diagnosis in 2011 and 2012 are plotted for the New England and east south central division (C) and control regions (D). In New England and east south central census divisions, women with a diagnosis after use of neoadjuvant chemotherapy increase abruptly in 2012 had superior survival then those with a diagnosis in 2011 (log rank P=0.02). Survival remained unchanged in control regions (log rank P=0.99)
Plots of annual mortality hazard rates from 2007 to 2012, and Kaplan-Meier survival curves for women treated in 2011 and 2012 in the New England and east south central census divisions (A and C) and control regions (B and D). While survival remained unchanged from 2011 to 2012 in control regions (log rank P=0.99), in New England and east south central regions survival improved in 2012 coincident with increased utilization of neoadjuvant chemotherapy (log rank P=0.02)
The proportion of women who died within 30 days of surgery declined from 3.1% to 1.8% from 2011 to 2012 in the New England and east south central regions compared with 1.9% and 2.2% in control regions (difference-in-differences P=0.02). During the same interval, 90 day postoperative mortality decreased from 7.0% to 4.0% in New England and east south central regions compared with 5.0% to 4.3% in controls (difference-in-differences P=0.01). The proportion of women who did not receive surgery and chemotherapy decreased from 20.0% to 17.4% from 2011 and 2012 in regions that rapidly increased NACT use compared with 19.0% to 19.5% in control regions (difference-in-differences P=0.04).
In sensitivity analyses assessing the robustness of the estimated causal effects to model specifications, we found that our main results were insensitive to functional form and period used to adjust for time trends (see appendix table C). Additionally, the use of exponential instead of Cox proportional hazard survival models did not alter our findings. The main results were also insensitive to inclusion of patient level covariates in the regression discontinuity and difference-in-differences models (see appendix table C). The effect of adoption of NACT was similar in New England and the east south central regions (see appendix table D). In a confirmatory regression discontinuity design, which analyzed the causal effect of NACT adoption in regions that did not increase use of NACT until 2013 (South Atlantic and east north central census divisions), adoption of NACT in 2013 also led to a reduction in mortality hazard (see appendix table E).
A cross sectional analysis of region-level associations between the prevalence of NACT and risk of all-cause mortality is illustrated in figure 3. In a given year, patients treated in census divisions with higher NACT use had significantly lower hazard of all cause mortality compared with those treated in regions with lower use of NACT (P=0.001). This association remained significant in a multivariable model adjusting for year of diagnosis, baseline regional hazard rates, age, race, stage, histology, grade, and comorbidity index (P=0.004).
For each year from 2004 to 2013, the relative prevalence of neoadjuvant chemotherapy in each of nine census divisions is plotted against the relative hazard of all cause mortality. Region specific relative mortality hazard and prevalence estimates utilize the national averages in each year as referents. Predicted relative hazards estimated from an exponential proportional hazard model are displayed as the blue line. After adjusting for year of diagnosis, treatment in regions with higher use of neoadjuvant chemotherapy was associated with a significantly lower hazard of death (P=0.001)
Discussion
In this study, we used quasi-experimental and cross sectional analyses to investigate the causal effect of NACT on all cause mortality in advanced ovarian cancer. We observed that a rapid increase in the use of NACT between 2011 and 2012 in New England and east south central census districts led to a reduction in mortality assessed through three years after diagnosis. In contrast, we found no difference in mortality in regions that did not increase NACT use. We confirmed these results in a cross sectional analysis of national data, wherein we observed that women treated in geographic regions with greater use of NACT had superior survival outcomes compared with those treated in regions with lower rates of NACT. Finally, we identified reductions in 30 day and 90 day postoperative mortality, and in the proportion of patients failing to receive both chemotherapy and surgery in high compared with low adopting regions as potential mechanisms underlying the lower mortality associated with NACT.
Two clinical trials found that NACT does not result in inferior overall survival compared with primary surgery.23 Although these trials did not show a statistically significant difference in overall mortality, a pooled analysis of the studies reported by Kehoe et al found that the point estimate favored NACT (hazard ratio 0.93, 95% confidence interval 0.82 to 1.05).3 Several factors may explain why increased uptake of NACT resulted in a significant overall survival benefit in routine practice that was not observed in clinical trials. Postoperative mortality after primary cytoreductive surgery is higher in population based studies in the United States, which include more heterogeneous populations, than in trials performed at specialty centers.21 Since it appears in this study and previous studies,22 that an important mechanism for the benefit of NACT is reduction in surgical morbidity and mortality, this benefit would be expected to be greater in routine clinical practice than in clinical trials.9 Furthermore, the clinical trials were conducted in Europe, Canada, and New Zealand, and it is believed that primary cytoreductive surgery may be more extensive in the United States, and our study suggests that it is associated with a greater risk of postoperative morbidity.5 Additionally, in the clinical trials treatment was randomly assigned, whereas in real world practice neoadjuvant chemotherapy has been adopted selectively for patients who incur the greatest risk and are likely to derive the least benefit from upfront surgery, such as older women and those with stage 4 disease.9 Additionally, the temporal proximity of the practice changes described in the present study allowed us to analyze mortality for up to three years after diagnosis, whereas the clinical trials analyzed longer term outcomes, and it is possible that the measured effect may attenuate with additional follow-up.
The survival benefit we observed also contradicts the results from some observational studies that have compared survival among women who received NACT with those who underwent primary surgery.10111213 Since patients with poor performance status and high disease burden are preferentially treated with NACT, observational studies that directly compare survival among women assigned to these treatment modalities are susceptible to selection bias.7 Even observational studies that adjust for measured confounders are likely to be biased by unobserved variables. Indeed, a propensity score matched analysis of data from the National Cancer Database found that women who received NACT fared considerably worse than those assigned to primary surgery; sensitivity analyses showed that this association was highly sensitive to unmeasured confounding by performance status.13 Similarly, in a study comparing primary chemotherapy with primary surgery among elderly women, Wright et al compared the results of multivariable regression, propensity score methods, and instrumental variables analyses, and concluded that results obtained from traditional multivariable adjustment and propensity score methods were likely to be biased by residual confounding.23
The quasi-experimental regression discontinuity design utilized in the current study eliminates patient selection as a source of bias, since neither patients nor providers choose the year that ovarian cancer is diagnosed. However, the use of year of diagnosis as the assignment variable in this analysis raises a question as to whether some unmeasured factor that affected survival may have coincided with the adoption of NACT between 2011 and 2012 in New England and east south central regions. While this possibility cannot be disproven, there are several observations that make such an explanation improbable. First, we identified a homogeneous effect in both New England and east south central divisions, so that any unobserved change would have had to occur simultaneously in two geographically distinct regions. Second, two of the three control regions in our main analysis increased utilization of NACT in 2013, and adoption of NACT in those regions was also associated with a survival benefit. Finally, results of the main quasi-experimental analysis were congruent with a cross sectional analysis, showing that women treated in regions with greater use of NACT had lower mortality than those undergoing treatment in regions with low utilization of NACT.
Importantly, our findings do not indicate that all patients with advanced epithelial ovarian cancer will benefit from NACT. The survival benefit measure in this study is consequence of expanded adoption of NACT, which occurred selectively among older patients and those with stage 4 disease.9 Furthermore, if the mechanism of benefit for NACT observed in this study is reduction in postoperative morbidity and mortality, NACT may be less beneficial in the context of expert centers that achieve better than average surgical outcomes. Indeed, in this study we observed that regions which adopted NACT rapidly had higher baseline perioperative mortality than control regions.
An important limitation of the current study is the data source: the National Cancer Database covers only the 70% of patients with cancer in the United States who obtain care at facilities accredited by the Commission on Cancer. As such, the present findings may not be generalizable to the few patients receiving care at non-accredited institutions. Reassuringly, we observed no major shifts in facilities participating in NCDB in either regions that rapidly adopted NACT between 2011 and 2012 or control regions, suggesting that our findings are not likely to be confounded by a shifting patient population.
Conclusion
Increased use of NACT for advanced epithelial ovarian cancer in rapidly adopting regions led to a mortality benefit within three years of diagnosis, which appears to be mediated by reduced postoperative morbidity and mortality. These findings should reassure clinicians and policy makers who have greeted increasing acceptance of NACT with some concern. Future research may elucidate how patients who may benefit most can be identified and selected to receive NACT.
What is already known on this topic
Two randomized trials found that women assigned to neoadjuvant chemotherapy (NACT) followed by interval surgery have better surgical outcomes and non-inferior survival than those who receive primary surgery and adjuvant chemotherapy for advanced ovarian cancer
However, in the United States primary surgery remains the preferred treatment, although use of NACT has increased in recent years
What this study adds
Rapid adoption of NACT for advanced ovarian cancer in two regions of the United States led to a decline in call cause mortality within three years of diagnosis
This effect appeared to be mediated by reduced postoperative morbidity and mortality
These findings should reassure clinicians and policy makers who have greeted increasing acceptance of NACT with some concern
Footnotes
Contributors: AM designed the study, initiated the collaboration, cleaned and analyzed the data, created the figures, and drafted and revised the paper. He is guarantor. GF designed the study, proposed the analytic methodology, and revised the draft manuscript. AAW contributed to the study design and revised the draft paper. NLK contributed to the study design, refined the analytic plan, and revised the draft paper. AG contributed to the study design and revised the draft paper. MDC contributed to the study design and revised the draft paper. JS obtained the data, contributed to the study design, and revised the draft manuscript. JAR-H designed the study, analyzed the data, and revised the draft manuscript.
Funding: This work was supported by the National Cancer Institute at the National Institutes of Health (R25CA092203 JAR-H, K24CA181510 NLK, and K07CA166210 AAW), and the Deborah Kelly Center for Outcome Research at Massachusetts General Hospital. The funders had no role in the design and execution of this study.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work
Ethical approval: This study was deemed exempt from peer review by the Partners Healthcare Research Committee.
Data sharing: The data on which this study is based are available through the American College of Surgeons to institutions participating in the National Cancer Database.
Transparency: The manuscript’s guarantor (AM) affirms that affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.