The volume-mortality relation for radical cystectomy in England: retrospective analysis of hospital episode statistics
BMJ 2010; 340 doi: https://doi.org/10.1136/bmj.c1128 (Published 22 March 2010) Cite this as: BMJ 2010;340:c1128
- Erik K Mayer, specialist registrar in urology and clinical lecturer in surgery12,
- Alex Bottle, lecturer3,
- Ara W Darzi, professor of surgery1,
- Thanos Athanasiou, clinical reader in surgery1,
- Justin A Vale, consultant urological surgeon2
- 1Division of Surgery, Department of Surgery and Cancer, Imperial College London, St Mary’s Hospital Campus, London W2 1NY
- 2Department of Urology, St Mary’s Hospital, Imperial College Healthcare NHS Trust, London
- 3Division of Epidemiology, Public Health and Primary Care, Imperial College London
- Correspondence to: E K Mayer e.mayer{at}imperial.ac.uk
- Accepted 11 November 2009
Abstract
Objectives To investigate the relation between volume and mortality after adjustment for case mix for radical cystectomy in the English healthcare setting using improved statistical methodology, taking into account the institutional and surgeon volume effects and institutional structural and process of care factors.
Design Retrospective analysis of hospital episode statistics using multilevel modelling.
Setting English hospitals carrying out radical cystectomy in the seven financial years 2000/1 to 2006/7.
Participants Patients with a primary diagnosis of cancer undergoing an inpatient elective cystectomy.
Main outcome measure Mortality within 30 days of cystectomy.
Results Compared with low volume institutions, medium volume ones had a significantly higher odds of in-hospital and total mortality: odds ratio 1.72 (95% confidence interval 1.00 to 2.98, P=0.05) and 1.82 (1.08 to 3.06, P=0.02). This was only seen in the final model, which included adjustment for structural and processes of care factors. The surgeon volume-mortality relation showed weak evidence of reduced odds of in-hospital mortality (by 35%) for the high volume surgeons, although this did not reach statistical significance at the 5% level.
Conclusions The relation between case volume and mortality after radical cystectomy for bladder cancer became evident only after adjustment for structural and process of care factors, including staffing levels of nurses and junior doctors, in addition to case mix. At least for this relatively uncommon procedure, adjusting for these confounders when examining the volume-outcome relation is critical before considering centralisation of care to a few specialist institutions. Outcomes other than mortality, such as functional morbidity and disease recurrence may ultimately influence towards centralising care.
Introduction
The relation between volume and outcome in health care, based on Donabedian’s structure-process-outcome paradigm,1 has influenced English health policy, with centralisation of oncology services across several specialties.2 Despite large amounts of evidence supporting higher volume-better outcomes,3 several studies have shown that surgeons or institutions with low volume caseloads can have excellent outcomes and some high volume surgeons or institutions can have poor outcomes.4 This, along with variability between studies in the magnitude of the volume-outcome relation reported, has led authors to question the methodological quality of existing studies and also show that improved methodology can attenuate previously reported results.5 The most consistent volume-outcome relations have been for procedures such as pancreatectomy and oesophagectomy, probably because they have sufficient case denominators and frequency of measurable adverse events to show differences between providers.6 Within urology, radical cystectomy has been commonly used for volume-outcome research and it has been included in reforms striving to improve outcomes for cancer treatments.2
The US Institute of Medicine critically appraised the volume-outcome relation in the context of healthcare quality and produced a methodological scoring system that determines a study’s ability to discern a true volume-outcome relation. For oncology studies published between 1984 and 2004 these scores were only modest at best6 and had not improved significantly from those in a previous systematic review.7 Similar findings have been reported for studies specific to uro-oncology,8 and over 90% of these studies originated outside of the UK healthcare system, with over 80% from the United States.8 Differences in the healthcare system limit the transferability of this information to the UK setting, particularly when several US datasets do not reflect the entire treated population. Assessing the existing literature using this methodological scoring system7 highlighted several important methodological considerations for volume-outcome research: the impact of the hierarchical structure of the institution and surgeon (multilevel modelling), appropriate handling of the provider volume variable, and the need for adjustment of structural or process of care measures. We took these considerations into account while investigating the volume-mortality relation for radical cystectomy in England.
Methods
We took data for inpatient elective cystectomies (code M34 from the Office of Population Censuses and Surveys Classification of Surgical Operations and Procedures, fourth revision, occurring in any procedure field in any episode) from hospital episode statistics for the six financial years from 2000/1 to 2005/6 and from the secondary users service for 2006/7. Hospital episode statistics is the national statistical administrative database for England of the care provided by National Health Service hospitals. Each record in the database constitutes a finished consultant episode, which covers the continuous period during which a patient, admitted to a hospital, is under the care of the same consultant, whose unique identification code (used by the doctor’s regulatory body, the GMC) is recorded. We linked episodes with admissions using the patient’s date of birth, sex, and postcode; the hospital; and date of admission, and we linked admissions together if the patient was transferred to another trust. A hospital trust provides secondary healthcare services within the NHS, which can either be planned specialist medical care, surgery, or emergency care. A trust may comprise several hospital sites.
We excluded people if they were admitted as an emergency, had an invalid age, sex, or length of stay, were day cases, or did not have a primary diagnosis of cancer (codes C66-68 and D090, international classification of diseases, 10th revision). For each year we noted the proportion of patients undergoing cystectomy for bladder cancer.
Assignment of institutional and surgeon volume bands
We counted the number of cystectomies by year and each of consultant code and trust. To account for trust mergers, we unified trust codes to reflect their status as of April 2007. Very low volume trusts (institutions) were excluded if they either had fewer than three years of data or recorded fewer than two cystectomies annually. This was to try to capture institutions with genuine activity—that is, no coding errors. If a provider had less than three years of data, we considered it unlikely that they were carrying out cystectomy regularly during the period analysed. This had to be balanced against not excluding providers (at the surgeon level) that either retired or were newly appointed during the analysed period and may therefore only appear for three or less years. We thought that this three year cut-off point, in combination with an annual cystectomy rate of two, would identify cases more likely appearing as a result of coding error, while not inappropriately removing large numbers of low volume providers—that is, those regularly carrying out between two and 10 operations per year at the institution level, or between one and five operations per year at the surgeon level. On this basis we excluded 13 institutions providing 41 cases (0.5% of total cases). We put the remainder into three volume bands of roughly equal total numbers of operations (table 1⇓).
Number of operations by institutional and surgeon volume bands, generated before regression modelling
We allocated consultants to volume bands in a similar fashion, but because surgeons could operate at more than one institution, and hence be included in more than one institutional volume band, allocation was done in two stages. After we excluded a record with a missing consultant code, we calculated the surgeon’s average annual cystectomy rate. We excluded consultants if their average cystectomy rate was two or less cases per year and they had fewer than three years of data. Because coding errors occur at the level of the institution, exclusion was done without summating caseload of consultants operating at multiple institutions. Overall, we excluded 189 consultants providing 313 operations (3.6% of total cases). We then recalculated the annual cystectomy rate for those surgeons operating at more than one institution, using the maximum number of years practised from whichever provider and total summation of operations. Consultants were then divided into three volume bands of roughly equal operation numbers (table 1).
Outcome measures
We assessed all cause mortality within 30 days of cystectomy (or admission if the date of procedure was missing) in two ways: in-hospital mortality and mortality either in or out of hospital (total mortality). We calculated total mortality by using a date of death linked to hospital episode statistics by the Office for National Statistics. This was unavailable for 2006/7, but to avoid losing a year’s records and to maintain statistical power, we assumed that no deaths occurred out of hospital in 2006/7; we undertook a sensitivity analysis on the small shortfall (see appendix 1 on bmj.com).
Volume-mortality analysis
We used four different models of increasing complexity to assess the volume-mortality relation (see appendix 1 for details). In summary, each model assessed the relation between each volume variable (surgeon and institution) and mortality. Model 1 did not adjust for any other variables. Model 2 adjusted for patient case mix. Model 3 also took account of the “clustering” of patients within surgeons and surgeons within hospitals. Finally, model 4 included adjustment for structural and process of care variables.
We present volume-mortality relations as odds ratios, with the low volume band as the reference group. The interaction effects between institution and surgeon volumes were assessed.
Statistical analysis
Using χ2 tests we assessed variations in case mix variables within volume groups and between volume groups. All logistic regression and other statistical analysis except for multilevel modelling (done using PROC GLIMMIX in SAS version 9.2) were done using the SPSS package version 14 for Windows.
Results
The number of radical cystectomies carried out each year increased steadily from 1130 in 2000/1 to 1296 in 2006/7 (P=0.005 for linear trend). Table 1 shows the distribution by institution and surgeon volume. A significant downward trend occurred in total mortality, from 3.5% in 2000/1 to 2.1% in 2005/6 (deaths out of hospital were not available for 2006/7). Table 2⇓ shows observed mortality by institution and surgeon volume.
Observed rates for in-hospital and total mortality within 30 days of cystectomy, stratified by institutional and surgeon volume bands. Values are numbers (percentages) of patients
Characteristics of patients and institutions
The proportion of treated patients aged 75 years or more increased from 19.8% in 2000/1 to 22.7% in 2006/7 (P=0.03). The proportion of men was significantly greater in 2000/1 and 2003/4 than in the other years (P=0.03). The proportion of patients in the highest Charlson category (6+)9 varied significantly year on year, but with no linear trend across the years. For every year the median Charlson score9 and Carstairs fifths10 were 2 and 3, respectively.
Medium volume institutions treated proportionally fewer patients in the most deprived Carstairs fifth (highest fifth) and a greater proportion of patients older than 75 (P<0.01, table 3⇓). The number of institutions with teaching status increased significantly from the low volume group to the high volume group (P<0.01). Medium and high volume surgeons treated a greater proportion of men and fewer patients in the most deprived Carstairs fifth (P<0.01). Medium and high volume surgeons more often worked in institutions with teaching status than low volume surgeons. Differences in the structural and process of care variables were significant across the bands for institutional volume (table 4⇓).
Differences in characteristics of patients and institutions by provider volume group
Variation in structural and process of care measures between institutional volume bands. Minimum and maximum values are in brackets
Volume-mortality relation
For medium volume institutions in model 3 the odds of in-hospital mortality was 1.16, but increased to 1.72 (95% confidence interval 1.00 to 2.98, P=0.05) in model 4 (including adjustment for the structural and process of care variables). Adjusting for the ratio of the number of nurses to occupied beds increased the odds to 1.31; this became 1.52 after adjusting for the ratio of urology registrars to total urology episodes, and 1.62 after including waiting time for operations. Adding the remaining structural and process of care variables to model 4 gave an odds ratio of 1.72. A similar pattern was seen for total mortality (model 4, 1.82, 1.08 to 3.06, P=0.02). Differences were not significant when using models 1-3. The only significant result for high volume institutions was a lower odds of in-hospital mortality in model 2 (0.67, 0.47 to 0.97, P=0.03; fig 1⇓).
Fig 1 Odds ratios and confidence intervals for institutional volume-mortality relations across four models
High volume surgeons had a significantly lower odds of in-hospital mortality for model 1 (0.67, 0.48 to 0.95, P=0.03) and model 2 (0.64, 0.44 to 0.91, P=0.01), but this became non-significant at the 5% level in models 3 and 4. Total mortality did not differ significantly (fig 2⇓).
Fig 2 Odds ratios and confidence intervals for surgeon volume-mortality relations across four models
Figure 3⇓ summarises the adjusted probabilities for in-hospital and total mortality across both institutional and surgeon volume bands. Adjusted probabilities were calculated using model 4.
Fig 3 Adjusted probabilities (derived from model 4) for in-hospital and total mortality across institutional and surgeon volume bands
Interaction effect
We tested for an interaction between the two volume variables. This resulted in a significant interaction term (P=0.035) but non-significant main effects (P>0.1), suggesting no evidence for a convincing interaction effect.
Discussion
Patients undergoing radical cystectomy for bladder cancer in medium volume (≥10 and <16 procedures per year) institutions had 72% and 82% greater odds of 30 day in-hospital and total mortality than patients undergoing the same procedure in low volume (>2 and <10 procedures per year) institutions. This increased risk in mortality was seen only after adjustment for institutional structural and process variables. Staffing levels for nursing and urology registrars seemed to have the greatest influence, in keeping with previous studies.11 12
Although not significant at the 5% level, weak evidence was found for a reduced odds of in-hospital mortality (by 35%) associated with the high volume (>8 procedures per year) surgeons. Surgeon volume also seemed to have some protective effect on in-hospital mortality at the institutional level. After adjustment for patient case mix alone, high volume institutions displayed statistically significantly lower odds of in-hospital mortality. This effect disappeared after adjusting for surgeon volume.
There was relatively little difference in the crude odds ratio and case mix adjusted odds ratio of mortality at both institutional and surgeon level. This is interesting in itself, as surgeons often use variation in case mix to explain differences in performance. It may be that adjustment for case mix, using variables as incorporated in this study, is not always necessary for analysis of effects at a highly aggregated level, as opposed to risk prediction at an individual patient level. One study13 also reported a minimal difference in the unadjusted and adjusted (for patient characteristics) odds ratios of mortality for cystectomy between low volume and high volume institutions.
Strengths and weaknesses of the study
The median Institute of Medicine quality score7 for previous studies appraising the volume-outcome relation for cystectomy was modest, at 8.5 (range 8-11).8 Using the same scoring criteria, the current study would achieve a higher score of 13 out of a possible 18. One mark was lost because, for practical purposes, we chose to report on only the volume-mortality relation and removed all other outcome measures. We acknowledge, however, that outcomes other than mortality are important when considering the reconfiguration of health care. Four marks were not achievable because we used an administrative database and therefore could not determine the appropriateness of patient selection for cystectomy or use clinical data for risk adjustment. Methods to circumvent this limitation could include linkage of administrative datasets with local or specialty society datasets and simultaneously serving to confirm the accuracy of coding. The need to centrally support such initiatives across the multiple available datasets has been acknowledged.14 Although we were unable to adjust data for cancer stage and grade, when this has been possible it seemed to have little effect on the reported findings.13 15
It was not possible to carry out a subanalysis of mortality risk for patients who had an ileal conduit diversion or a continent diversion because of underreporting of continent diversions. Similar difficulties have also been experienced with US administrative databases.11 Clinicians have for some time questioned the reliability of coding within hospital episode statistics. In England, training of coding staff, regular monitoring of data quality, and a commissioning system that financially reimburses healthcare providers according to case mix adjusted activity will have improved the quality of data completeness and accuracy since the early years of hospital episode statistics. If institutional resources for coding are proportional to the overall workload of that institution, it is possible that miscoding rates are not systematically different by volume of hospital. This will have biased the estimated odds ratios towards 1 and therefore underestimated any volume-mortality relation.
Our definitions for mortality outcome used a 30 day follow-up only, as is usual.15 Overall, 90 day mortality is known to be higher than 30 day mortality after cystectomy,16 but whether this differs across providers is unclear; work on this and longer term cancer specific survival should consider postoperative structural and process of care factors such as ancillary services for oncology.12 The only previous study that used hospital episode statistics to assess mortality after cystectomy included in-hospital mortality only.17 We assumed that no deaths occurred out of hospital in 2006/7, rather than the four on average for the other years. Although this assumption is not expected to have been true in reality, the sensitivity analysis showed this was unlikely to have biased the results for this outcome.
The finding that medium volume institutions have higher odds of mortality is surprising. Without further evaluation of potential causative factors, this is difficult to fully explain. We did not assess the relative contribution of the institution and surgeon to the volume-mortality relation. We also did not attempt to define causal pathways of influence of the structural and process of care variables on mortality. Previous studies, for instance, have shown that for colorectal resections medium volume surgeons can achieve results for inpatient mortality similar to those of high volume surgeons when operating in medium volume or high volume hospitals but not low volume hospitals.18 Similarly, the contribution of individual institutional structural variables can be important. When low volume institutions have as many residents or interns and registered nurses per 100 beds as high volume institutions, overall mortality for paediatric heart surgery and heart transplants appeared equivalent.19
From the summary of adjusted probabilities (fig 3), medium volume institutions do seem to have worse in-hospital and total mortality than high volume institutions, which is consistent with previous studies. What is not clear is why low volume institutions seem to have comparable outcomes to high volume institutions. Although it would be impossible to refute this finding, it may be artefactual. The division of volumes into thirds does not assume a linear relation between volume and mortality but rather allows for a non-linear one such as the middle volume having lower or higher mortality than the lower and higher volumes. It is possible that our cut-offs may not be optimal and that more complex functional forms (for example, quadratics, splines) may perform better. The plotting of mortality at unit level against unit volume for our dataset did not reveal any obvious relations, and we chose thirds because of statistical power and because such an approach is commonly used. We did not exclude large numbers of very low volume providers, which may have contaminated the low volume third if it contained patient level records present only through coding errors.
Some of the structural and process of care factors used in the model may themselves be acting as proxies for volume. Several of the variables, such as staffing levels, critical care facilities, and hospital capacity do increase across the institutional volume bands (table 4). In extreme cases this can lead to “colinearity,” with multiple variables trying to describe the same quantity, but no convincing evidence of this was shown by inspection of the standard errors. However, the noticeable increase in the width of the confidence interval for the institutional volume group with the addition of the structural and process variables suggests that we were at the limit of acceptability for the number of variables included.
The use of an incremental modelling approach for volume-outcome research is in itself important for helping to decide whether volume should be defined at the institutional or surgeon level, or both. Using the institution allows for the importance of overall teamwork on outcomes by factoring in institutional factors that cannot always be measured. It has been acknowledged that existing studies have rarely considered the relative effects of the unit of analysis and their possible interactions.20
Comparisons with other studies
A previous study17 used only two volume categories in its hospital episode statistics based analysis and defined an “optimum” annual institutional caseload of 11 cases per year. Shifting the case volume threshold to 16 cases per year included more centres with optimum mortality outcomes in the group below the threshold (<16 cases/year) and therefore found no significant difference in mortality between the groups.
A systematic review, using a random effects analysis, found an 88% higher odds of mortality for cystectomy in low volume institutions than in higher volume ones.8 Only five studies could be included, and their methodological quality was assessed as only modest at best. That the difference was greater than ours can be partly explained by the limitation of meta-analysis when studies use different volume cut-off categories and adjustment methods and amalgamate studies from different healthcare settings.8 US studies using the healthcare utilisation project found 3.2 times and 1.96 times the case mix adjusted risk of dying postoperatively in low volume institutions compared with high volume institutions (cystectomy rate 1-5/year v ≥20/year12 and <1.5/year v ≥2.75/year,21 respectively). The direction of effect is consistent with the results of our study, although their magnitude of effect was greater.
It could be suggested that the differences in the annual caseloads between surgeon volume categories defined in this study are only marginal. Compared with other studies, however, the cut-offs are relatively far apart. One study22 showed an adjusted odds ratio for death postoperatively for patients with a low volume surgeon (<2 cases/year) compared with those with a high volume surgeon (>3.5 cases/year) of 1.83 (95% confidence interval 1.37 to 2.45). Conversely, another study21 found no overall differences in in-hospital mortality across surgeon volume but used only marginal volume cut-offs (low volume ≤1 cases/year and high volume >1.5 cases/year).
Few studies have investigated the effects of institutional structural variables and processes of care in the treatment pathway for cystectomy. Some evidence from the United States suggests that adjusting for processes of care before, during, and after radical cystectomy accounts for 23% of the volume-mortality effect.13 Institutional structural variables can attenuate the volume-mortality relation by up to 59%.12
Implications for clinicians and policymakers
Interpretation of the institutional volume-mortality relation reported in this study was only possible after adjustment for institutional structural and process of care variables. Adjusting for these confounders, in addition to case mix, must be done before considering the use of volume-outcome data to support centralisation of care to a few high volume centres specialising in radical cystectomy for bladder cancer. We made no effort to determine causal relations between individual structural and process of care measures and mortality after radical cystectomy, and this will need to be the subject of further research. This will also need to be combined with research exploring the volume-outcome relation for outcomes other than mortality. Mortality is only one of several outcomes that need to be examined when trying to improve the quality of care for patients. Longer term outcomes including functional morbidity and disease recurrence may ultimately influence towards centralising care and so help inform the future design and reconfiguration of services.
Longitudinal data analysis is essential for assessing causality and causal direction, as it indicates whether outcomes improve if annual volume of activity increases over time. As previously reported,23 studies that have shown volume-outcome relations on cross sectional analysis have subsequently found no relation between changes in volume and outcome over time. The absence of reliable longitudinal data in volume-outcome research has been highlighted as a concern when trying to predict, with any certainty, the success of centralising services to improve outcomes.20
Adjustment for structural and process of care variables seemed to have the most impact at the level of medium volume institutions. The difficulties and dangers of trying to define an absolute minimum caseload are evident. More work is needed to define medium volume institutions and also the surgeon’s volume that makes it protective for institutional outcomes. Centralising care to specified institutions without accounting for the inherent structural and process factors that result in optimum care is meaningless. Equally, the outcomes a surgeon can achieve depend on the support network provided at the institutional level. The relative contribution of all these factors needs to be considered when reconfiguring service delivery to improve the quality of care for patients.20
Unanswered questions and future research
The volume-mortality relation defined in this study is only one component of a much broader future quality assessment matrix that must combine other integral factors, such as adherence to process of care measures encompassing the entire patient treatment pathway; other outcomes of surgery, including longer term clinical outcomes (cancer recurrence rates and cancer specific survival); patient reported measures, including quality of life assessment; and assessment of patient satisfaction and experience. We have not included volumes of other related operations that may influence mortality risk. The impact of overall uro-oncological operative volumes has been highlighted and, for cystectomy, adjustment for institutional volumes of nephrectomies and prostatectomies attenuated the institutional cystectomy volume-mortality relation by 60%.24 This is particularly important for health care in England, and guidance for improving outcomes for urological cancer services, as the minimum caseload guidance for radical pelvic surgery of 50 cases per year considers cystectomy and prostatectomy in combination.
Further research needs to explore the relative contributions of each of the factors, the institution, the surgeon, and individual characteristics of structural and process of care, and explore the relation between caseload volume and operative outcomes other than mortality. This, in combination with further efforts to improve the methods used for volume-outcome research, will be important for informing future changes to healthcare services, such as centralising care for relatively uncommon operations such as radical cystectomy, and improving the quality of care delivered.
What is already known on this topic
Healthcare reconfiguration, such as centralisation of cancer services, arose from evidence supporting a relation between higher volume and better outcomes
Variability in the magnitude of volume-outcome relations questioned the methodological quality of existing studies
Over 90% of studies on volume-outcomes specific to uro-oncology have originated outside the United Kingdom
What this study adds
A volume-mortality relation after radical cystectomy exists at the institutional level, and to a lesser degree at the surgeon level, in England
Appropriate interpretation of the volume-mortality relation was only possible after adjusting for institutional and surgeon volumes and structural and process of care confounders
For radical cystectomy, centralising care to a few institutions should only be considered once the volume-outcome relation has been adjusted for structural and process of care variables such as staffing levels of nurses and junior doctors, as well as case mix
Notes
Cite this as: BMJ 2010;340:c1128
Footnotes
Contributors: EKM, TA, and JAV conceived and designed the study. EKM and AB analysed and interpreted the data and drafted the article. AWD, TA, and JAV revised the manuscript critically for important intellectual content. All authors gave final approval of the version to be published. EKM is guarantor.
Funding: None.
Competing interests: AB is based at the Dr Foster Unit, which is funded by a research grant from Dr Foster Intelligence, an independent health service research organisation.
Ethical approval: We have section 60 approval from the Patient Information Advisory Group to hold confidential data and analyse them for research purposes.
Data sharing: No additional data available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.