Derivation and validation of a risk adjustment model for predicting seven day mortality in emergency medical admissions: mixed prospective and retrospective cohort study
BMJ 2012; 344 doi: https://doi.org/10.1136/bmj.e2904 (Published 01 May 2012) Cite this as: BMJ 2012;344:e2904
- Steve Goodacre, professor of emergency medicine,
- Richard Wilson, research associate,
- Neil Shephard, medical statistician,
- Jon Nicholl, professor of health service research
- on behalf of the DAVROS Research Team
- 1School of Health and Related Research (ScHARR), University of Sheffield, Sheffield S1 4DA, United Kingdom
- Correspondence to: S Goodacre s.goodacre{at}sheffield.ac.uk
- Accepted 20 March 2012
Abstract
Objectives To derive and validate a risk adjustment model for predicting seven day mortality in emergency medical admissions, to test the value of including physiology and blood parameters, and to explore the constancy of the risk associated with each model variable across a range of settings.
Design Mixed prospective and retrospective cohort study.
Setting Nine acute hospitals (n=3 derivation, n=9 validation) and associated ambulance services in England, Australia, and Hong Kong.
Participants Adults with medical emergencies (n=5644 derivation, n=13 762 validation) who were alive and not in cardiac arrest when attended by an ambulance and either were admitted to hospital or died in the ambulance or emergency department.
Interventions Data were either collected prospectively or retrospectively from routine sources and extraction from ambulance and emergency department records.
Main outcome measure Mortality up to seven days after hospital admission.
Results In the derivation phase, age, ICD-10 code, active malignancy, Glasgow coma score, respiratory rate, peripheral oxygen saturation, temperature, white cell count, and potassium and urea concentrations were independent predictors of seven day mortality. A model based on age and ICD-10 code alone had a C statistic of 0.80 (95% confidence interval 0.78 to 0.83), which increased to 0.81 (0.79 to 0.84) with the addition of active malignancy. This was markedly improved only when physiological variables (C statistic 0.87, 0.85 to 0.89), blood variables (0.87, 0.84 to 0.89), or both (0.90, 0.88 to 0.92) were added. In the validation phase, the models with physiology variables (physiology model) and all variables (full model) were tested in nine hospitals. Overall, the C statistics ranged across centres from 0.80 to 0.91 for the physiology model and from 0.83 to 0.93 for the full model. The rank order of hospitals based on adjusted mortality differed markedly from the rank order based on crude mortality. ICD-10 code, Glasgow coma score, respiratory rate, systolic blood pressure, oxygen saturation, haemoglobin concentration, white cell count, and potassium, urea, creatinine, and glucose concentrations all had statistically significant interactions with hospital.
Conclusion A risk adjustment model for emergency medical admissions based on age, ICD-10 code, active malignancy, and routinely recorded physiological and blood variables can provide excellent discriminant value for seven day mortality across a range of settings. Using risk adjustment markedly changed hospitals’ rankings. However, evidence was found that the association between key model variables and mortality were not constant.
Supplementary data appendix
Footnotes
We thank Susan Proctor for clerical assistance; Mike Bradburn for statistical assistance; John Wooller, Ellis Frampton, James Gray, and Peter Mortimer for their help with collecting ambulance service data; and Mike Clancy, James Munro, Gareth Parry, Michael Schull, and David Harrison for help with the development of the project.
The DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) Research Team includes the Project Management Group (Steve Goodacre, Richard Wilson, Neil Shephard, Jon Nicholl, Martina Santarelli, Jim Wardrope); the principal investigators (Alison Walker (Yorkshire Ambulance Service), Anne Spaight (East Midlands Ambulance Service), Julian Humphrey (Barnsley District General Hospital), Simon McCormick (Rotherham District General Hospital), Anne-Maree Kelly (Western Hospital, Footscray, Victoria), Tim Rainer (Chinese University of Hong Kong), Tim Coats (Leicester Royal Infirmary), Vikki Holloway (Northampton General Hospital), Will Townend (Hull Royal Infirmary), Steve Crane (York District General Hospital)); and the Steering Committee (Fiona Lecky, Mark Gilthorpe, Enid Hirst, Rosemary Harper).
Contributors: SG and JN conceived the project and designed it with help from A-MK and JW. RW, MS, and the principal investigators were responsible for data collection. NS and JN analysed the data. SG wrote the first draft of the paper. The Steering Committee provided independent advice and oversight of the project. All authors assisted in the interpretation of data and revising the paper and approved the final draft. SG is the guarantor.
Funding: The DAVROS project was funded by the Medical Research Council. The researchers were independent from the funders. The funders had no role in conducting the study, writing the paper, or the decision to submit the paper for publication.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that the Medical Research Council provided grant funding to the participating organisations, but the authors did not receive any personal financial reward, have no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and have no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: The study was approved by the Leeds East Research Ethics Committee, the United Kingdom (UK) National Information Governance Board, and ethics committees in Melbourne and Hong Kong.
Data sharing: No additional data available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.