From research to practice: results of 7300 mortality retrospective case record reviews in four acute hospitals in the North-East of England

Identification of cases for review

The four trusts used different methods to identify cases, depending on the functional ability of their electronic or paper-based clinical records. Records were selected for review in order to maximise the opportunity to learn from problems in care. Deaths were identified: where a complaint or incident was recorded in incident reporting systems; where one of the various hospital standardised mortality indicator measures used by the NHS suggested mortality might be higher than expected; through referral to the mortality review process by clinical teams; and deaths following elective admission. Hospitals supplemented these with unselected deaths identified through convenience sampling, based on the availability of case notes. A high proportion of deaths were reviewed. Mortality reviews were instigated in these trusts as a practical approach to clinical quality improvement and assurance, rather than as a formal research study to ascertain the rate of potentially avoidable mortality.

For the purposes of this study, we removed duplicate patients and any patients with a discharge code that indicated that this was not an inhospital death.

A key objective was to test the feasibility of introducing a centralised method for reviewing deaths. Many people believed that reviewing a high proportion of hospital deaths is impractical, given that approximately half of all deaths in the NE of England occur in hospital.

Case note review methods

Basic data from the hospital record, including patient demographics, date of admission, date of death and method of admission (elective or unplanned), were obtained from the hospital’s Patient Administration Systems. The case notes were reviewed against a structured questionnaire with data being entered into spreadsheets or databases at each site. The questionnaires used were local adaptations of PRISM 1. Questions were not identical in all trusts and there was some evolution over time in each, but were broadly similar and concerned: cause of death, prehospital care, initial hospital clerking, first review and/or consultant review, management of deterioration, grading of preventability and quality of care, and a clinical resume of the case. Reviews were conducted largely by consultants, in some cases supported by nursing colleagues. Each hospital organised their review process differently, but all were aware of the work of the other hospitals, partly through a Regional Mortality Group, which had been established to share learning in this area. There was a small amount of peer review work, with teams reviewing cases in each other’s hospitals as a way of increasing the consistency of reviews. When a case was controversial or interpretation problematic, this was discussed within the team present. Grading of care used the 6-point preventability Likert scale used by PRISM and the 5-point NCEPOD grading of quality of care.

The choice of open and closed questions facilitated ease of data handling and provided a detailed understanding of the patient’s journey. Data from the four databases were amalgamated into a single database for reporting.

Ethical approval

Mortality reviews were conducted in the four trusts as part of normal clinical governance and quality assurance processes. All trusts sought to fulfil their Duty of Candour22 obligations. They were intended as audits aimed at improving care processes for patients and were not originally conceived as research nor intended for publication. Hence, it was believed that research ethics approval was neither appropriate nor needed. When we analysed the data together, it became clear that we had collected a larger number of reviews than had been published previously and that our estimates of preventability differed from the PRISM studies and we therefore sought to publish our data in order to make it available to inform the current debate about routine publication of mortality review information in the English National Health Service which is due to begin in September 2017.

Feedback methods

The four trusts used a variety of methods to communicate outputs from the RCRR to specialty teams and to the wider organisation. Where problems were identified in the care of specific patients, feedback to the team responsible for the clinical care was provided by the review team. In most cases, the managing team was then asked to review the case and comment. In some cases, incident reporting was already in place (eg, because of a patient fall, hospital-associated infection or other safety event) or an incident report was instigated. If the problem identified was serious, discussion was escalated to the Medical Director of the trust.

Individual trusts produced thematic mortality reports (detailing recurring themes within the patient journeys), which were presented to the trusts’ mortality committee or other parts of the governance structure. Reports were also discussed at trust and directorate governance or education meetings. Although we have not performed a formal thematic analysis, we know that problems in care (or documentation of care) at end of life, in identifying and responding to acute deterioration and in communication between multiple clinical teams looking after complex patients were found in all four trusts. Trusts either used or created ‘mortality leads’ in clinical departments and directorates or used existing governance structures. This was in an attempt to provide awareness that the trust had established centralised mortality review systems, which complemented existing specialty-based morbidity and mortality meetings.

Patient characteristics

In total, 7370 mortality review records were analysed; the cumulative figures are provided in table 1. The data are shown in two time periods to show that more than 60% of the reviews were conducted in the latter 2 years and to show that a review rate of more than 50% is achievable once the process is established.

Quantitative scores

Our analysis has focused on two key quantitative scores, NCEPOD scale and Preventability scale. Tables 2 and 3 provide details of results for each of these scores.

Trust Preventability scales compared with PRISM 2

Figure 1 and table 4 provide a comparison between NE total and the deaths graded as preventable reported in PRISM 2. There were no statistically significant differences between the four trusts (using CI method, not shown).

There is a statistically significant difference between the series of deaths in the  NE cohort and PRISM 2, as there is a gap of 1.77% CI between the lower limit of PRISM 2 preventability grading and the upper limit of the NE trust group.

Main findings

We have provided data for 3 years relating to over 7300 inpatient deaths in four acute trusts in the NE of England. Each trust conducted between 1200 and 2200 reviews over this period. During 2014–2015, this equates to an average of 54% of all inhospital deaths. Data relevant to these deaths was uploaded to a single database. Internal reviews were conducted at each trust and scoring determined across two nationally accepted standardised parameters for each death. Using a unified database and standardised data collection enabled a consistent approach to understanding deaths by each trust, data analysis and the possibility of shared learning across a region.

The overwhelming majority of all deaths reviewed were categorised as having had ‘good’ care and were ‘definitely not’ preventable. There was a small number of deaths (range 0.1%–0.8%, average 0.5%) in this cohort, which have been identified as having ‘room for improvement’ in care and a ‘greater than 50% chance of being preventable’. If we extrapolate this finding, it equates to 1143 inhospital deaths across NHS England, which could potentially be described as ‘avoidable’.

We believe that this regional unified approach delivers a pragmatic platform to learn from inhospital deaths. In addition, as all trusts seek to implement the recommendations of the Care Quality Commission (CQC) report ‘Learning, candour and accountability’,23 our approach allows providers to learn from our experience.

Comparison with existing evidence

Our data shows that the reviews conducted by the four participating trusts by internal reviewers produced lower estimates of preventability than previously published data (PRISM 2) with the trusts all showing significantly lower preventability than the PRISM 2 estimate.11

The PRISM 2 study produced an estimate of the proportion of reviews graded as greater than 50% chance of being preventable (that is preventability >3) of 3.0% with a 95% CI 2.7% to 3.7%. Comparing the NE figures (and their CIs) with this estimate shows that they are significantly lower than the PRISM 2 estimate for preventability. Hence, the reviews carried out by the NE differ significantly from the reviews carried out by the PRISM 2 study. PRISM 2 is used as the benchmark for comparison as it is the largest and most recent study of retrospective mortality review data in England.

It is not possible to fully explain this difference, but there are a number of possibilities. First, that the quality of care provided in the NE may be better than elsewhere in NHS England. Two of the trusts have been rated as ‘requires improvement’ by CQC, one as ‘good’ and one as ‘outstanding’ during this period; however, the large difference in results makes this explanation implausible. Second, that the ratings generated by the four trusts are graded on a single review, as opposed to the ‘multiple review’ approach undertaken in the PRISM studies. Third, the NE reviews are non-random and are instead orientated by the availability of notes and identification of cases where problems in care may be present, for example, incident reporting. We note that as the samples are large (over 50% for most trusts), it is unlikely that this potential sampling bias is large enough to account for the resultant difference in grading. Fourthly, that internal reviewers are more ‘lenient’ than external reviewers and produce systematically different results to those produced by external reviewers (as was the case for the PRISM studies). This may be a matter of ‘calibration’: PRISM reported inter-rater reliability as only moderate (κ 0.45), similar to that reported in other studies, and it may be that NE reviewers were less likely to judge a death to be preventable ’more likely than not' than PRISM reviewers. Results in table 3 show there is some room for reviewers to judge a death to be preventable without crossing this threshold (Preventability scores 2–6=5.8%, 95% CI 0.0% to 13.7%). The scale encourages reviewers to select a category just above or below the 50% threshold but modest inter-rater reliability may mean where a reviewer allocates a death on this spectrum is subjective. It is possible that as reviewers are examining deaths within their own institutions, they are subject to unconscious biases due to accepting as normal, care processes that would not be so regarded by external reviewers.

Further work is required to examine which of these possible causes (or a combination of these or some other unconsidered factor) is in fact the source of the difference for the order of magnitude difference in results from our internal reviewers compared with PRISM. Perhaps embedding prospective structured mortality reviews into the clinical governance process and generating continuous targeted reminders to departments to be vigilant in areas of care identified as potentially weak had an impact on preventability. Trust processes are iterative and encourage continuous internal review, reflection and recommendations in departments. In contrast, PRISM could only provide feedback to trusts after completion of their study, and while researchers are equally keen to learn, this difference may be affecting results. Further research into the amount of learning taking place and whether avoidable mortality falls over time will be needed to support or reject this  suggestion.

When NHS providers publish estimates of avoidable mortality from December 2017 (for deaths from April 2017), as was announced by the Secretary of State on 13 December 2016,25 our data suggest the results will be markedly different from PRISM. Exploration of the published data should include a detailed thematic analysis of future prospective mortality reviews. Widening the focus to include multidisciplinary involvement including prehospital input from primary care and ambulance services should also inform improved learning after a death.

Overall, we found routine, hospital-based mortality review to be feasible and useful in that it identified quality-of-care issues, but likely to report lower levels of preventability than published research.