What are safety cases?

Safety cases form part of a proactive safety management approach. The purpose of a safety case is to communicate why a product, system or service is deemed acceptably safe for use in a particular environment. A safety case comprises two complementary components: (1) a structured and explicit argument that (2) is supported by a body of evidence. The argument is usually risk based, and is intended to demonstrate that all relevant risks have been understood and dealt with sufficiently. The evidence can come from diverse safety management activities, such as hazard and risk analyses, design specifications, testing and empirical evaluation. For complex settings, such evidence is rarely self-evident, and hence the argument helps explain, appraise and challenge the extent to which the evidence is able to support the safety claims.

There are over 1000 medical device standards, several hundred of which are used for regulatory purposes. Many of the standards cover horizontal issues, for example, electrical safety, that is, they cover one specific hazard. These standards contain requirements, which—when followed—are intended to demonstrate compliance with regulations. While standards such as ISO 14971 (Medical Devices—Application of Risk Management to Medical Devices) take a risk-based approach, the lack of regulatory expectation for providing an explicit argument for how the body of evidence meets the regulatory requirements can reduce transparency and weaken confidence. As an analogy, the safety case can be thought of as the discussion in a research paper, as it explains and critically appraises the safety-related evidence and reflects on the limitations of the safety evidence and the safety activities that produced the evidence.

Where are safety cases being used?

A review by the Health Foundation describes safety case practices across six industries: automotive, civil aviation, defence, nuclear, petrochemical and railways.5 Safety cases are used widely across these safety-critical industries, particularly in the UK (see a review10 for further details). The UK nuclear industry adopted safety cases in 1965, following the Windscale fire accident in 1957. Accidents were major drivers for the adoption of safety cases also in other industries, such as offshore oil and gas production (Piper Alpha oil platform explosion 1988) and railways (eg, King’s Cross escalator fire 1987; Clapham main line derailment 1988). In the automotive domain, the increased complexity of interconnected electronics and software components was reflected in a requirement for an automotive safety case specified in the international standard on automotive functional safety (ISO 26262).

In healthcare, the application of safety cases has been limited. In 2010 (draft version, then finalised in 2014), the Food and Drug Administration in the USA issued guidance to manufacturers of infusion pumps that recommends the use of an assurance (safety) case as part of the premarket notification 510(k) submission route. This was triggered by high numbers of reported incidents involving such devices. However, the impact on adverse event rates has not been evaluated since.

In England, NHS Digital issued two risk management standards for health information technology (IT), which specify safety assurance requirements and practices including the development of clinical safety cases for both manufactures and health organisations (referred to as DCB 0129 and DCB 0160, respectively). Although compliance with these requirements is mandated by NHS England, the standards are only enforced for systems that directly connect to the national infrastructure.

Why do safety-critical industries do safety cases?

The use of safety cases is usually part of a regulatory approach that is known as ‘goal-based’ as opposed to the more traditional prescriptive regulatory approach. Prescriptive regulation sets out in standards detailed requirements for which risks need to be controlled, and how. Such prescriptive standards are based on past experiences and work well for established and well-understood systems. However, in settings where there is a fast pace of technological innovation and change, prescriptive standards quickly become outdated and might even hinder innovation. Goal-based approaches are more flexible, because they only specify what needs to be achieved, but leave open how this is done. If there are applicable standards, which are deemed relevant, there is still the expectation that these are complied with. Otherwise, a good argument needs to be provided for why the standards are not followed.

Part of the regulatory requirements in a safety case regime is the duty to demonstrate that risks have been reduced as low as reasonably practicable (ALARP), or similar wording with the same intent. This means that operators of hazardous systems need to consider all reasonable ways of reducing risk, even if these are not prescribed in existing standards.

In the literature, a range of different reasons for why industries have adopted safety cases can be found.11 Among these are expectations that safety cases:

• Promote structured risk assessment and management.

• Tell the story of a system’s safety to a wider and diverse readership.

• Show how high-level safety requirements are implemented in the detailed design.

• Establish confidence in safety.

• Stimulate critical thought around safety.

• Explain safety evidence.

• Focus on regulatory inspection.

Do safety cases improve safety?

Even though safety cases have been used across diverse industries for many years, there is a lack of conclusive evidence that the use of safety cases improves outcomes.12 There are two reasons for this. First, safety cases are used traditionally for high-hazard settings, where the focus is on high-severity, low-frequency events, that is, the rare, but catastrophic failure of a system, such as the loss of an aircraft. Given the low frequency of such events, it is difficult to provide meaningful statistical data about the impact of a regulatory instrument, such as safety cases.13 Second, the practice of safety cases is very varied, and it is frequently not explicitly articulated what kinds of benefits safety cases might have (see above) and how these are achieved.12 Consequently, the adoption of safety cases is usually based on a face validity principle, that is, regulators and industry act on the assumption that it is a good idea to use safety cases.

Critics point to this lack of evidence as well as to the fact that high-profile accidents continue to happen in countries that require safety cases.14 15 A frequently used example is the catastrophic loss of a Royal Air Force Nimrod aircraft in Afghanistan in 2006. The independent Haddon-Cave review16 highlighted significant weaknesses in safety case practices as part of wider criticisms of poor risk management systems across the different organisations that were involved in the design, operation and assessment of the aircraft. Such a culture was found to undermine the intended value of a safety case leading to a ‘tick-box’ and compliance-driven approach to safety.

Making safety cases work in healthcare

In the NHS in England, the clinical safety case concept promoted by NHS Digital is suggested for wider use in the HSIB safety observation. However, bearing in mind the complexity and contested nature of safety case practices in safety-critical industries there is a danger that in healthcare the concept will be misunderstood, misused and ultimately fail to make care safer.

An Australian report identifies five key criteria for successful safety case regimes13: (1) an established risk or hazard management framework; (2) a legal requirement to make the case to the regulator; (3) a competent and independent regulator; (4) workforce involvement; and (5) a general duty of care imposed on the operator. It is clear that most health systems do not currently meet these success criteria, not least because much of the patient safety improvement work is driven by outcomes (reactive) rather than by consideration of risk in processes and systems (proactive), while regulators also do not provide incentives for reducing risk as such.17

Bearing in mind the differences between safety-critical industries and healthcare, the Health Foundation convened in 2013 a multiprofessional working group to investigate the potential use of safety cases in healthcare,18 and the findings remain highly relevant. The report suggested that the health sector might benefit from the use of safety cases because they provide a structure for proactively assessing risk, they can have a positive impact on safety culture and because they bring together and synthesise a range of information and evidence relating to a particular service. These benefits might best be realised when safety cases are used as part of service improvement or as part of an assurance process. For example, a review of clinical safety cases submitted to NHS Digital found that many organisations were struggling to define the functionality of health IT and how it integrates into their local clinical context.19 This is reflected by the HSIB report, which suggests that the investigated organisations lacked an understanding of how smart pump functionality might differ from current practice, who the users of the smart pumps were, how smart pumps would interface and interact with other IT systems and what risks might need to be addressed. Irrespective of regulatory requirements, the use of clinical safety cases could support organisations in considering more adequately the scope of change that comes with the adoption of digital health technologies, and making explicit their risk position so that risks do not go undetected or undocumented. This is illustrated in table 1 based on the reference investigations described in the HSIB report.4

However, in order to facilitate and achieve successful adoption of safety case practices in healthcare, suggestions for the use of clinical safety cases (as in the HSIB safety observations) need to be underpinned by additional work and changes to the (patient) safety management infrastructure:

• Evidence base: Researchers need to articulate the mechanisms by which safety cases can improve outcomes and build a persuasive evidence base about benefits and the conditions that create the most fertile ground for using safety cases. In this respect, healthcare might be better placed than other industries, because the rigorous evaluation of complex interventions has gained a lot of traction in recent years, and because (sadly) adverse events happen at a rate that is more amenable to statistical analysis.

• Capability: Safety experts and patient safety specialists need to identify the level of training and support that healthcare staff and regulators require in order to support and to implement a safety case approach. In England, bodies such as NHS Digital and Health Education England should consider how capability can be built at scale. NHS Digital offers courses, but these might not scale up, and there are few publicly available examples of clinical safety cases. Health Education England has developed the national patient safety syllabus, which includes consideration of proactive safety management and safety cases, but questions remain about how the syllabus can be implemented and delivered across the NHS. Internationally, the patient safety curriculum developed by the WHO could potentially be a vehicle, but does not currently include safety cases.

• Criteria for risk reduction: Health systems should develop and adopt a healthcare-specific notion of acceptable levels of risk, and a framework that can be used in the decision-making process about the management of risk. In safety-critical industries, decisions about risk reduction are based on the ALARP principle, but the health systems face different challenges, such as the duty to provide care to an ageing population with complex health needs while at the same time being bound by a budget set by the government. There is a need for a broader dialogue around the criteria based on which healthcare organisations should manage the trade-off between risk reduction and cost, and to inform their evaluation of whether services are acceptably safe.