Re: Is WHO’s surgical safety checklist being hyped?
Arvid Steinar Haugen1, Stig Harthug2,3, Nick Sevdalis4, Eirik Søfteland1,5
1Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway; 2Department of Research and Development, Haukeland University Hospital, Bergen, Norway; 3Department of Clinical Science, University of Bergen, Bergen, Norway; 4Centre for Implementation Science, Health Service & Population Research Department, King’s College London, United Kingdom; 5Department of Clinical Medicine, University of Bergen, Bergen, Norway.
It’s not miraculous, it’s a no-brainer
The scientific evidence of the World Health Organisation’s surgical safety checklist is not about pseudoscientific or paranormal phenomena as Urbach & Dimik imply. In all types of research, it is important not to over or under interpret your findings, one needs to be critical and sceptic when reviewing, but balanced and based on the best available methodologies possible to address the problem. In their scepticism and reasoning against studies that report positive checklist effects, there are however few traces of such a balance [1].
First, Urbach & Dimick refer to the one published randomized study of the checklists; a stepped wedge cluster randomized controlled trial that was carried out in our hospitals [2]. It is correct that we did not find an overall reduction of mortality, only a significant reduction of mortality in one of the hospitals. We also observed a significant reduction of morbidity. Urbach & Dimick’s statement that our study did not report the intention-to-treat group in the ‘control-arm’ is directly incorrect. The numbers are laid out in Table 2, with an absolute risk reduction of complications in the intention to treat group at 7.5 (95% confidence interval, 5.5 to 9.5), and even higher absolute risk reduction when using all three parts of the checklist 8.4 (95% confidence interval, 6.3 to 10.5) [2]. Their statement seems not balanced on this point.
Second, they state that there was a lack of blinding in our study. Well, it is indeed true that the healthcare staff was not blinded for using the checklist, but no checklist study could possibly be designed for that. Using a cluster RCT design is more robust than a pre- and post-study [3], but labelling this as pseudoscience by Urbach & Dimick presents itself more as ‘click bate’ reasoning than a scientific balanced viewpoint. In fact, as far as possible, healthcare workers performing the checklist were in our study blinded for the study outcomes, data assessors were blinded for the checklist use and post-operative staff was blinded for the use of checklist and outcome [2].
Third, why did we find such a large effect on complications when the checklist had been used? We followed Avedis Donabedian’s hypothesis for quality improvement: structure improves process, and improved processes (and structure) provide better outcome. In a causal analysis of the checklist impact on care processes and patient outcomes, we found that the checklist improved patient care with more use of warm fluids, forced air warming blankets, and better timing of antibiotics. These improved care processes were again associated directly with improved patient outcome [4]. The checklist effect on complications is not caused by magic or a miraculous phenomena, it’s a no-brainer. The effect is caused by the actions the checklist ask for. If the checklist does not induce change in actions or improvement of care, better outcomes could not be obtained [5].
We support the statement from Urbach & Dimick that checklists have real and meaningful benefits and strongly encourage the use of them, which requires engagement from perioperative nurses, surgeons, and anaesthetic personnel, and not at least managers. In line with Haynes & Gawande's points, we endorse having focus on implementation approaches and team communication.
References
1. Urbach DR, Dimick JB, Haynes AB, et al. Is WHO’s surgical safety checklist being hyped? British Medical Journal 2019;366:l4700. doi: 10.1136/bmj.l4700
2. Haugen AS, Søfteland E, Almeland SK, et al. Effect of the World Health Organization Checklist on Patient Outcomes: A Stepped Wedge Cluster Randomized Controlled Trial. Annals of Surgery 2015;261(5):821-28. doi: 10.1097/sla.0000000000000716
3. Hemming K, Haines TP, Chilton PJ, et al. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. British Medical Journal 2015;350:h391. doi: 10.1136/bmj.h391
4. Haugen AS, Wæhle HV, Almeland SK, et al. Causal Analysis of World Health Organization's Surgical Safety Checklist Implementation Quality and Impact on Care Processes and Patient Outcomes: Secondary Analysis From a Large Stepped Wedge Cluster Randomized Controlled Trial in Norway. 2019;269(2):283-90. doi: 10.1097/sla.0000000000002584
5. Leape LL. The Checklist Conundrum. N Engl J Med 2014;370(11):1063-64. doi: doi:10.1056/NEJMe1315851
Competing interests:
Arvid Steinar Haugen received postdoctoral grant from the Western Norwegian Regional Health Authority with grant number: HV1172. He represent the International Federation of Nurse Anaesthetists in the European Society of Anaesthesiologists' Patient Safety & Quality Committee. Nick Sevdalis research is funded by the NIHR via the ‘Collaboration for Leadership in Applied Health Research and Care South London’ at King's College Hospital NHS Foundation Trust, London, UK. NS is also a member of King’s Improvement Science, which is part of the NIHR CLAHRC South London and comprises a specialist team of improvement scientists and senior researchers based at King’s College London. Its work is funded by King’s Health Partners (Guy’s and St Thomas’ NHS Foundation Trust, King’s College Hospital NHS Foundation Trust, King’s College London and South London and Maudsley NHS Foundation Trust), Guy’s and St Thomas’ Charity, the Maudsley Charity and the Health Foundation. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. SH had a role as lead of the Scientific Advisory Board appointed by the Norwegian Directorate of Health 2011-2013. The Directorate had no role in planning the design, conduct, or analysis of this study. The funders had no role in the design, conduct, or analysis of this study. NS is also the Director of London Safety and Training Solutions Ltd, which provides quality and safety training and advisory services on a consultancy basis to healthcare organization globally. The other authors report no conflicts of interest.
13 August 2019
Arvid Steinar Haugen
Post-doctor and Head of Research and Development Section, Department of Anaesthesia and Intensive Care
Rapid Response:
It’s not miraculous, it’s a no-brainer
Re: Is WHO’s surgical safety checklist being hyped?
Arvid Steinar Haugen1, Stig Harthug2,3, Nick Sevdalis4, Eirik Søfteland1,5
1Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway; 2Department of Research and Development, Haukeland University Hospital, Bergen, Norway; 3Department of Clinical Science, University of Bergen, Bergen, Norway; 4Centre for Implementation Science, Health Service & Population Research Department, King’s College London, United Kingdom; 5Department of Clinical Medicine, University of Bergen, Bergen, Norway.
It’s not miraculous, it’s a no-brainer
The scientific evidence of the World Health Organisation’s surgical safety checklist is not about pseudoscientific or paranormal phenomena as Urbach & Dimik imply. In all types of research, it is important not to over or under interpret your findings, one needs to be critical and sceptic when reviewing, but balanced and based on the best available methodologies possible to address the problem. In their scepticism and reasoning against studies that report positive checklist effects, there are however few traces of such a balance [1].
First, Urbach & Dimick refer to the one published randomized study of the checklists; a stepped wedge cluster randomized controlled trial that was carried out in our hospitals [2]. It is correct that we did not find an overall reduction of mortality, only a significant reduction of mortality in one of the hospitals. We also observed a significant reduction of morbidity. Urbach & Dimick’s statement that our study did not report the intention-to-treat group in the ‘control-arm’ is directly incorrect. The numbers are laid out in Table 2, with an absolute risk reduction of complications in the intention to treat group at 7.5 (95% confidence interval, 5.5 to 9.5), and even higher absolute risk reduction when using all three parts of the checklist 8.4 (95% confidence interval, 6.3 to 10.5) [2]. Their statement seems not balanced on this point.
Second, they state that there was a lack of blinding in our study. Well, it is indeed true that the healthcare staff was not blinded for using the checklist, but no checklist study could possibly be designed for that. Using a cluster RCT design is more robust than a pre- and post-study [3], but labelling this as pseudoscience by Urbach & Dimick presents itself more as ‘click bate’ reasoning than a scientific balanced viewpoint. In fact, as far as possible, healthcare workers performing the checklist were in our study blinded for the study outcomes, data assessors were blinded for the checklist use and post-operative staff was blinded for the use of checklist and outcome [2].
Third, why did we find such a large effect on complications when the checklist had been used? We followed Avedis Donabedian’s hypothesis for quality improvement: structure improves process, and improved processes (and structure) provide better outcome. In a causal analysis of the checklist impact on care processes and patient outcomes, we found that the checklist improved patient care with more use of warm fluids, forced air warming blankets, and better timing of antibiotics. These improved care processes were again associated directly with improved patient outcome [4]. The checklist effect on complications is not caused by magic or a miraculous phenomena, it’s a no-brainer. The effect is caused by the actions the checklist ask for. If the checklist does not induce change in actions or improvement of care, better outcomes could not be obtained [5].
We support the statement from Urbach & Dimick that checklists have real and meaningful benefits and strongly encourage the use of them, which requires engagement from perioperative nurses, surgeons, and anaesthetic personnel, and not at least managers. In line with Haynes & Gawande's points, we endorse having focus on implementation approaches and team communication.
References
1. Urbach DR, Dimick JB, Haynes AB, et al. Is WHO’s surgical safety checklist being hyped? British Medical Journal 2019;366:l4700. doi: 10.1136/bmj.l4700
2. Haugen AS, Søfteland E, Almeland SK, et al. Effect of the World Health Organization Checklist on Patient Outcomes: A Stepped Wedge Cluster Randomized Controlled Trial. Annals of Surgery 2015;261(5):821-28. doi: 10.1097/sla.0000000000000716
3. Hemming K, Haines TP, Chilton PJ, et al. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. British Medical Journal 2015;350:h391. doi: 10.1136/bmj.h391
4. Haugen AS, Wæhle HV, Almeland SK, et al. Causal Analysis of World Health Organization's Surgical Safety Checklist Implementation Quality and Impact on Care Processes and Patient Outcomes: Secondary Analysis From a Large Stepped Wedge Cluster Randomized Controlled Trial in Norway. 2019;269(2):283-90. doi: 10.1097/sla.0000000000002584
5. Leape LL. The Checklist Conundrum. N Engl J Med 2014;370(11):1063-64. doi: doi:10.1056/NEJMe1315851
Competing interests: Arvid Steinar Haugen received postdoctoral grant from the Western Norwegian Regional Health Authority with grant number: HV1172. He represent the International Federation of Nurse Anaesthetists in the European Society of Anaesthesiologists' Patient Safety & Quality Committee. Nick Sevdalis research is funded by the NIHR via the ‘Collaboration for Leadership in Applied Health Research and Care South London’ at King's College Hospital NHS Foundation Trust, London, UK. NS is also a member of King’s Improvement Science, which is part of the NIHR CLAHRC South London and comprises a specialist team of improvement scientists and senior researchers based at King’s College London. Its work is funded by King’s Health Partners (Guy’s and St Thomas’ NHS Foundation Trust, King’s College Hospital NHS Foundation Trust, King’s College London and South London and Maudsley NHS Foundation Trust), Guy’s and St Thomas’ Charity, the Maudsley Charity and the Health Foundation. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. SH had a role as lead of the Scientific Advisory Board appointed by the Norwegian Directorate of Health 2011-2013. The Directorate had no role in planning the design, conduct, or analysis of this study. The funders had no role in the design, conduct, or analysis of this study. NS is also the Director of London Safety and Training Solutions Ltd, which provides quality and safety training and advisory services on a consultancy basis to healthcare organization globally. The other authors report no conflicts of interest.