Education as a low-value improvement intervention: often necessary but rarely sufficient
========================================================================================

* Christine Soong
* Kaveh G Shojania

*   medication safety
*   quality improvement
*   educational outreach
*   academic detialing

Since the launch of Choosing Wisely in the United States,1 efforts to raise awareness about avoiding low-value care have spread internationally,2 prompting numerous commentaries,3–7 descriptive studies and improvement interventions,8–10 as well as inspiring new hospital job descriptions (eg, Chief Value Officer), journal sections11 and conferences devoted to the ‘Less is More’ paradigm. Low-value clinical care refers to services or interventions that provide little to no benefit to patients in specific clinical scenarios, may cause harm and/or incur unnecessary cost.6 12 13 

One example of a commonly encountered low-value practice is the continuation of proton pump inhibitors (PPIs) in patients without indication for ongoing use. Following completion of a defined period of therapy for appropriate indications (eg, peptic ulcer disease), continued use of PPIs provides little value, yet de-prescribing occurs infrequently. Moreover, this low-value use unnecessarily exposes patients to associated PPI-related adverse events such as pneumonia and *Clostridioides difficile* infections.14 15 Like many other areas of low-value care, PPI deprescribing is the focus of numerous quality improvement interventions.16–18 

In this issue of *BMJ Quality and Safety*, Bruno and colleagues examined the impact of a national educational intervention aimed at reducing outpatient PPI prescriptions in Australia.19 Australia’s NPS MedicineWise (previously the National Prescribing Service) developed the study intervention released in association with Choosing Wisely Australia’s similar educational materials and alerts highlighting the importance of reducing or ceasing PPI prescribing in the absence of specific indications. The programme began with mailings to general practitioners (GPs) showing data about their own PPI prescribing compared with other GPs nationwide. The remainder of the programme consisted of educational interventions, which the authors describe as a mixture of ‘passive’ and ‘active’ components. Passive components of the intervention included online educational resources, mailed evidenced-practice summary sheets and other informational materials.

Active components of the educational programme reported by Bruno *et al* included: an online self-audit tool allowing GPs to upload information pertaining to 10 of their patients and receive immediate and dynamic patient-specific recommendations; and interactive clinical scenarios related to PPI therapy, with feedback and expert commentary. Uptake of active components was not monitored. As an incentive to complete these educational activities, GPs could receive continuing professional development points.

Over the course of the study period, the investigators observed a small (1.7%) decrease in dispensings of PPIs, without a reduction in statins which represented a control comparator. Yet, the authors found no significant changes in the monthly rate of PPI discontinuation or dose reductions—the main outcomes targeted by the intervention. As the authors noted, one aspect of the Australian context which may have in part limited the intervention’s impact is that the publicly subsidised drug system does not cover the lower strength formulation for esomeprazole, the most commonly prescribed PPI in Australia. Still, as the authors also acknowledged, ‘educational initiatives working alone are unlikely to make the inroads required to curb overuse of PPIs’.

## The limited impact of education as an improvement intervention

Numerous studies have reached similar conclusions—that relying on educational interventions to change clinicians’ behaviours tends to produce no improvement, making this category of interventions the most predictably disappointing among improvement efforts. For instance, two systematic reviews examining the effectiveness of quality improvement interventions involving outpatient diabetes management analysed the relative effectiveness of different categories of intervention components (eg, case management, team changes, audit and feedback and clinician education).20 21 When compared with other intervention types, clinician education had negligible impact on glycaemic control. Other systematic reviews have demonstrated similar findings of little to no improvements when examining the impact of education on physician behaviour and clinical outcomes.22 23 Recognising this reality, the Infectious Diseases Society of America’s guideline on implementing antimicrobial stewardship explicitly recommends ‘against relying solely on didactic educational materials’ as a strategy to reduce antimicrobial overuse.24 

In the classic ‘hierarchy of effectiveness’ often shown in human factors engineering (figure 1), education ranks as the least effective intervention, right below new rules and policies and far below more system focused categories such as forcing functions and automation.20 21 25 26 Passive educational activities, such as didactic sessions, online modules and delivery of informational materials, produce particularly low impact. Active educational strategies such as educational outreach to clinicians (similar to the ‘academic detailing’ undertaken by pharmaceutical representatives) can achieve better results with improvements comparable to those of audit and feedback and computerised decision support.26 While active educational interventions tend to produce greater improvements than do passive ones, they also require greater investments of personnel time, hence their lower uptake as improvement strategies.

![Figure 1](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/29/5/353/F1.medium.gif)

[Figure 1](http://qualitysafety.bmj.com/content/29/5/353/F1)

Figure 1 
The hierarchy of intervention effectiveness (Adapted from the Institute for Safe Medication Practices25 and Patientsafe Implementing effective safety solutions.43 

As a sole strategy, education rarely results in sustained behaviour change, earning it a ‛necessary but insufficient’ status among improvement interventions. A classic adage in quality improvement recommends making the right thing to do the easy thing to do. Education never achieves this. Just as clinical practices can have low value, so can improvement interventions. Passive educational interventions, such as lectures and educational handout materials, often fall into this category. If educational interventions consistently delivered small to moderate improvements, they would have moderate value since they cost so little to deliver. Unfortunately, educational interventions often achieve minimal to zero improvement while requiring at least some resources to implement, hence our characterisation of education as having low value as an improvement strategy.

Below, we describe common scenarios in which over-reliance on low-value educational interventions fails to produce results.

## Scenario #1: education applied to problems that do not involve knowledge deficits

When a nurse or physician forgets to check for allergies before administering or prescribing a medication, this oversight usually represents a ‘slip’, not a conscious mistake reflecting a lack of knowledge. Consequently, education about the importance of asking patients about allergies will achieve little to no reduction in the frequency with which clinicians forget to check for allergies. A successful intervention would prompt clinicians to check for allergies at the time of entering a medication order or administering a medication—as with alerts in electronic prescribing or bar-coded medication administration systems. These systems are problematic in other ways,27 28 but they at least address the correct underlying problem.

Hand hygiene provides another example. Few clinicians do not know the recommendation to perform hand hygiene to prevent spread of infections. But, hand hygiene has not become an ingrained habit for most healthcare workers.29 Some investigators have looked at ways to foster the development of this habit.30 But, the need for hand hygiene arises so often throughout a given day that slips can constitute a problem even for clinicians with the habit. Thus, effective educational interventions need to focus on appropriate knowledge targets (eg, common misconceptions about when hand hygiene might not be needed) and must be accompanied by interventions involving cues to remind clinicians as they enter and exit patient rooms, as well as attention to convenient placement of sinks and hand hygiene dispensers.

Forgetting to apply knowledge may also occur as a result of distraction in the midst of other considerations during a given patient encounter. For instance, a GP seeing a patient with diabetes may focus on the patient’s glycaemic control, the need for referral to an eye specialist and other aspects of chronic disease management, but forget to discuss discontinuation of the PPI the patient has been taking for several years.

Various researchers have highlighted the importance of ‘having a theory’ for an improvement intervention—a clearly articulated mechanism for how a proposed intervention addresses the main causes of a target quality problem.31–33 Too often an educational intervention is chosen without a plausible theory for lack of knowledge as the main cause of a quality problem. And, even when aspect of knowledge does play a role, a compelling theory for solving the problem on the basis of education alone seldom exists.

## Scenario #2: education makes sense in principle but requires too frequent repetition

Even when a target quality problem clearly involves a lack of knowledge or skills, the opportunity to apply successfully acquired educational content may occur infrequently. In such circumstances, initially successful acquisition of the requisite knowledge or skills may erode over time. For this reason, an intervention to help emergency physicians with a rarely performed but potentially life-saving intervention used a ‘just-in-time’ educational video.34 This video included a brief (30 s) refresher with audio narration of the key steps in the procedure, followed by a step-by-step interactive checklist for performing the procedure. Delivering this information in a lecture or other passive educational intervention would almost certainly have achieved no improvement, as clinicians would simply forget what they had learnt by the time (months or years later) they had to perform the procedure.

Repeated delivery to sustain education directed at uncommon situations clearly represents a low-value proposition. But, education can have low-value even when the subject of educational interventions involves more common situations because staff turnover necessitates periodic re-delivery of the same education. This represents a particular problem in teaching hospitals, where trainees deliver much frontline yet rotate in and out of units and clinics on a monthly (sometimes weekly) basis.

## Scenario #3: other factors impede application of the requisite knowledge or skills

Educating clinicians about the lack of benefit from, say, prescribing antibiotics for the common cold35 does not make it any easier to dissuade a patient who came to the clinic desiring precisely this outcome from the visit.36–39 Similarly, education delivered to clinicians about recommendations against routine screening for prostate cancer in men over 75 years of age40 does not equip clinicians with the materials or communication techniques likely to reassure patients interested in such screening.

System factors may also thwart the objectives of educational interventions. A GP may have taken on board the educational message of a Choosing Wisely initiative not to order advanced imaging for patients who have low back pain without any high-risk features.9 But, she might also know that the only way to obtain a timely consultation with a spine surgeon is to have an MRI available for review. A patient admitted to the hospital with heart failure might have had an echocardiogram 6 months ago at an outpatient facility, but it seems more expedient for the inpatient medical team to order another echocardiogram to have it handy right away.

In teaching hospitals, there is also the issue of ordering tests to show that one knows what one is doing—a trainee may order a number of tests unlikely to show anything useful for the patient but do serve to demonstrate to the attending physician that the trainee has considered an appropriately broad differential diagnosis.41 And, of course, financial incentives can drive practice, rewarding clinicians for the volume of care delivered rather than the health outcomes achieved or consistency with best practice.

## Conclusion

Passive educational methods such as lectures and distributing informational materials are frequently misapplied to address quality problems that do not primarily reflect a knowledge gap. Even in situations of known deficits in knowledge and/or skills, other systems factors diminish the effectiveness of educational interventions. Education can support improvement interventions by engaging clinicians or familiarising them with the justification for an intervention. But education on its own, especially passive education, typically delivers little value as a change strategy. Admittedly, education can also serve the purpose of ‘raising awareness,’ as has been the case with educating patients and providers about ‘low-value care’,2 42 such as deprescribing PPIs and other Choosing Wisely targets. But achieving worthwhile impact requires designing high-value improvement interventions featuring more effective systems-based changes. These higher value improvement strategies make ‘the right thing to do the easy thing to do’ and include education only when it has a clear role to play.

## Footnotes

*   Twitter @christinesoong

*   Contributors All authors contributed equally to the design, concept and drafting of the manuscript.

*   Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

*   Competing interests None declared.

*   Patient consent for publication Not required.

*   Provenance and peer review Commissioned; internally peer reviewed.

## References

1.   Cassel CK , Guest JA . Choosing wisely: helping physicians and patients make smart decisions about their care. JAMA 2012;307:1801–2.[doi:10.1001/jama.2012.476](http://dx.doi.org/10.1001/jama.2012.476) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.2012.476&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22492759&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000303386800016&link_type=ISI) 

2.   Levinson W , Kallewaard M , Bhatia RS , et al . ‘Choosing Wisely’: a growing international campaign. BMJ Qual Saf 2015;24:167–74.[doi:10.1136/bmjqs-2014-003821](http://dx.doi.org/10.1136/bmjqs-2014-003821) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI0LzIvMTY3IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.   Marcotte LM , Schuttner L , Liao JM . Measuring low-value care: learning from the US experience measuring quality. BMJ Qual Saf 2020;29:154–6.[doi:10.1136/bmjqs-2019-010191](http://dx.doi.org/10.1136/bmjqs-2019-010191) pmid:http://www.ncbi.nlm.nih.gov/pubmed/31649163 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI5LzIvMTU0IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.   Moriates C , Valencia V . Emerging principles for health system value improvement programmes. BMJ Qual Saf 2019;28:434–7.[doi:10.1136/bmjqs-2019-009427](http://dx.doi.org/10.1136/bmjqs-2019-009427) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI4LzYvNDM0IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

5.   Mafi JN , Parchman M . Low-value care: an intractable global problem with no quick fix. BMJ Qual Saf 2018;27:333–6.[doi:10.1136/bmjqs-2017-007477](http://dx.doi.org/10.1136/bmjqs-2017-007477) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI3LzUvMzMzIjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

6.   Chalmers K , Pearson S-A , Elshaug AG . Quantifying low-value care: a patient-centric versus service-centric lens. BMJ Qual Saf 2017;26:855–8.[doi:10.1136/bmjqs-2017-006678](http://dx.doi.org/10.1136/bmjqs-2017-006678) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjk6IjI2LzEwLzg1NSI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzUvMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.   Willson A . The problem with eliminating ‘low-value care’. BMJ Qual Saf 2015;24:611–4.[doi:10.1136/bmjqs-2015-004518](http://dx.doi.org/10.1136/bmjqs-2015-004518) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjk6IjI0LzEwLzYxMSI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzUvMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

8.   Mafi JN , Godoy-Travieso P , Wei E , et al . Evaluation of an intervention to reduce low-value preoperative care for patients undergoing cataract surgery at a safety-net health system. JAMA Intern Med 2019;179:648–57.[doi:10.1001/jamainternmed.2018.8358](http://dx.doi.org/10.1001/jamainternmed.2018.8358) 
    
    

9.   Kullgren JT , Krupka E , Schachter A , et al . Precommitting to choose wisely about low-value services: a stepped wedge cluster randomised trial. BMJ Qual Saf 2018;27:355–64.[doi:10.1136/bmjqs-2017-006699](http://dx.doi.org/10.1136/bmjqs-2017-006699) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI3LzUvMzU1IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10.  Chatfield SC , Volpicelli FM , Adler NM , et al . Bending the cost curve: time series analysis of a value transformation programme at an academic medical centre. BMJ Qual Saf 2019;28:449–58.[doi:10.1136/bmjqs-2018-009068](http://dx.doi.org/10.1136/bmjqs-2018-009068) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI4LzYvNDQ5IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11.  Caverly TJ , Combs BP , Moriates C , et al . Too much medicine happens too often: the teachable moment and a call for manuscripts from clinical trainees. JAMA Intern Med 2014;174:8–9.[doi:10.1001/jamainternmed.2013.9967](http://dx.doi.org/10.1001/jamainternmed.2013.9967) 
    
    

12.  McAlister FA , Shojania KG . Inpatient bedspacing: could a common response to hospital crowding cause increased patient mortality? BMJ Qual Saf 2018;27:1–3.[doi:10.1136/bmjqs-2017-007524](http://dx.doi.org/10.1136/bmjqs-2017-007524) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjY6IjI3LzEvMSI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzUvMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13.  Badgery-Parker T , Pearson S-A , Chalmers K , et al . Low-value care in Australian public hospitals: prevalence and trends over time. BMJ Qual Saf 2019;28:205–14.[doi:10.1136/bmjqs-2018-008338](http://dx.doi.org/10.1136/bmjqs-2018-008338) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI4LzMvMjA1IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14.  Islam MM , Poly TN , Walther BA , et al . Adverse outcomes of long-term use of proton pump inhibitors: a systematic review and meta-analysis. Eur J Gastroenterol Hepatol 2018;30:1395–405.[doi:10.1097/MEG.0000000000001198](http://dx.doi.org/10.1097/MEG.0000000000001198) 
    
    

15.  Dial S , Delaney JAC , Barkun AN , et al . Use of gastric acid-suppressive agents and the risk of community-acquired Clostridium difficile-associated disease. JAMA 2005;294:2989–95.[doi:10.1001/jama.294.23.2989](http://dx.doi.org/10.1001/jama.294.23.2989) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.294.23.2989&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=16414946&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000234087700025&link_type=ISI) 

16.  Murie J , Allen J , Simmonds R , et al . Glad you brought it up: a patient-centred programme to reduce proton-pump inhibitor prescribing in general medical practice. Qual Prim Care 2012;20:141–8.
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22824567&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 

17.  Walsh K , Kwan D , Marr P , et al . Deprescribing in a family health team: a study of chronic proton pump inhibitor use. J Prim Health Care 2016;8:164–71.[doi:10.1071/HC15946](http://dx.doi.org/10.1071/HC15946) 
    
    

18.  Hamzat H , Sun H , Ford JC , et al . Inappropriate prescribing of proton pump inhibitors in older patients. Drugs Aging 2012;29:681–90.[doi:10.1007/BF03262283](http://dx.doi.org/10.1007/BF03262283) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2165/11632700-000000000-00000&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22775478&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 

19.  Bruno C , Pearson S-A , Daniels B , et al . Passing the acid test? evaluating the impact of national education initiatives to reduce proton pump inhibitor use in Australia. BMJ Qual Saf 2020;29:365–73.[doi:10.1136/bmjqs-2019-009897](http://dx.doi.org/10.1136/bmjqs-2019-009897) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI5LzUvMzY1IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

20.  Shojania KG , Ranji SR , McDonald KM , et al . Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA 2006;296:427–40.[doi:10.1001/jama.296.4.427](http://dx.doi.org/10.1001/jama.296.4.427) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.296.4.427&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=16868301&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000239242500029&link_type=ISI) 

21.  Tricco AC , Ivers NM , Grimshaw JM , et al . Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet 2012;379:2252–61.[doi:10.1016/S0140-6736(12)60480-2](http://dx.doi.org/10.1016/S0140-6736(12)60480-2) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(12)60480-2&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22683130&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000305511400032&link_type=ISI) 

22.  Grimshaw JM , Shirran L , Thomas R , et al . Changing provider behavior: an overview of systematic reviews of interventions. Med Care 2001;39:II2–45.
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1097/00005650-200108002-00002&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=11583120&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000170313600002&link_type=ISI) 

23.  Grimshaw J , Thomas R , MacLennan G , et al . Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:1–72.[doi:10.3310/hta8060](http://dx.doi.org/10.3310/hta8060) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=14960258&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 

24.  Barlam TF , Cosgrove SE , Abbo LM , et al . Implementing an antibiotic stewardship program: guidelines by the infectious diseases Society of America and the Society for healthcare epidemiology of America. Clin Infect Dis. 2016;62:e51–77.[doi:10.1093/cid/ciw118](http://dx.doi.org/10.1093/cid/ciw118) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1093/cid/ciw118&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=27080992&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 

25. Institute for Safe Medication Practices (ISMP). Medication Error Prevention “Toolbox”, 1999. Available: [https://www.ismp.org/resources/medication-error-prevention-toolbox](https://www.ismp.org/resources/medication-error-prevention-toolbox) 
    
    

26.  O'Brien MA , Rogers S , Jamtvedt G , et al . Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2007;308.[doi:10.1002/14651858.CD000409.pub2](http://dx.doi.org/10.1002/14651858.CD000409.pub2) 
    
    

27.  Slight SP , Beeler PE , Seger DL , et al . A cross-sectional observational study of high override rates of drug allergy alerts in inpatient and outpatient settings, and opportunities for improvement. BMJ Qual Saf 2017;26:217–25.[doi:10.1136/bmjqs-2015-004851](http://dx.doi.org/10.1136/bmjqs-2015-004851) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI2LzMvMjE3IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

28.  Wong A , Amato MG , Seger DL , et al . Prospective evaluation of medication-related clinical decision support over-rides in the intensive care unit. BMJ Qual Saf 2018;27:718–24.[doi:10.1136/bmjqs-2017-007531](http://dx.doi.org/10.1136/bmjqs-2017-007531) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI3LzkvNzE4IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29.  Vogus TJ , Hilligoss B . The underappreciated role of habit in highly reliable healthcare. BMJ Qual Saf 2016;25:141–6.[doi:10.1136/bmjqs-2015-004512](http://dx.doi.org/10.1136/bmjqs-2015-004512) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI1LzMvMTQxIjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

30.  Gilmartin H , Saint S , Rogers M , et al . Pilot randomised controlled trial to improve hand hygiene through mindful moments. BMJ Qual Saf 2018;27:799–806.[doi:10.1136/bmjqs-2017-007359](http://dx.doi.org/10.1136/bmjqs-2017-007359) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjk6IjI3LzEwLzc5OSI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzUvMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

31.  Davidoff F , Dixon-Woods M , Leviton L , et al . Demystifying theory and its use in improvement. BMJ Qual Saf 2015;24:228–38.[doi:10.1136/bmjqs-2014-003627](http://dx.doi.org/10.1136/bmjqs-2014-003627) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI0LzMvMjI4IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

32.  Reed JE , McNicholas C , Woodcock T , et al . Designing quality improvement initiatives: the action effect method, a structured approach to identifying and articulating programme theory. BMJ Qual Saf 2014;23:1040–8.[doi:10.1136/bmjqs-2014-003103](http://dx.doi.org/10.1136/bmjqs-2014-003103) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjEwOiIyMy8xMi8xMDQwIjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

33.  Foy R , Ovretveit J , Shekelle PG , et al . The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf 2011;20:453–9.[doi:10.1136/bmjqs.2010.047993](http://dx.doi.org/10.1136/bmjqs.2010.047993) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjIwLzUvNDUzIjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

34.  Branzetti JB , Adedipe AA , Gittinger MJ , et al . Randomised controlled trial to assess the effect of a Just-in-Time training on procedural performance: a proof-of-concept study to address procedural skill decay. BMJ Qual Saf 2017;26:881–91.[doi:10.1136/bmjqs-2017-006656](http://dx.doi.org/10.1136/bmjqs-2017-006656) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjk6IjI2LzExLzg4MSI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzUvMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

35. Choosing Wisely. Choosing wisely: infectious diseases Society of America list, 2015. Available: [https://www.choosingwisely.org/clinician-lists/infectious-diseases-society-antbiotics-for-upper-respiratory-infections/](https://www.choosingwisely.org/clinician-lists/infectious-diseases-society-antbiotics-for-upper-respiratory-infections/) [Accessed 25 Nov 2019].
    
    

36.  Courtenay M , Rowbotham S , Lim R , et al . Antibiotics for acute respiratory tract infections: a mixed-methods study of patient experiences of non-medical prescriber management. BMJ Open 2017;7:e013515.[doi:10.1136/bmjopen-2016-013515](http://dx.doi.org/10.1136/bmjopen-2016-013515) 
    
    

37.  Fletcher-Lartey S , Yee M , Gaarslev C , et al . Why do general practitioners prescribe antibiotics for upper respiratory tract infections to meet patient expectations: a mixed methods study. BMJ Open 2016;6:e012244.[doi:10.1136/bmjopen-2016-012244](http://dx.doi.org/10.1136/bmjopen-2016-012244) 
    
    

38.  Mustafa M , Wood F , Butler CC , et al . Managing expectations of antibiotics for upper respiratory tract infections: a qualitative study. Ann Fam Med 2014;12:29–36.[doi:10.1370/afm.1583](http://dx.doi.org/10.1370/afm.1583) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiMTIvMS8yOSI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzUvMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

39.  King LM , Fleming-Dutra KE , Hicks LA . Advances in optimizing the prescription of antibiotics in outpatient settings. BMJ 2018;22.[doi:10.1136/bmj.k3047](http://dx.doi.org/10.1136/bmj.k3047) 
    
    

40. Choosing Wisely. Choosing wisely: American Academy of family physicians 2018:513–23.
    
    

41.  Sedrak MS , Patel MS , Ziemba JB , et al . Residents' self-report on why they order perceived unnecessary inpatient laboratory tests. J Hosp Med 2016;11:869–72.[doi:10.1002/jhm.2645](http://dx.doi.org/10.1002/jhm.2645) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1002/jhm.2645&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=27520384&link_type=MED&atom=%2Fqhc%2F29%2F5%2F353.atom) 

42.  Born KB , Coulter A , Han A , et al . Engaging patients and the public in choosing wisely. BMJ Qual Saf 2017;26:687–91.[doi:10.1136/bmjqs-2017-006595](http://dx.doi.org/10.1136/bmjqs-2017-006595) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI2LzgvNjg3IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNS8zNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

43. Patientsafe implementing effective safety solutions. The hierarchy of intervention effectiveness, 2015. Available: [https://patientsafe.wordpress.com/the-hierarchy-of-intervention-effectiveness/](https://patientsafe.wordpress.com/the-hierarchy-of-intervention-effectiveness/) [Accessed 12 Nov 2019].