Multicentre trial to introduce the Ottawa ankle rules for use of radiography in acute ankle injuries
====================================================================================================

* Ian Stiell
* George Wells
* Andreas Laupacis
* Robert Brison
* Richard Verbeek
* Katherine Vandemheen
* C David Naylor

## Abstract

**Objective**: To assess the feasibility and impact of introducing the Ottawa ankle rules to a large number of physicians in a wide variety of hospital and community settings over a prolonged period of time.

**Design**: Multicentre before and after controlled clinical trial.

**Setting**: Emergency departments of eight teaching and community hospitals in Canadian communities (population 10000 to 3000000).

**Subjects**: All 12777 adults (6288 control, 6489 intervention) seen with acute ankle injuries during two 12 month periods before and after the intervention.

**Intervention**: More than 200 physicians of varying experience were taught to order radiography according to the Ottawa ankle rules.

**Main outcome measures**: Referral for ankle and foot radiography.

**Results**: There were significant reductions in use of ankle radiography at all eight hospitals and within a priori subgroups: for all hospitals combined 82.8% control v 60.9% intervention (P<0.001); for community hospitals 86.7% v 61.7%; (P<0.001); for teaching hospitals 77.9% v 59.9%; (P<0.001); for emergency physicians 82.1% v 61.6%; (P<0.001); for family physicians 84.3% v 60.1%; (P<0.001); and for housestaff 82.3% v 60.1%; (P<0.001). Compared with patients without fracture who had radiography during the intervention period those who had no radiography spent less time in the emergency department (54.0 v 86.9 minutes; P<0.001) and had lower medical charges ($70.20 v $161.60; P<0.001). There was no difference in the rate of fractures diagnosed after discharge from the emergency department (0.5 v 0.4%).

**Conclusions**: Introduction of the Ottawa ankle rules proved to be feasible in a large variety of hospital and community settings. Use of the rules over a prolonged period of time by many physicians of varying experience led to a decrease in ankle radiography, waiting times, and costs without an increased rate of missed fractures. The multiphase methodological approach used to develop and implement these rules may be applied to other clinical problems.

#### Key messages

*   Key messages

*   The Ottawa ankle rules can be successfully applied by physicians of varying experience in many different settings to reduce incidence of ankle radiography

*   With proper application of the rules the risk of patient dissatisfaction or missed fractures is negligible

*   Widespread use of the Ottawa ankle rules would lead to large savings in health care costs

*   The multiphase methodological approach used to develop, validate, and implement these decision rules may be applied to other clinical problems

## Introduction

Ankle and foot injuries are a common complaint among patients seen in emergency departments. Though only a few of these cases have suffered a fracture,1 2 3 4 5 6 nearly all typically undergo plain radiography of the ankle or foot, or both.7 8 9 10 11 12 To deal with this clinical problem decision rules for the use of radiography in acute ankle injuries have been recently developed13 14 and validated15 and have been found to be highly sensitive in identifying fractures. These Ottawa ankle rules are based on the assessment of ability to bear weight and areas of bone tenderness and allow physicians to determine quickly which patients are at negligible risk of fracture (figure).

![Figure1](https://www.bmj.com/https://www.bmj.com/content/bmj/311/7005/594/F1.medium.gif)

[Figure1](https://www.bmj.com/content/311/7005/594/F1)

Ottawa ankle rules for use of radiography in acute ankle injuries (adapted from Stiell et al18)

We know of few clinical decision rules that have been studied to determine their impact on patient care in “usual clinical practice.”16 17 A recent study at a single hospital showed that implementation of the Ottawa ankle rules led to a significant reduction in the use of ankle radiography.18 This clinical trial was designed to assess whether the rules could be shown to reduce the use of radiography without affecting quality of care when used by many physicians of varying experience in a variety of different hospital settings.

### Subjects and methods SUBJECTS

All adult patients with ankle injury seen in the emergency departments of the study hospitals during control (before) and intervention (after) periods of 12 months each were included in this controlled clinical trial. The eight hospitals were chosen because they were able to identify eligible cases retrospectively and because they represented various community population sizes (10000 to 3000000), hospital types (community, teaching), annual volumes of patients in emergency departments (20000 to 68000), and staffing patterns (emergency physician, family physician, house officers). Patients with acute ankle trauma from any mechanism of injury were eligible. “Ankle” was explicitly defined anatomically.15 19 The institutional research ethics committees approved the study.

### INTERVENTION

We introduced the Ottawa ankle rules15 to the study hospitals before the intervention period by means of a single one hour lecture given by the principal investigator (IS) as well as handouts, pocket cards, and two posters mounted in each department. Physicians were asked to complete a data form and to distribute information sheets to patients. The decision to order radiography for individual patients was solely at the discretion of the treating physician.

### OUTCOME MEASURES

For both study periods review of patients' eligibility was made from patients' records independently for each case by three members of the research team, and differences were resolved by consensus. Patients' records were identified (retrospectively for the control period and prospectively for the intervention period) in a consistent fashion by a combination of computerised retrieval from International Classification of Diseases (ninth revision) diagnostic codes and a search of daily census logs. The primary outcome measure, the proportion of patients referred for radiography of the ankle, was determined from radiology reports. Clinically important features were defined as fractures greater than three millimetres in breadth.

During the intervention period only, all patients without fracture were followed up by telephone call at 10 days if they had had neither ankle nor foot radiography or if they were seen during the first seven days of each month and did have radiography. Patients who had not improved according to explicit criteria regarding pain, ambulation, and ability to work were asked to return for assessment. Patients who had fractures diagnosed after discharge were questioned after six months about possible effects. For patients followed up by telephone calls the total charges for all emergency department and follow up physician visits and radiographic series were estimaed in 1993 US dollars.

### STATISTICAL ANALYSIS

All patients who met the inclusion-exclusion criteria during the control and intervention periods were included in the analysis regardless of whether physicians completed a data collection form or were compliant with the decision rules. For each hospital separately the uncorrected χ2 analysis was used to test the primary hypothesis that there was no difference in the proportion of patients referred for a standard ankle radiographic series between the control and intervention study groups. We calculated 95% confidence intervals of the relative differences in referral rates for radiography between groups.20 Overall point and confidence interval estimates for the relative reductions were also derived. A similar analysis was used to test the secondary hypothesis that there was no difference in the referral for radiographic series of the foot.

We used χ2 analysis to compare the primary outcome between the control and intervention study periods within the a priori subgroups of hospital type (community, teaching) and physician type (emergency, family, housestaff). Comparisons of characteristics of patients and other outcomes were tested with χ2, Student's t test, or Mann-Whitney test as appropriate.

An absolute change of 10% for the referral rate for ankle radiography from the estimated baseline rate of 80% was considered to be clinically important. We assumed a statistical power of 80% and a two tailed 5% type I error so a sample size of 293 patients was estimated for each hospital during each study period.

## Results

### SUBJECTS

The 12777 eligible patients seen at the eight hospitals during the control (1 May 1991 to 30 April 1992) and intervention (1 January to 31 December 1993) periods were similar for all characteristics (table I).

View this table:
[TABLE I](https://www.bmj.com/content/311/7005/594/T1)

TABLE I 
Characteristics of all patients with ankle injury seen at study hospitals during 12 month control and intervention study periods. Figures are numbers (percentages) of patients unless stated otherwise

### REDUCTION IN RADIOGRAPHY

The overall proportion of patients referred for ankle radiography was 82.8% (5207/6288) during the control period and 60.9% (3955/6489) during the intervention period (P<0.001), with an observed relative reduction between periods of 26.4% (table II). The proportion referred for ankle radiography was significantly less at all eight hospitals for both hospital subgroups and for each physician subgroup.

View this table:
[TABLE II](https://www.bmj.com/content/311/7005/594/T2)

TABLE II 
Referral for ankle radiographic series of all patients with ankle injury seen during the 12 months control and intervention study periods. Figures are numbers (percentages) of patients

Three hospitals had significant reductions and five had no significant change in referral for foot radiography. During the intervention period more patients did not have radiography (21.7% v 8.4%) and fewer patients had both ankle and foot series (9.2% v 21.8%). Furthermore, patients without fracture spent less time in the emergency department if they had no radiography compared with those who had radiography (54.0 (SD 42.0) v 86.9 (46.9) minutes; P<0.001).

### COMPLIANCE BY PHYSICIANS

During the intervention period physicians completed the study data sheet for 77.1% (5003) of the 6489 patients in the study. The physicians accurately interpreted the rules (97.1% for ankle and 97.6% for foot) and satisfactorily complied with the rules (95.3% for ankle and 95.0% for foot). Radiography was performed but judged to have been unnecessary according to the rules in 4.9% of cases but in only 0.5% of cases because the patient insisted on radiography. Physicians indicated that they were uncomfortable with implementing the rules in 3.8% of cases.

Six (0.6%) of the 1090 important malleolar and midfoot fractures during the intervention period were diagnosed before discharge from the emergency department in cases in which the rules were interpreted to be negative (table III). Physicians had ordered radiography based on other clinical findings, primarily gross swelling.

View this table:
[TABLE III](https://www.bmj.com/content/311/7005/594/T3)

TABLE III 
Characteristics of six cases (of 1090 important fractures) in which rules were interpreted as negative and important fractures were diagnosed before discharge from emergency department

### FOLLOW UP OF SUBJECTS

Of 2171 patients in the radiography and no radiography groups, 93.6% (2032) were successfully reached by telephone, and there were no important differences in outcomes (table IV). The mean total charges for those who had no radiography in the emergency department were less than for those who had radiography ($70.20 (SD 52.1) v $161.60 (71.2); P<0.001).

View this table:
[TABLE IV](https://www.bmj.com/content/311/7005/594/T4)

TABLE IV 
Follow up of 2033 patients with ankle injury but without fracture discharged with and without radiography during the intervention period. Figures are numbers (percentages) of patients unless stated otherwise

Ten (0.5%) of the 2033 patients in the follow up group had a fracture diagnosed after discharge from the emergency department despite no repeat injury (table V). Three (0.4%) of 732 had undergone radiography in the emergency department. Seven (0.5%) of 1301 had received no radiography, but in only one of these had the rules been correctly applied. All 10 patients were contacted after six months and all had healed without delay or long term effects. No litigation was initiated. No data about missed fractures were available for the control period.

View this table:
[TABLE V](https://www.bmj.com/content/311/7005/594/T5)

TABLE V 
Characteristics of 10 cases (of 2033 in follow up group) in which fracture was diagnosed after discharge from emergency department

## Discussion

We have shown that introduction of the Ottawa ankle rules led to significant reductions in the ordering of ankle radiographic series over a sustained period of time in a variety of community and hospital settings. These reductions were achieved by many physicians with differing experience in emergency medicine and who had not been involved in the development of the rules. Concomitant reductions in the use of the less common radiographic series of the foot were achieved in three hospitals. Physicians accurately interpreted the rules after briefteaching sessions and indicated low levels of discomfort with their application.21 Patients who did not undergo radiography were satisfied with their care and were no more likely to have a fracture missed in the emergency department than those who did undergo radiography.

The major benefits of introducing the Ottawa ankle rules are time savings for patients and cost savings for the health care system. Our data suggest the potential for large savings: the average medical charges for patients who had no radiography were estimated to be $90 less than for those patients without fracture who did undergo radiography. Ankle and foot radiographs are typical “little ticket” items,22 23 the many minor but high volume procedures which may collectively contribute as much to health care costs as “big ticket” items such as magnetic resonance imaging scans or coronary bypass surgery. In these eight hospitals alone more than 6000 patients with ankle injury were seen each year, so even a 25% reduction in radiography would translate into considerable savings.

This study represents the final phase of a multiphase project to develop and test the Ottawa ankle rules. To our knowledge very few other decision rules have been shown to alter clinical practice.17 The success of the rules may be attributed to the rigour with which they were derived and tested16 as well as to their clinical sensibility24--that is, their ease of use and their high sensitivity. The rules are easy to remember and present a simple “radiograph or no radiograph” decision which may be easier for busy clinicians to incorporate into their practice than a probability of fracture. Future studies should deal with the acceptability of the rules to physicians in other countries25 26 and methods of disseminating information about their use.27 28 29 30 31

We have developed rules with a sensitivity approaching 100%. Patients and physicians alike can be reassured that if the Ottawa ankle rules are properly applied and interpreted the chances of missing a clinically important fracture are remote. Fractures were diagnosed after discharge equally often in those who did and did not have radiography. In the former group, fractures were not diagnosed because either the physician misinterpreted the radiographs or the radiographs did not show a fracture on the first visit. In the group who did not have radiography, fractures were missed because the rules were not used or were misinterpreted and in one case because of gross swelling (table V). None of these patients suffered delayed healing or added morbidity.32 The most common errors in interpretation of the rules were disregarding the presence of medial malleolar tenderness or failing to palpate the entire distal 6 cm of the posterior edge of the fibula. The latter is especially important because some fractures exit posteriorly 5 or 6 cm proximal to the tip of the fibula.

The rules may be unreliable in cases when clinical assessment is difficult--for example, with altered mentation, intoxication, other painful injuries, diminished sensation in the lower extremities, or a language barrier. In this study physicians thought that the rules were unreliable in a few patients in whom gross swelling made palpation of the posterior edge of the malleolus impossible (table IV). Whether or not radiography is ordered patients must always be advised to seek follow up if their pain or ability to bear weight has not improved in five to seven days.

Our data indicate that patients with ankle injury are satisfied with care that does not include radiography. Integral to this satisfaction is adequate communication from the physician and use of printed instructions. If the physician has carefully applied the rules, documented the clinical findings, used good judgment, and provided advice on follow up the risk of patient dissatisfaction33 34 35 or litigation is extemely low even in the unlikely event of a missed fracture.

This study showed that the Ottawa ankle rules can be successfully applied by many different physicians in many settings to achieve considerable reductions in ankle radiography, waiting times, and health care costs without an increase in patient dissatisfaction or the rate of missed fractures. The multiphase methodological approach used to develop, validate, and implement these decision rules may be applied to other clinical problems.

We acknowledge the support and cooperation of the following investigators (MARS Study Group): Steven McMurray, Brockville General Hospital, Brockville; Alan Drummond, Peter Jechel, Great War Memorial Hospital, Perth; Terry O'Brien, Kingston General and Hotel Dieu Hospitals, Kingston; Edward Brown, Keith Greenway, Peel Memorial Hospital, Brampton; Wayne Helmer, Roger Hoag, Queensway-Carleton Hospital, Nepean; Raphael Shew, Smiths Falls Community Hospital, Smiths Falls; Andrew McDonald, Sunnybrook Health Science Center, Toronto, Ontario; Gary Greenberg, Ottawa, Ontario; Douglas McKnight, Vancouver, British Columbia.

## Acknowledgments

We thank the many physicians at the participating hospitals for their cooperation in completing data forms and emergency department nursing and clerical staff for their help with the study.

## Acknowledgments

We also thank the following MARS study staff members: Susan Delong, Sandra deMunnik, Mary McCarthy, and Helen Niezgoda (site research assistants); Fiona Campbell-Daigle, My-Linh Tran, and Elizabeth Yetisir (data management); Anne Gray and Jane Lockhart (administrative support).

## Footnotes

*   Funding nstitute for Clinical Evaluative Sciences in Ontario, Toronto, Canada.

*   Conflict of interest he institute is cosponsored by the Ontario Ministry of Health and the Ontario Medical Association. IS and CDN are career scientists of the Ontario Ministry of Health. Endorsement by the supporting or sponsoring agencies is not implied.

## References

1.  Stother IG. Incidence of minor fractures in twisting injuries of the ankle. Injury 1974; 5: 213–4. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1016/S0020-1383(74)80007-0&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=4208065&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1974T024000007&link_type=ISI) 

2.  Brooks SC, Potter BT, Rainey JB. Inversion injuries of the ankle: clinical assessment and radiographic review. BMJ 1981; 282: 607–8. 
    
    

3.  Vargish T, Clarke WR, Young RA, Jensen A. The ankle injury: indications for the selective use of x-rays. Injury 1983; 14: 507–12. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1016/0020-1383(83)90053-0&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=6409805&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1983QT73900004&link_type=ISI) 

4.  Montague AP, McQuillan RF. Clinical assessment of apparently sprained ankle and detection of fracture. Injury 1985; 16: 545–6. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1016/0020-1383(85)90082-8&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=4066016&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1985AQJ4800007&link_type=ISI) 

5.  Sujitkumar P, Hadfield JM, Yates DW. Sprain or fracture:? An analysis of 2000 ankle injuries. Arch Emerg Med 1986; 3: 101–6. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1136/emj.3.2.101&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=3089238&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1986D144100002&link_type=ISI) 

6.  Diehr P, Highley R, Dehkordi F, Wood R, Krueger L, Tietz C, et al. Prediction of fracture in patients with acute musculoskeletal ankle trauma. Med Decis Making 1988; 8: 40–7. 
    
    

7.  Brand DA, Frazier WH, Kohlhepp WC, Shea KM, Hoefer AM, Ecker MD, et al. A protocol for selecting patients with injured extremities who need x-rays. N Engl J Med 1982; 306: 333–9. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=7054709&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1982NA39400004&link_type=ISI) 

8.  Dunlop MG, Beattie TF, White GK, Raab GM, Doull RI. Guidelines for selective radiological assessment of inversion ankle injuries. BMJ 1986; 293: 603–5. 
    
    

9.  Lloyd S. Selective radiographic assessment of acute ankle injuries in the emergency department: barriers to implementation. Can Med Assoc J 1986; 135: 973–4. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=3093044&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 

10. Gleadhill DNS, Thomson JY, Simms P. Can more efficient use be made of x ray examinations in the accident and emergency department? BMJ 1987; 294: 943–7. 
    
    

11. Stiell IG, McDowell I, Nair RC, Acta A, Greenberg GH, McKnight RD, et al. Use of radiography in acute ankle injuries: physicians' attitudes and practice. Can Med Assoc J 1992; 147: 1671–8. 
    
    [Abstract](https://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTQ3LzExLzE2NzEiO3M6NDoiYXRvbSI7czoyMjoiL2Jtai8zMTEvNzAwNS81OTQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

12. McCaig LF. National hospital ambulatory medical care survey: 1992 emergency department summary. Advance Data 1994; 245: 1–12. 
    
    

13. Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Worthington JR. A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med 1992; 21: 384–90. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1016/S0196-0644(05)82656-3&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=1554175&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1992HL85300008&link_type=ISI) 

14. Stiell IG, McKnight RD, Greenberg GH, Nair RC, McDowell I, Wallace GJ. Interobserver agreement in the examination of acute ankle injury patients. Am J Emerg Med 1992; 10: 14–7. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1016/0735-6757(92)90117-G&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=1736906&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1992GY74700004&link_type=ISI) 

15. Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Reardon M, et al. Decision rules for the use of radiography in acute ankle injuries: refinement and prospective validation. JAMA 1993; 269: 1127–32. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1001/jama.1993.03500090063034&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=8433468&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1993KN61100029&link_type=ISI) 

16. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: application and methodological standards. N Engl J Med 1985; 313: 793–9. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1056/NEJM198509263131306&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=3897864&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1985ARA8900006&link_type=ISI) 

17. Lee TH. Evaluating decision aids: the next painful step. J Gen Intern Med 1990; 5: 528–9. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=2266437&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1990EJ76100014&link_type=ISI) 

18. Stiell IG, McKnight RD, Greenberg GH, McDowell I, Nair RC, Wells GA, et al. Implementation of the Ottawa ankle rules. JAMA 1994; 271: 827–32. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1001/jama.1994.03510350037034&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=8114236&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1994MZ49800028&link_type=ISI) 

19. Stiell IG. A study to develop clinical decision rules for the emergency department use of radiography in acute ankle injuries [thesis]. Ottawa: University of Ottawa, 1991. 
    
    

20. Katz D, Baptista J, Azan S, Pike MC. Obtaining confidence intervals for the risk ratio in cohort studies. Biometrics 1978; 34: 469–74. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.2307/2530610&link_type=DOI) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1978FP56900014&link_type=ISI) 

21. Feinstein AR. The “chagrin factor” and qualitative decision analysis. Arch Intern Med 1985; 145: 1257–9. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1001/archinte.1985.00360070137023&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=4015276&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1985ALE3000018&link_type=ISI) 

22. Moloney TW, Rogers DE. Medical technology: a different view of the contentious debate over costs. N Engl J Med 1979; 301: 1413–9. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=117359&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1979HY34300003&link_type=ISI) 

23. Angell M. Cost containment and the physician. JAMA 1985; 254: 1203–7. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1001/jama.1985.03360090093027&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=3927017&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1985APW6700022&link_type=ISI) 

24. Feinstein AR. Clinimetrics. New Haven: Yale University Press, 1987. 
    
    

25. Royal College of Radiologists Working Party. Influence of the Royal College of Radiologists' guidelines on hospital practice: a multicentre study. BMJ 1992; 304: 740–3. 
    
    

26. Royal College of Radiologists Working Party. Influence of Royal College of Radiologists' guidelines on referral from general practice. BMJ 1993; 306: 110–1. 
    
    

27. Basinski ASH, Naylor CD, Cohen MM, Ferris LE, Williams JI, Llewellyn-Thomas HA. Standards, guidelines, and clinical policies. Can Med Assoc J 1992; 146: 833–7. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=1544073&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 

28. Anderson G. Implementing practice guidelines. Can Med Assoc J 1993; 148: 753–5. 
    
    

29. Anonymous. Clinical trials and clinical practice. Lancet 1993; 342: 877–8. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1016/0140-6736(93)91937-H&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=8105158&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1993MA96300001&link_type=ISI) 

30. Hayward RSA, Laupacis A. Initiating, conducting and maintaining guidelines development programs. Can Med Assoc J 1993; 148: 507–12. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=8431814&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 

31. Greco PJ, Eisenberg JM. Changing physicians' practices. N Engl J Med 1993; 329: 1271–4. 
    
    [CrossRef](https://www.bmj.com/lookup/external-ref?access_num=10.1056/NEJM199310213291714&link_type=DOI) 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=8413397&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1993MB98800014&link_type=ISI) 

32. Ryd L, Bengtsson S. Isolated fracture of the lateral malleolus requires no treatment: 49 prospective cases of supination-eversion type II ankle fractures. Acta Orthop Scand 1992; 63: 443–6. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=1529699&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 
    
    [Web of Science](https://www.bmj.com/lookup/external-ref?access_num=A1992JM63800020&link_type=ISI) 

33. Long AE. Radiographic decision-making by the emergency physician. Emerg Med Clin North Am 1985; 3: 437–46. 
    
    [PubMed](https://www.bmj.com/lookup/external-ref?access_num=3926451&link_type=MED&atom=%2Fbmj%2F311%2F7005%2F594.atom) 

34. Svenson J. Need for radiographs in the acutely injured ankle. Lancet 1988; i: 244–5. 
    
    

35. Matthews MG. Guidelines for selective radiological assessment of inversion ankle injuries. BMJ 1986; 293: 959. 
    
    [FREE Full Text](https://www.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIyOTMvNjU1Mi85NTkiO3M6NDoiYXRvbSI7czoyMjoiL2Jtai8zMTEvNzAwNS81OTQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)