Validation of the second-generation Olympus colonoscopy simulator for skills assessment

 

 Buy Article Permissions and Reprints

Background and study aims: Simulators have potential value in providing objective evidence of technical skill for procedures within medicine. The aim of this study was to determine face and construct validity for the Olympus colonoscopy simulator and to establish which assessment measures map to clinical benchmarks of expertise.

Patients and methods: Thirty-four participants were recruited: 10 novices with no prior colonoscopy experience, 13 intermediate (trainee) endoscopists with fewer than 1000 previous colonoscopies, and 11 experienced endoscopists with more than 1000 previous colonoscopies. All participants completed three standardized cases on the simulator and experts gave feedback regarding the realism of the simulator. Forty metrics recorded automatically by the simulator were analyzed for their ability to distinguish between the groups.

Results: The simulator discriminated participants by experience level for 22 different parameters. Completion rates were lower for novices than for trainees and experts (37 % vs. 79 % and 88 % respectively, P < 0.001) and both novices and trainees took significantly longer to reach all major landmarks than the experts. Several technical aspects of competency were discriminatory; pushing with an embedded tip (P = 0.03), correct use of the variable stiffness function (P = 0.004), number of sigmoid N-loops (P = 0.02); size of sigmoid N-loops (P = 0.01), and time to remove alpha loops (P = 0.004). Out of 10, experts rated the realism of movement at 6.4, force feedback at 6.6, looping at 6.6, and loop resolution at 6.8.

Conclusions: The Olympus colonoscopy simulator has good face validity and excellent construct validity. It provides an objective assessment of colonoscopic skill on multiple measures and benchmarks have been set to allow its use as both a formative and a summative assessment tool.

References

• 1 Marshall J B. Technical proficiency of trainees performing colonoscopy: a learning curve.  Gastrointest Endosc. 1995;  42 287-291
  CrossrefPubMedSearch in Google Scholar
• 2 Wigton R S, Nicolas J A, Blank L L. Procedural skills of the general internist. A survey of 2500 physicians.  Ann Intern Med. 1989;  111 1023-1034
  PubMedSearch in Google Scholar
• 3 Tassios P S, Ladas S D, Grammenos I. et al . Acquisition of competence in colonoscopy: the learning curve of trainees.  Endoscopy. 1999;  31 702-706
  Thieme ConnectPubMedSearch in Google Scholar
• 4 Wexner S D, Eisen G M, Simmang C. Principles of privileging and credentialing for endoscopy and colonoscopy.  Surg Endosc. 2002;  16 367-369
  CrossrefPubMedSearch in Google Scholar
• 5 Cass O, Freeman M L, Cohen J. et al . Acquisition of competency in endoscopic skills (ACES) during training; a multicenter study.  Gastrointest Endosc. 1996;  43 308
  PubMedSearch in Google Scholar
• 6 Cass O W. Objective evaluation of competence: technical skills in gastrointestinal endoscopy.  Endoscopy. 1995;  27 86-89
  Thieme ConnectPubMedSearch in Google Scholar
• 7 Parry B R, Williams S M. Competency and the colonoscopist: a learning curve.  Aust N Z J Surg. 1991;  61 419-422
  CrossrefPubMedSearch in Google Scholar
• 8 Church J, Oakley J, Milsom J. et al . Colonoscopy training: the need for patience (patients).  Aust N Z J Surg. 2002;  72 89-91
  CrossrefPubMedSearch in Google Scholar
• 9 Bowles C J, Leicester R, Romaya C. et al . A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow?.  Gut. 2004;  53 277-283
  CrossrefPubMedSearch in Google Scholar
• 10 Sarker S K, Albrani T, Zaman A, Patel B. Procedural performance in gastrointestinal endoscopy: an assessment and self-appraisal tool.  Am J Surg. 2008;  196 450-455
  CrossrefPubMedSearch in Google Scholar
• 11 Park J, MacRae H, Musselman L J. et al . Randomized controlled trial of virtual reality simulator training: transfer to live patients.  Am J Surg. 2007;  194 205-211
  CrossrefPubMedSearch in Google Scholar
• 12 Grantcharov T P, Carstensen L, Schulze S. Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator.  JSLS. 2005;  9 130-133
  PubMedSearch in Google Scholar
• 13 Felsher J J, Olesevich M, Farres H. et al . Validation of a flexible endoscopy simulator.  Am J Surg. 2005;  189 497-500
  CrossrefPubMedSearch in Google Scholar
• 14 Sedlack R E, Kolars J C. Validation of a computer-based colonoscopy simulator.  Gastrointest Endosc. 2003;  57 214-218
  CrossrefPubMedSearch in Google Scholar
• 15 Mahmood T, Darzi A. A study to validate the colonoscopy simulator.  Surg Endosc. 2003;  17 1583-1589
  CrossrefPubMedSearch in Google Scholar
• 16 MacDonald J, Ketchum J, Williams R G, Rogers L Q. A lay person versus a trained endoscopist: can the PreOp endoscopy simulator detect a difference?.  Surg Endosc. 2003;  17 896-898
  CrossrefPubMedSearch in Google Scholar
• 17 Datta V, Mandalia M, Mackay S, Darzi A. The PreOp flexible sigmoidoscopy trainer. Validation and early evaluation of a virtual reality based system.  Surg Endosc. 2002;  16 1459-1463
  CrossrefPubMedSearch in Google Scholar
• 18 Moorthy K, Munz Y, Orchard T R. et al . An innovative method for the assessment of skills in lower gastrointestinal endoscopy.  Surg Endosc. 2004;  18 1613-1619
  PubMedSearch in Google Scholar
• 19 Koch A D, Buzink S N, Heemskerk J. et al . Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy.  Surg Endosc. 2007;  22 158-162
  PubMedSearch in Google Scholar
• 20 Mahmood T, Darzi A. The incidence of bowel perforations is positively related to the intensity of pain experienced during simulated colonoscopy.  Gastrointest Endosc. 2004;  59 P125, S1554
  PubMedSearch in Google Scholar
• 21 Aabakken L, Adamsen S, Kruse A. Performance of a colonoscopy simulator: experience from a hands-on endoscopy course.  Endoscopy. 2000;  32 911-913
  Thieme ConnectPubMedSearch in Google Scholar
• 22 Fitzgerald T N, Duffy A J, Bell R L. et al . Computer-based endoscopy simulation: emerging roles in teaching and professional skills assessment.  J Surg Educ. 2008;  65 229-235
  CrossrefPubMedSearch in Google Scholar
• 23 Gerson L B. Evidence-based assessment of endoscopic simulators for training.  Gastrointest Endosc Clin N Am. 2006;  16 489-509, vii-viii
  CrossrefPubMedSearch in Google Scholar
• 24 Williams C B, Thomas-Gibson S. Rational colonoscopy, realistic simulation, and accelerated teaching.  Gastrointest Endosc Clin N Am. 2006;  16 457-470
  CrossrefPubMedSearch in Google Scholar
• 25 Koch A D, Haringsma J, Schoon E J. et al . A second-generation virtual reality simulator for colonoscopy: validation and initial experience.  Endoscopy. 2008;  40 735-738
  Thieme ConnectPubMedSearch in Google Scholar
• 26 Shah S G, Saunders B P, Brooker J C, Williams C B. Magnetic imaging of colonoscopy: an audit of looping, accuracy and ancillary maneuvers.  Gastrointest Endosc. 2000;  52 1-8
  CrossrefPubMedSearch in Google Scholar
• 27 NHS Modernisation Agency .Improving endoscopy services. Meeting the challenges. Leicester, UK; NHS Modernisation Agency 2004
  PubMedSearch in Google Scholar
• 28 General Medical Council .Principles of good medical education and training. 2004
  PubMedSearch in Google Scholar
• 29 Joint Advisory Group for Gastrointestinal Endoscopy .Guidelines for the training, appraisal and assessment of trainees in gastrointestinal endoscopy. British Society of Gastroenterology, London; 2004. Available from: http://www.bsg.org.uk/pdf_word_docs/jag_recommendations_2004.pdf
  PubMed
• 30 Michelson J D, Manning L. Competency assessment in simulation-based procedural education.  Am J Surg. 2008;  196 609-615
  CrossrefPubMedSearch in Google Scholar
• 31 Sedlack R E, Baron T H, Downing S M, Schwartz A J. Validation of a colonoscopy simulation model for skills assessment.  Am J Gastroenterol. 2007;  102 64-74
  CrossrefPubMedSearch in Google Scholar
• 32 Shah S G, Brooker J C, Williams C B. et al . Effect of magnetic endoscope imaging on colonoscopy performance: a randomised controlled trial.  Lancet. 2000;  356 1718-1722
  CrossrefPubMedSearch in Google Scholar
• 33 Hoff G, Bretthauer M, Dahler S. et al . Improvement in caecal intubation rate and pain reduction by using 3-dimensional magnetic imaging for unsedated colonoscopy: a randomized trial of patients referred for colonoscopy.  Scand J Gastroenterol. 2007;  42 885-889
  CrossrefPubMedSearch in Google Scholar
• 34 Wehrmann K, Fruhmorgen P. Evaluation of a new three-dimensional magnetic imaging system for use during colonoscopy.  Endoscopy. 2002;  34 905-908
  Thieme ConnectPubMedSearch in Google Scholar
• 35 Cheung H Y, Chung C C, Kwok S Y. et al . Improvement in colonoscopy performance with adjunctive magnetic endoscope imaging: a randomized controlled trial.  Endoscopy. 2006;  38 214-217
  Thieme ConnectPubMedSearch in Google Scholar
• 36 Shah S G, Pearson H J, Moss S. et al . Magnetic endoscope imaging: a new technique for localizing colonic lesions.  Endoscopy. 2002;  34 900-904
  Thieme ConnectPubMedSearch in Google Scholar
• 37 Shah S G, Brooker J C, Thapar C. et al . Effect of magnetic endoscope imaging on patient tolerance and sedation requirements during colonoscopy: a randomized controlled trial.  Gastrointest Endosc. 2002;  55 832-837
  CrossrefPubMedSearch in Google Scholar

A. V. HaycockMRCP 

Wolfson Unit for Endoscopy
St. Mark’s Hospital
Imperial College London

London
UK

Fax: +44-20-84233588

Email: ahaycock@imperial.ac.uk