Surgical Simulators: which work best?
Most Surgical Simulators offer similar curricula and learning modules. But which one works best? Is there a clear leader in terms of validation? And how do the simulators compare with each other?
Few studies have compared simulator performance, but those that have, strongly favor the ProMIS™ hybrid or augmented reality approach over pure virtual reality.
In every study, the ProMIS™ mixed reality simulator out-performs VR in terms of validity and user preference.
In a comparison of Suturing Tasks on ProMIS and SurgicalSIM (METI), "Post-task survey scores for ProMIS were significantly higher for overall realism and thread behavior, reflection of clinical ability, and overall educational value." [1 ]ProMIS and LapSim (Surgical-Science) were compared for basic skills and suturing. "ProMIS was regarded by all participants as a better simulator for laparoscopic skills training on all tested features". ProMIS was "more realistic, had better haptic feedback and was more useful".[4]Versus the Xitact LS500, ProMIS was "more realistic" (70% vs. 20%) and "more useful" (65% vs. 36%). On a Dissection task, only ProMIS was able to distinguish between advanced trainees and beginners and had "significantly higher scores for all performance parameters".[3] Basic surgical skills were compared in five simulators: Lap Mentor (Simbionix); LapSim; SurgicalSIM; ProMIS and LTS2000 ISM60 (RealSim). Surgeons were asked to evaluate overall simulator effectiveness: ProMIS out-performed LapSim and SurgicalSim with a mean effectiveness rating of 3.56 versus 3.22 and 3.11 respectively. LapMentor also scored 3.56 while the LTS200 scored 3.78.[5]Compared with a Robotic Surgery Simulator (SimSurgery) and SurgicalSim for suturing and knot-tying, ProMIS metrics were "better at discerning predicted performance differences between experts and non-experts with a single task iteration".[2 ]LapMentor, LapSim and SimSurgery SEP were compared for construct and face validity on their Laparoscopic Cholecystectomy (LC) modules. No VR simulator achieved excellent face validity and very few simulator metrics demonstrated construct validity. Construct validity was demonstrated in 2 of 19 metrics on Lap Mentor using training status (cautery time, safe cautery) and 1 of 19 metrics using number of LCs or number of advanced cases (cautery time). In LapSim, this was 1 of 16 metrics using number of LCs (ripped ducts or vessels) and 2 of 16 using number of advanced cases (ducts or vessels ripped, blood loss); SimSurgery SEP showed no construct validity for any of the 5 metrics.[6]
This paper has been accepted for publication as an Emerging Technology Abstract at SAGES 2008
More about Validation on ProMIS >>>
1. Complex Laparoscopic Task Performance on Two New Computer-Based Training Devices; Erika K Fellinger MD, Michael E Ganey MD, Anthony G Gallagher PhD, Daniel J Scott MD, Ron W Bush BS, Neal E Seymour MD, Department of Surgery, Baystate Medical Center, Springfield, MA. SAGES 20062. Computer-Based Laparoscopic and Robotic Surgical Simulators: Performance Characteristics and Perceptions of New Users; David W Lin MD, John R Romanelli MD, Renee E Thompson MD, Michael E Ganey MD, Ron W Bush BS, Neal E Seymour MD, Baystate Medical Center, Department of Surgery. SAGES 20073. Comparison and Validation of Two Different Surgical Skills Simulators; Dieter Hahnloser MD, Rachel Rosenthal MD, Christian Hammel, Daniel Oertli, Markus Müller, Pierre-Alain Clavien, Department of Visceral and Transplantation Surgery, University Hospital Zurich, Switzerland. SAGES 20074. Augmented Versus Virtual Reality Laparoscopic Simulation: What is the Difference?: A Comparison of the ProMIS Augmented Reality Simulator Versus LapSim Virtual Reality Laparoscopic Simulator: Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ. Catharina Hospital, Eindhoven, The Netherlands. World J Surg. 2007 Apr;31(4):764-725. Criterion-Based Training with Surgical Simulators: Proficiency of Experienced Surgeons; Heinrichs, Wm, Lukoff, Brian, Youngblood, Patricia, Dev, Parvati, Shavelson, Richard, Hasson, M Harrith, Satava, M Richard, Mcdougall, M Elspeth, Wetter, Paul Alan; JSLS, Journal of the Society of Laparoendoscopic Surgeons, Vol. 11, No. 3. (September 2007), pp. 273-302.6. Comparison of Procedure Specific Virtual Reality Simulators: Construct and Face Validity; James R Korndorffer Jr. MD1, Thomas P. McIntyre MD2, John T. Paige MD3, Kent R. Van Sickle MD4, Shishir K. Maithel MD2, Dimitris Stefanidis MD1, Daniel J Scott MD5, Daniel B. Jones2;1 Tulane University Health Sciences Center; 2 Beth Israel Deaconess Medical Center; 3 Louisiana State University Health Sciences Center; 4 University of Texas Health Sciences Center at San Antonio; 5 Department University of Texas Health Sciences Center at Dallas. SAGES 2006