ProMIS Validation

<body topmargin=0 leftmargin=0 marginheight=0 marginwidth=0> <table cellpadding=0 cellspacing=0 border=0><tr> <td valign="top" rowspan="2" width=175 BGCOLOR="#FFFFFF" TEXT="#080000"> <div> <B><IMG SRC="1a764700.jpg" border=0 width="100" height="112" ALIGN="BOTTOM" HSPACE="0" VSPACE="0"></B></div> <div> <A HREF="index.htm" TARGET="_top" TITLE="The worldleading ProMIS Surgical Simulat"><B>Home</B></A></div> <div> <A HREF="promis.htm" TARGET="_top" TITLE="ProMIS surgical simulator"><B>ProMIS Basic Skills</B></A></div> <div> <A HREF="id64.htm" TARGET="_top" TITLE="SAGES/ACS FLS in ProMIS"><B>ProMIS and FLS</B></A></div> <div> <A HREF="promis.htm" TARGET="_top" TITLE="ProMIS surgical simulator"><B>ProMIS Procedures</B></A></div> <div> <A HREF="id59.htm" TARGET="_top" TITLE="ProMIS Laparoscopic Colectomy"><B>ProMIS LapColectomy</B></A></div> <div> <A HREF="id77.htm" TARGET="_top" TITLE="Create Modules and Tasks"><B>NEW! Design&Edit Tasks</B></A></div> <div> <A HREF="id25.htm" TARGET="_top" TITLE="ProMIS Validation"><B>ProMIS Validation</B></A></div> <div> <A HREF="id62.htm" TARGET="_top" TITLE="ProMISJ: a junior ProMIS"><B>ProMIS-J</B></A></div> <div> <A HREF="id52.htm" TARGET="_top" TITLE="ProMIS for industry: custom simulation s"><B>ProMIS for industry</B></A></div> <div> <A HREF="id44.htm" TARGET="_top" TITLE="ProMIS Technology"><B>ProMIS Technology</B></A></div> <div> <A HREF="id61.htm" TARGET="_top" TITLE="Downloads"><B>DOWNLOADS</B></A></div> <bR> <bR> </td> <td valign="top" colspan=2 height=95 style=" border-bottom: 1px solid #000066;" BGCOLOR="#FFFFFF" TEXT="#080000"> <bR> <bR> <div> <A HREF="index.htm" TARGET="_top" TITLE="The worldleading ProMIS Surgical Simulat"><B>Home</B></A><B>   </B><A HREF="promis.htm" TARGET="_top" TITLE="ProMIS surgical simulator"><B>ProMIS™</B></A><B>  </B><A HREF="id55.htm" TARGET="_top" TITLE="About simulation"><B>About simulation</B></A><B>  </B><A HREF="id49.htm" TARGET="_top" TITLE="About Haptica"><B>About Haptica</B></A><B>   </B><A HREF="id51.htm" TARGET="_top" TITLE="Contact us"><B>Contact us</B></A><B>  </B><A HREF="http://www.promis.org" TARGET="_top" TITLE="http://www.promis.org"><B>Support</B></A></div> <div> </div> </td> </tr><tr> <td valign="top" height=385 width="100%" colspan=2 BGCOLOR="#FFFFFF" TEXT="#080000"> <div> <B>ProMIS Validation</B></div> <div> <B>ProMIS: the best validated simulator - proven with FLS and shown to be better than pure VR simulators</B></div> <div> Much validation of ProMIS has been conducted: to date over 25 studies have been published and -uniquely - <I>all</I> are positive in terms of ProMIS' application for surgical training and research. Below are highlights and full abstracts. Click on >>> to go directly to the full abstract.</div> <div> Few studies have compared simulator performance, but those that have, strongly favor the ProMIS™ hybrid or augmented reality approach over pure virtual reality. </div> <div> <B>In every study, the ProMIS™ mixed reality simulator out-performs VR in terms of validity and user preference </B><A HREF="surgical_simulators_which_wo.htm" TARGET="_top" TITLE="Surgical Simulators: which work best?"><B>>>></B></A></div> <div> <B>ProMIS and FLS</B></div> <div> <B><IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">FLS tasks are transferable to the ProMIS simulator with traditional FLS scoring and intrinsic ProMIS metrics being good measurement tools. </B><A HREF="#sages_2008_p174" TITLE="ProMIS Validation"><B>>>></B></A></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">ProMIS metrics are <B>“excellent predictors of scores</B> in the standard FLS simulator” and “<B>predict readiness for FLS Certification</B>”.<FONT SIZE="1" COLOR="#000000" FACE="Arial" > </FONT><FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract1" TITLE="ProMIS Validation">1  2</A><A HREF="#abstract1" TITLE="ProMIS Validation"> </A><A HREF="#abstract1" TITLE="ProMIS Validation">3</A> <A HREF="#abstract1" TITLE="ProMIS Validation">>>></A></FONT></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">ProMIS <B>metrics are valid on peg transfer, pre-tied loop placement and knot-tying</B> tasks<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract4" TITLE="ProMIS Validation">4 >>></A></FONT></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">“Initial trials of the <B>metrics on the FLS Precision-Cutting Task show that ProMIS is as accurate as the current method</B>”<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract5" TITLE="ProMIS Validation"> </A></FONT><FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract5" TITLE="ProMIS Validation">5</A><A HREF="#abstract5" TITLE="ProMIS Validation"> >>></A></FONT></div> <div> <B>ProMIS vs pure VR</B></div> <div> <B><IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">Scores for ProMIS were significantly higher than for SurgicalSIM</B> for overall realism, thread behavior, reflection of clinical ability, and overall educational value.<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract6" TITLE="ProMIS Validation">6 >>></A></FONT></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">In comparison with LapSim, ProMIS was regarded by all participants as <B>a better simulator for laparoscopic skills</B> training on all tested features<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract7" TITLE="ProMIS Validation">7</A><A HREF="#abstract7" TITLE="ProMIS Validation"> >>></A></FONT></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">“Only [ProMIS] was able to distinguish between advanced trainees and beginners (and) was <B>graded more realistic (70% vs 33%) and more useful </B>(83% vs 62%)” than Xitact<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract8" TITLE="ProMIS Validation">8 >>></A></FONT></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">ProMIS <B>out-performed the virtual reality simulators</B> with a mean effectiveness rating of 3.56 versus 3.22 and 3.11 for LapSim and SurgicalSim (LapMentor also scored 3.56) <A HREF="#stanford_study" TITLE="ProMIS Validation">>>></A></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">The ability of performance metrics of [ProMIS] to discern predicted performance differences between experts and non-experts was better than for SimSurgery's robotic surgery simulator and SurgicalSIM<FONT SIZE="1" COLOR="#000000" FACE="Arial" > </FONT><FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract9" TITLE="ProMIS Validation">9 >>></A></FONT></div> <div> <B>ProMIS: the preferred Simulator</B></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">ProMIS can be used effectively with the DaVinci robot to obtain performance data with robotic instrumentation<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract10" TITLE="ProMIS Validation">10  >>></A></FONT></div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">Residents believe that ProMIS is easy to use and improved their operative skills<FONT SIZE="1" COLOR="#000000" FACE="Arial" ><A HREF="#abstract11" TITLE="ProMIS Validation">11  >>></A></FONT></div> <div> <I><B>_________________________________</B></I></div> <div> JSLS, Journal of the Society of Laparoendoscopic Surgeons, Vol. 11, No. 3. (September 2007), pp. 273-302. </div> <div> <I><B> <A NAME="stanford_study"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>CRITERION-BASED TRAINING WITH SURGICAL SIMULATORS: PROFICIENCY OF EXPERIENCED SURGEONS</B></I></div> <div> Heinrichs, Wm, Lukoff, Brian, Youngblood, Patricia, Dev, Parvati, Shavelson, Richard, Hasson, M Harrith, Satava, M Richard, Mcdougall, M Elspeth, Wetter, Paul Alan</div> <div> <B>Objective: </B></div> <div> In our effort to establish criterion-based skills training for surgeons, we assessed the performance of 17 experienced laparoscopic surgeons on basic technical surgical skills recorded electronically in 26 modules selected in five commercially available, computer-based simulators.</div> <div> <B>Methods/Procedures:</B></div> <div> Performance data were derived from selected surgeons randomly assigned to simulator stations, and practicing repetitively during three one-half day sessions on five different simulators. We measured surgeon proficiency defined as efficient, error-free performance and developed proficiency score formulas for each module. Demographic and opinion data were also collected.</div> <div> <B>Results:</B></div> <div> Surgeons’ performance demonstrated a sharp learning curve with the most performance improvement seen in early practice attempts. Median scores and performance levels at the 10th, 25th, 75th, and 90th percentiles are provided for each module. Construct validity was examined for two modules by comparing experienced surgeons’ performance with that of a convenience sample of lessexperienced surgeons.</div> <div> <B>Conclusions:</B></div> <div> A simple mathematical method for scoring performance is applicable to these simulators. Proficiency levels for training courses can now be specified objectively by residency directors and by professional organizations for different levels of training or post-training assessment of technical performance. But data users should be cautious due to the small sample size used in this study and the need for further study into the reliability and validity of the use of surgical simulators as assessment tools.</div> <div> <B>Summary comment</B></div> <div> The simulators used included pure Virtual Reality Simulators and two simulators with real haptics (including ProMIS):</div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">Lap Mentor from Simbionix (pure Virtual Reality)</div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">LapSim from Surgical-Science AB (pure Virtual Reality)</div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">SurgicalSIM from METI (pure Virtual Reality)</div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">ProMIS from Haptica (real haptics with metrics)</div> <div> <IMG BORDER="0" SRC="Symb075c00b700c800000000.png" HEIGHT="13" WIDTH="24" ALIGN="bottom" HSPACE="0" VSPACE="0">LTS2000 ISM60 from RealSim (real haptics with metrics)</div> <div> ProMIS out-performed the virtual reality simulators with a mean effectiveness rating of 3.56 versus 3.22 and 3.11 for LapSim and SurgicalSim (LapMentor also scored 3.56).</div> <div> <I><B>_________________________________</B></I></div> <bR> <div>  <A NAME="sages_2008_p174"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A>SAGES 2008 P174</div> <div> <I><B>FLS TEST IS TRANSFERABLE TO PROMIS SIMULATOR</B></I></div> <div> Eric S Hungness MD,Albert Amini BA,Deb E Rooney MS,Eric T Volckman MD,Nathaniel J Soper MD, Feinberg School of Medicine, Northwestern University, Chicago, IL </div> <div> <B>INTRODUCTION: </B></div> <div> The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is comprised of five tasks with an objective scoring system, and has been incorporated by SAGES in their Fundamentals of Laparoscopic Surgery (FLS) program. MISTELS has high inter-rater and test-retest reliability and correlates with operative skill. However, the FLS program is labor intensive, requiring a trained proctor. The ProMIS simulator allows for assessment of physical tasks (instrument path length and instrument smoothness) through instrument tracking technology. We hypothesized that the FLS scores obtained in the ProMIS simulator as well as ProMIS metrics would correlate with standard FLS scoring. </div> <div> <B>METHODS: </B></div> <div> Twenty general surgery residents (13 junior and 7 senior) had baseline laparoscopic skills assessed using MISTELS in the standard FLS and ProMIS simulators (pre-test). Nine junior and 4 senior residents had a post-test after four weeks of training. Tasks were scored by FLS and ProMIS metrics. Total path length (TP) and total smoothness (TS) were calculated by adding the path lengths and smoothness of each individual task. ANOVA was used to compare the mean (SD) of total and individual task scores for pre- and post-tests in the FLS and ProMIS simulators. Student's t-test was used to compare ProMIS metrics. Pearson's correlations were calculated for standard FLS scores in relation to ProMIS FLS scores, TP and TS. Significance was defined as p < 0.01. </div> <div> <B>RESULTS: </B></div> <div> All residents showed statistically significant improvement in post-test total and individual task FLS scores on either the FLS or ProMIS simulator. 100% and 88% of residents achieved passing post-test scores on the FLS and ProMIS simulator, compared to 30% and 29.2% on the pre-test. There was no difference in junior and senior resident posttest scores (87.6 vs 79.1). ProMIS path length and smoothness were significantly reduced across all tasks (range 14- 68%). Total ProMIS FLS scores (0.729), TP (-0.753) and TS (- 0.769) significantly correlated with total standard FLS simulator scores. All residents with TP < 4000mm or TS < 6000 achieved a passing total FLS score. </div> <div> <B>CONCLUSIONS: </B></div> <div> All surgical residents achieved a passing FLS score after a 4-week laparoscopic skills curriculum. <B>FLS tasks are transferable to the ProMIS simulator with traditional FLS scoring and intrinsic ProMIS metrics being good measurement tools. </B>A ProMIS total path length <4000 mm or total smoothness <6000 reliably predicts a passing FLS score. </div> <bR> <bR> <div> SAGES 2006 Scientific Session S064 </div> <div> <I><B> <A NAME="abstract1"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>1. CONCURRENT VALIDITY OF AUGMENTED REALITY METRICS APPLIED TO THE FUNDAMENTALS OF LAPAROSCOPIC SURGERY (FLS)</B></I></div> <div> E. Matt Ritter MD, Tamara W Kindelan MD, Curtis Michael, Elisabeth A Pimentel BA, Mark W Bowyer MD, 1NCA Medical Simulation Center, Department of Surgery, Uniformed Services University, 2Division of General Surgery, National Naval Medical Center, Bethesda Maryland </div> <div> <B>Objective </B></div> <div> Current skills assessment in the Fundamentals of Laparoscopic Surgery (FLS) program is labor intensive requiring one proctor for every 1-2 subjects. The ProMIS Augmented Reality (AR) simulator (Haptica, Dublin IRE) allows for objective assessment of physical tasks through instrument tracking technology. We hypothesized that the ProMIS metrics could differentiate between ability groups as well as standard FLS scoring with fewer personnel requirements.</div> <div> <B>Methods</B></div> <div> We recruited 60 volunteer subjects. Subjects were stratified based on their laparoscopic surgical experience. Those who had performed more than 100 laparoscopic procedures were considered experienced (n=8). Those with less than 10 laparoscopic procedure were considered novices (n=44). The rest were intermediates (n=8). All subjects performed up to 5 trials of the peg transfer task from FLS in the ProMIS simulator. FLS score, instrument path length, and instrument smoothness assessment were generated for each trial. </div> <div> <B>Results</B></div> <div> For each of the 5 trials, experienced surgeons outperformed intermediates who in turn out performed novices. Statistically significant differences were seen between the groups across all trials for FLS score (p < 0.001), ProMIS path length (p <0.001) and ProMIS smoothness (p < 0.001). When the FLS score was compared to the path length and smoothness metrics, a strong relationship between the scores was apparent for novices (r = 0.78, r = 0.94 , p < 0.001) respectively), intermediates (r = 0.5, p = 0.2 , r = 0.98, p < 0.001), and experienced surgeons (r = 0.86, p = 0.006, r = 0.99, p < 0.001) </div> <div> <B>Conclusions</B></div> <div> The construct that the standard scoring of the FLS peg transfer task can discriminate between experienced, intermediate, and novice surgeons is validated. The same construct is valid when the task is assessed using the metrics of the ProMIS. The high correlation between these scores establishes the concurrent validity of the ProMIS metrics. The use of AR for objective assessment of FLS tasks could reduce the personnel requirements of assessing these skills while maintaining the objectivity.</div> <div> __________________________________________</div> <bR> <div> SAGES 2006 Education/Outcomes-P237 </div> <div> <I><B> <A NAME="abstract2"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>2.  VALIDATION OF THE PROMIS HYBRID SIMULATOR USING A STANDARD SET OF LAPAROSCOPIC TASKS</B></I></div> <div> A L McCluney MD, L S Feldman MD, G M Fried, Steinberg-Bernstein Centre for Minimally Invasive Surgery, McGill University Health Centre, Montreal, QC, Canada </div> <div> <B>Introduction</B></div> <div> SAGES Fundamentals of Laparoscopic Surgery (FLS) tasks are validated measures of technical skills, however FLS scoring requires a trained proctor. The ProMIS simulator (Haptica; Dublin, IR) is a `hybrid' system with physical and virtual reality tasks. It has the flexibility to incorporate any physical task and score it with ProMIS metrics. Metrics are automated and report motion analysis data as instrument path length (PL) and instrument smoothness (IS). The purpose of this study was to test for construct and concurrent validity using FLS tasks in the ProMIS simulator. </div> <div> <B>Methods</B></div> <div> 5 laparoscopic novices and 5 experts performed FLS tasks in both the standard FLS simulator box and the ProMIS simulator. Assessments were made based on FLS metrics, as well as PL and IS. Student's t-test was used to compare the mean (SD) of total scores for novices and experts. Pearson's correlations were calculated for standard FLS scores in relation to ProMIS FLS scores, total PL, and total IS. Significance was defined as p < 0.01 (*). </div> <div> <B>Results</B></div> <div> Standard FLS scores correlated strongly with ProMIS FLS scores (r=0.90), total PL (r=-0.83), and total IS (r=-0.78) (p&lt; 0.01). </div> <div> <B>Conclusions</B></div> <div>  FLS tasks performed in ProMIS, when scored by either traditional FLS metrics or by intrinsic ProMIS metrics, discriminate effectively between novices and experts. Based on the observed correlations, ProMIS FLS scores, total PL, and total IS are excellent predictors of scores in the standard FLS simulator.</div> <div> __________________________________________</div> <bR> <div> SAGES 2007 Education/Outcomes - P279</div> <div> <I><B> <A NAME="abstract3"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>3.  AUTOMATED PROMIS SIMULATOR METRICS PREDICT READINESS FOR FLS CERTIFICATION</B></I></div> <div> Anthony L McCluney MD, J Cao, G N Polyhronopoulos MD, D D Stanbridge, L S Feldman MD, G M Fried MD, Steinberg-Bernstein Centre for Minimally Invasive Surgery, McGill University, Montreal, QC, Canada </div> <div> <B>Introduction</B></div> <div> SAGES Fundamentals of Laparoscopic Surgery (FLS) tasks are validated measures of technical skills. Certification requires travel to a testing site and a fee, thus a reliable method of predicting readiness for the exam would be advantageous. The ProMIS simulator (Haptica) provides automated scoring. FLS tasks can be placed in the ProMIS simulator and scored using time (TT), as well as motion analysis metrics: instrument path length (PL) and instrument smoothness (IS). This study was designed to evaluate these automated ProMIS metrics and their ability to predict readiness for FLS certification. </div> <div> <B>Methods</B></div> <div> 33 subjects (12 students, 16 residents PGY 1-4, and 5 experts) performed FLS tasks in the standard simulator and in ProMIS. Tasks were scored by FLS and ProMIS metrics. For each ProMIS metric, the total score was calculated by summing the scores for the 5 FLS tasks. Pearson's correlations were calculated for ProMIS metrics versus standard FLS scores. Multivariate regression analysis identified independent predictors of standard FLS performance. These variables were then used for sensitivity and specificity calculations in order to establish a ProMIS pass-fail score for predicting readiness for FLS certification. Significance was defined as p<0.05. </div> <div> <B>Results</B></div> <div> TT (r= -0.82), PL (r= -0.56), and IS (r= -0.75) all correlated significantly with standard FLS score. Multivariate regression analysis identified TT as the strongest predictor of FLS score. A TT score of 1000 maximizes sensitivity and specificity and was identified as the pass-fail for reliably predicting FLS performance. </div> <div> <B>Conclusions</B></div> <div> Automated ProMIS metrics correlate well with standard FLS performance. In this study sample, a TT score less than 1000 reliably predicted a passing FLS certification score. </div> <div> __________________________________________</div> <bR> <div> SAGES 2006 Scientific Sessions S065</div> <div> <I><B> <A NAME="abstract4"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>4.  WHAT CAN MOTION DERIVATIVES TELL US ABOUT SKILL </B></I><FONT SIZE="2" COLOR="#336699" FACE="Univers-Bold" ><I><B>PERFORMANCE? </B></I></FONT></div> <div> Laurel N Vuong BS, Steven D Schwaitzberg MD, Caroline G Cao PhD, Tufts University School of Medicine, Cambridge Health Alliance, Tufts University School of Engineering </div> <div> Surgical simulators are a popular topic of discussion on training in laparoscopic surgery. They reduce the need to use human cadavers or animal models for skills development. A subset of the MISTELS methodology has been employed in the manual skills assessment for the Fundamentals Laparoscopic Skills (FLS) program because it was shown to be a valid discriminator of surgical experience. Pure performance outcome, such as time to task completion and number of errors, is used for scoring, which is dependent on the consistency of the scorer. A new simulator environment has been created which uses motion tracking for measurement of performance outcome measures and motion derivatives such as smoothness and efficiency. </div> <div> The purpose of the study was to determine if the motion derivatives can be used to automatically and objectively discriminate experience levels. Twenty-one subjects (6 medical students, 14 surgical residents, and 1 expert surgeon) were recruited to perform the following tasks: peg transfer, pattern cutting, pre-tied loop placement, extracorporeal and intracorporeal knot-tying in the new simulator environment. Subjects were evaluated on time to completion, errors, smoothness and total path length (used to calculate efficiency). </div> <div> Results show that experience level is still distinguishable when using task-dependent parameters to evaluate performances during peg transfer (p= 0.035), pre-tied loop placement (p= 0.022), extracorporeal (p= 0.0006) and intracorporeal (p= 0.025) knot tying in this new simulator environment. </div> <div> Evaluation of performance using task-independent parameters significantly distinguished training level in three tasks: (1) smoothness of the left instrument was significant as a function of experience level in extracorporeal knot-tying (p= 0.016), (2) efficiency (total path length divided by time to completion) was also significant in the movement of the right tool as a function of experience level in peg transfer (p= 0.0011) and (3) pre-tied loop placement (p= 0.013979). </div> <div> This preliminary analysis shows that automatic and objectively measured motion derivatives can be associated with the level of experience. These results indicate a potential for the application of an automatic and objective means of skills evaluation.<FONT SIZE="2" COLOR="#000000" FACE="Univers-Bold" > </FONT></div> <div> __________________________________________</div> <bR> <div> SAGES 2007 ETP057</div> <div> <I><B> <A NAME="abstract5"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>5.  OBJECTIVE MEASUREMENT OF FLS PRECISION CUTTING TASK</B></I></div> <div> Derek Young, Fiona Slevin, Derek Cassidy, Donncha Ryan, Haptica Inc </div> <div> The Precision-Cutting Task in the SAGES/ACS FLS Program requires the user to dissect a circle of specific size and shape from a marked piece of mesh. Currently, measurement of the accuracy and area dissected is done by observation and by measuring the dissected mesh on a measurement grid. Using advanced vision-tracking, the ProMIS surgical simulator takes an image of the dissected mesh and automatically generates a metric, indicating the accuracy of the shape and area dissected. </div> <div> <B>Method </B></div> <div> 1. Once the user has completed the Precision-Cutting Task, ProMIS takes an image of the dissected mesh. The image is converted to binary image and then scanned using a blob detection algorithm which produces a list of blobs. </div> <div> 2. The blob with the largest area is taken as the cut out area and the number of pixels are counted inside this area. </div> <div> 3. The actual measurement for area is given in cm2. This is calculated by counting the number of pixels in a known area of the image and then using ratios to determine the area of the cut out. (Note: this calibration step is achieved by taking the tissue off the tray and running the blob detection and pixel count on the uncovered black foam of which the exact area is known)</div> <div> <B>Results</B></div> <div> Initial trials of the metrics on the FLS Precision- Cutting Task show that ProMIS is as accurate - and frequently more accurate - than the current human observation method. </div> <div> __________________________________________</div> <bR> <div> SAGES 2006 Education/Outcomes-P224 </div> <div> <I><B> <A NAME="abstract6"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>6.  COMPLEX LAPAROSCOPIC TASK PERFORMANCE ON TWO NEW COMPUTER-BASED SKILLS TRAINING DEVICES</B></I></div> <div> Erika K Fellinger MD, Michael E Ganey MD, Anthony G Gallagher PhD, Daniel J Scott MD, Ron W Bush BS, Neal E Seymour MD, Department of Surgery, Baystate Medical Center, Springfield, MA </div> <div> <B>Introduction </B></div> <div> New computer-based skills training devices can simulate and measure performance of complex surgical tasks. The aim of this study is to determine basic face and construct validity characteristics of two new devices configured for laparoscopic suturing and knot-tying tasks. </div> <div> <B>Methods </B></div> <div> At the 2005 SAGES meeting, Learning Center attendees evaluated two computer-based skills training platforms: SurgicalSIM (SS), a virtual reality (VR) device (METI, Sarasota, FL; SimSurgery, Oslo, Norway) and ProMIS (PM), a computer-enhanced video trainer (Haptica, Ltd., Dublin, Ireland). Demographic and training data were collected from 73 subjects. All were asked to perform 2 iterations of laparoscopic suturing and intracorporeal knot-tying (10-minute time limit) on each device. A 6-question survey was used to define impressions of task realism, relevance, and execution using a 5-point Likert scale. Performance data (SS: time, path length, errors; PM: time, path length, smoothness) were collected on both devices and comparisons made between user-defined expert and nonexpert (intermediate and novice) groups (ANOVA and Mann Whitney U test). </div> <div> <B>Results</B></div> <div> 46 subjects used SS and 56 used PM. Task completion rate was 80% for SS and 93% for PM. Experts performed better than nonexperts for all performance measures on SS (composite score 496±41 vs 699±60, p < 0.005) and PM (974±111 vs 1466±89, p < 0.005). Post-task survey scores for PM were significantly higher for perceived realism (overall realism and thread behavior), reflection of clinical ability, and overall educational value. Perception of educational value was not significantly different between the devices among subjects with prior VR experience. </div> <div> <B>Conclusions</B></div> <div> Using subject-defined expert and nonexpert groups, construct validity was demonstrated for all performance measures on both training devices. Surveyed face validity measures favored the non-VR device, but results also suggest that subjects with prior VR training experience are more apt to accept a new VR surgical training platform. </div> <div> __________________________________________</div> <bR> <div> World J Surg. 2007 Apr;31(4):764-72.</div> <div> <I><B> <A NAME="abstract7"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>7.  Augmented versus Virtual Reality Laparoscopic Simulation: What Is the Difference? : A Comparison of the ProMIS Augmented Reality Laparoscopic Simulator versus LapSim Virtual Reality Laparoscopic Simulator.</B></I></div> <div> <A HREF="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Botden+SM%22%5BAuthor%5D" TARGET="_top" TITLE="http://www.ncbi.nlm.nih.gov/entrez/query">Botden SM</A>, <A HREF="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Buzink+SN%22%5BAuthor%5D" TARGET="_top" TITLE="http://www.ncbi.nlm.nih.gov/entrez/query">Buzink SN</A>, <A HREF="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Schijven+MP%22%5BAuthor%5D" TARGET="_top" TITLE="http://www.ncbi.nlm.nih.gov/entrez/query">Schijven MP</A>, <A HREF="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Search&itool=pubmed_Abstract&term=%22Jakimowicz+JJ%22%5BAuthor%5D" TARGET="_top" TITLE="http://www.ncbi.nlm.nih.gov/entrez/query">Jakimowicz JJ</A>. Catharina Hospital, Eindhoven, The Netherlands.</div> <div> <B>Background </B></div> <div> Virtual reality (VR) is an emerging new modality for laparoscopic skills training; however, most simulators lack realistic haptic feedback. Augmented reality (AR) is a new laparoscopic simulation system offering a combination of physical objects and VR simulation. Laparoscopic instruments are used within an hybrid mannequin on tissue or objects while using video tracking. This study was designed to assess the difference in realism, haptic feedback, and didactic value between AR and VR laparoscopic simulation. </div> <div> <B>Methods</B></div> <div> The ProMIS AR and LapSim VR simulators were used in this study. The participants performed a basic skills task and a suturing task on both simulators, after which they filled out a questionnaire about their demographics and their opinion of both simulators scored on a 5-point Likert scale. The participants were allotted to 3 groups depending on their experience: experts, intermediates and novices. Significant differences were calculated with the paired t-test. </div> <div> <B>Results </B></div> <div> There was general consensus in all groups that the ProMIS AR laparoscopic simulator is more realistic than the LapSim VR laparoscopic simulator in both the basic skills task (mean 4.22 resp. 2.18, P < 0.000) as well as the suturing task (mean 4.15 resp. 1.85, P < 0.000). The ProMIS is regarded as having better haptic feedback (mean 3.92 resp. 1.92, P < 0.000) and as being more useful for training surgical residents (mean 4.51 resp. 2.94, P < 0.000). </div> <div> <B>Conclusions </B></div> <div> In comparison with the VR simulator, the AR laparoscopic simulator was regarded by all participants as a better simulator for laparoscopic skills training on all tested features.</div> <div> __________________________________________</div> <bR> <div> SAGES 2007 Education/Outcomes - P270</div> <div> <I><B> <A NAME="abstract8"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>8.  COMPARISON AND VALIDATION OF TWO DIFFERENT SURGICAL SKILLS SIMULATORS</B></I></div> <div> Dieter Hahnloser MD,Rachel Rosenthal MD,Christian Hammel,Daniel Oertli,Markus Müller,Pierre-Alain Clavien, Department of Visceral and Transplantation Surgery, University Hospital Zurich, Switzerland </div> <div> <B>Background</B></div> <div> Simulators are increasingly incorporated in surgical training and validation is important. The simulations need to resemble the task they are based upon (face validity) and the simulator should be able to differentiate between levels of experience (construct validity). </div> <div> <B>Aim</B></div> <div> To assess two different types of computer-based simulators: the fully computerised virtual reality (VR) simulator Xitact LS500 (VR-simulator) and the hybrid ProMisTM simulator. Methods: 146 participants (61%) of the 22nd Davos International Gastrointestinal Surgery Workshop performed on a voluntary basis three similar exercises (camera navigation, clip and cut, and dissection) on the two different simulators. Objective performance parameters recorded by either simulator and subjective evaluation by questionnaire were compared between beginner (n=73) and advanced participants (n=73). </div> <div> <B>Results</B></div> <div> The camera navigation exercise was completed by 52% of the participants on the VR- and by 47% on the hybrid simulator with no difference in performance parameters between beginners and advanced trainees. The hybrid simulator was graded more realistic (70% vs. 20%, p=.001) and more useful (65% vs. 36%, p=.043) than the VR-simulator. Participation was higher at the clip and cut exercise (75% VR- and. 52% hybrid simulator) and advanced trainees performed significantly better (shorter tool-tip-travel distance, smoother, quicker and with higher score) on both simulators compared to beginners. The clip and cut exercise was graded more realistic on the hybrid (81% vs. 44%, p=.007) and similar useful on both simulators (77% vs. 72%). The dissection exercise was completed more often on the hybrid simulator (47% vs. 23%, p=0.002). Only the hybrid simulator was able to distinguish between advanced trainees and beginners, with significantly higher scores for all performance parameters for the latter. The hybrid simulator was graded more realistic (70% vs. 33%, p=.016) and more useful (83% vs. 62%, p=.12). Overall, acceptance of requirement to train on and to be evaluated by such simulators is still low (53% and 50%, respectively). </div> <div> <B>Conclusion</B></div> <div> Fully computerized VR- or hybrid simulator performance parameters can distinguish between beginner and advanced trainees for perceptual motor skills (proving construct validity), but not for visuo-spatial exercises such as the camera navigation. </div> <div> __________________________________________</div> <bR> <div> SAGES 2007: S077</div> <div> <I><B> <A NAME="abstract9"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>9.  COMPUTER-BASED LAPAROSCOPIC AND ROBOTIC SURGICAL SIMULATORS: PERFORMANCE CHARACTERISTICS AND PERCEPTIONS OF NEW USERS</B></I></div> <div> David W Lin MD, John R Romanelli MD, Renee E Thompson MD,Michael E Ganey MD, Ron W Bush BS, Neal E Seymour MD, Baystate Medical Center, Department of Surgery </div> <div> The expanding inventory of advanced surgical training devices now includes simulators for laparoscopic and robotic surgery. In order to define perceptions of the need and value of such devices, we evaluated the initial experience of surgeons using both in the course of performance of an advanced laparoscopic skill. </div> <div> <B>Methods</B></div> <div> At the 2006 SAGES meeting, 62 Learning Center attendees evaluated a new virtual reality (VR) robotic surgery simulator (RS) [SimSurgery, Oslo, Norway] as well as either a computer-enhanced laparoscopic [ProMIS (PM), Haptica, Ltd, Dublin, Ireland] or a VR simulator [SurgicalSIM (SS), SimSurgery and METI, Inc, Sarasota, FL]. Demographic and training data were collected and all were assessed during one iteration of laparoscopic suturing and knot-tying on RS and either PM or SS. An 8-question survey was used to determine users? impressions of task realism, interface quality, and educational value (5-point Likert scale). Performance data [time, path length, smoothness (PM), errors (SS/RS)] were collected and comparisons made between user-defined groups and different simulation platforms (Mann-Whitney Test, ANOVA). </div> <div> <B>Results</B></div> <div> Task completion rate was greater for experts than nonexperts on all platforms (PM 100% vs 75%; SS 100% vs 36%; RS 93% vs 36%). Experts performed better than nonexperts on all performance measures on PM (p<0.05: time 154±16 vs 205±12; path length 820±97 vs 1287±97; smoothness 952±111 vs 1582±127). There were no significant differences between experts and nonexperts for SS and RS performance measures. Perception of value of haptic features was less for subjects with prior robot experience (n=10; p<0.05). Otherwise realism, interface quality, and educational value scores did not differ on the basis of prior simulator or robot use. Nonexperts found that robotic simulation better reflected clinical skill than did experts. Overall, subjective quality was scored higher for PM than for SS or RS. </div> <div> <B>Conclusions</B></div> <div> The ability of performance metrics of the computer-enhanced simulator to discern predicted performance differences between experts and nonexperts was better than for VR devices with a single task iteration. Initial use of VR devices was associated with a lower overall perception of realism and educational value as compared to use of physical objects in the non-VR simulator. This may reflect the need for familiarization with the computer-generated environment before the educational potential of VR can be realized. </div> <div> __________________________________________</div> <bR> <div> SAGES 2006 Scientific Sessions S096 </div> <div> <I><B> <A NAME="abstract10"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>10.  A COMPUTERIZED ANALYSIS OF ROBOTIC VERSUS LAPAROSCOPIC TASK PERFORMANCE </B></I></div> <div> V K Narula MD, W C Watson MD, S S Davis MD, K Hinshaw BS, B J Needleman MD, D J Mikami MD, J W Hazey MD, J H Winston MD, P Muscarella MD, M Rubin, V Patel MD, W S Melvin MD, The Ohio State University. CMIS. Columbus, OH </div> <div> <B>Introduction </B></div> <div> Robotic technology has been postulated to improve performance in advanced surgical skills. We utilized a novel computerized assessment system to objectively describe the technical enhancement in task performance comparing robotic and laparoscopic instrumentation. </div> <div> <B>Methods and Procedures</B></div> <div> Advanced laparoscopic surgeons (2- 10 yrs experience) performed 3 unique task modules using laparoscopic and Telerobotic Surgical Instrumentation (Intuitive Surgical, Sunnyvale, CA). Performance was evaluated using a computerized assessment system (ProMIS, Dublin, Ireland) and results were recorded as time (sec), total path (mm) and precision. Each surgeon had an initial training session followed by two testing sessions for each module. A Paired Student's T-Test was used to analyze the data. </div> <div> <B>Results</B></div> <div> 10 surgeons completed the study. Objective assessment of the data is presented in the table below. 8/10 surgeons had significant technical enhancement utilizing robotic technology Laparoscopic vs Robotic Time (sec) Total Path(mm) Precision Module 1210 vs 161 # 11649 vs 5571 * 1434 vs 933 * Module 2119 vs 68 * 5573 vs 1949 * 853 vs 406 * Module 377 vs 55 * 4488 vs 2390 * 552 vs 358 * # = p < 0.009 * = p<0.001 Conclusions: The ProMIS computerized assessment system can be modified to objectively obtain task performance data with robotic instrumentation. All the tasks were performed faster and with more precision using the robotic technology than standard laparoscopy.</div> <div> __________________________________________</div> <bR> <div> SAGES 2006 Education/Outcomes-P219 </div> <div> <I><B> <A NAME="abstract11"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A></B></I><I><B>11.  INTEGRATING SIMULATION LAB TRAINING INTO A SURGICAL RESIDENCY PROGRAM: IS VOLUNTARY PARTICIPATION EFFECTIVE</B></I></div> <div> Lily Chang MD, James Petros MD, Donald Hess MD, Caroline Rotondi BA, Timothy Babineau MD, Boston Medical Center </div> <div> <B>Objective</B></div> <div> Surgical training programs nationwide are struggling with the integration of simulation training into their curriculum given the constraints of the 80-hour work week. We examine the effectiveness of voluntary training in a simulation lab as part of the surgical curriculum. Methods: The ProMIS simulator was introduced into the general surgery residency at Boston University Medical Center. All categorical residents (28) were required to attend a 2-hour training session and curriculum review. Non-categorical residents (23) were given the option to complete training. After the introductory session, time spent in the lab was encouraged, but voluntary. Use of the simulator was tracked for all residents. Participation in the simulation curriculum was defined as 3 or more uses of the simulator. After 3 months, all residents completed a survey regarding the simulation lab and their simulator usage. Results: 26 (93%) categorical residents and 3 (6%) non-categorical residents completed the introductory simulator training session. Over a 3 month period, use of the simulator at least once was 31% among all eligible residents; 80% of PGY1, 40% of PGY2, 60% of PGY3, and 0% of PGY4 and PGY5. Four residents (14%) participated in the simulation curriculum. 71% of simulator usage was during working hours while 29% was completed post-call or off duty. Most residents agreed that the simulator was easy to use and improved their operative skills, but did not think it was a good substitute for actual operative experience. Reported reasons for not using the simulator included off-site rotation (44%), no time (30%), and no interest (11%). </div> <div> <B>Conclusions</B></div> <div> Voluntary use of a surgical simulation lab leads to minimal participation in a training curriculum. Participation should be mandatory if it is to be an effective part of a residency curriculum.</div> <div>  <FONT SIZE="2" COLOR="#336699" FACE="Arial" >__________________________________________</FONT></div> <bR> <div> SAGES 2007: S080</div> <div> <I><B>12.  THE IMPACT OF KNOWLEDGE OF RESULTS IN SURGICAL SKILLS TRAINING</B></I></div> <div> A. O'Connor MD, C. Cao PhD, S. Schwaitzberg MD, Department of Mechanical Engineering, Tufts University. </div> <div> <B>Background</B></div> <div> Concerns about the adequacy of advanced laparoscopic training continue to be raised despite a proliferation of training systems exist. The manner in which the training modules are structured to maximize learning has not been examined. There are many aspects to the accumulation of laparoscopic skills during training, one of which is Knowledge of Results (KR), i.e. the information provided to individuals about the outcomes of their motor responses in their environment. We studied the effects of KR on the learning curve of laparoscopic suturing and knot tying. </div> <div> <B>Aims</B></div> <div> We evaluated the learning curves of 9 medical students with no previous laparoscopic surgical experience under three different conditions, each with different levels of knowledge of results. </div> <div> <B>Methods</B></div> <div> Subjects were randomly assigned to one of three groups. Each subject attended a training session for 1 hour each day, 6 days a week for 4 weeks. Group 1 (No feedback) received no knowledge of results (KR) and no performance feedback. Group 2 (feedback only) received factual KR following each training session, but no coaching. Group 3 (feedback and coaching) received KR and coaching. Learning curves were plotted based on task time, smoothness of instruments and instrument's path length. The task used was an intracorporeal suture/knot tying in the ProMIS laparoscopic simulator. Perceived workload for each session was recorded using a standardized NASA TLX workload score. </div> <div> <B>Results</B></div> <div> The variability across each session for each student was calculated for each of the three parameters. There was statistical significance between the groups for all parameters (p-values 0.0002, 0.0002 and 0.009). Significant differences were found between groups 2 and 3 and group 1 (p values 0.0314-0.0410) Groups 2 and 3 learned significantly faster than those in Group 1, reaching performance plateaus at earlier sessions. There were no significant differences between groups 2 and 3 (p-values 0.1211, 0.1758 and 0.1375). Providing individuals with knowledge of results lowered their perceived workload, adding instructional feedback lowered this even further. These results demonstrate that KR is essential for efficient surgical skill acquisition. Individual coaching, a labor intensive proposition, reduces workload but has NO added beneficial effect on the speed of learning. These results provide a useful basis for developing efficient and cost effective surgical skills training curriculum. </div> <div> __________________________________________</div> <bR> <div> SAGES 2007: Education/Outcomes - P267</div> <div> <I><B>13.  THE CONSTRUCT VALIDITY OF COMPUTER-DERIVED PERFORMANCE METRICS FOR SELECTED SIMULATED LAPAROSCOPIC TASKS</B></I></div> <div> J. A Oostema MD, Matthew Abdel BS, Jon C Gould MD, University of Wisconsin School of Medicine and Public Health, Department of Surgery </div> <div> <B>Introduction</B></div> <div> A surgical skills assessment tool is said to demonstrate evidence of construct validity if users with more experience, and by inference more skill, perform better or more efficiently. Computer derived motion metrics such as smoothness (the number of times an instrument tip changes velocity during a task) and path length may be more sensitive measures of skill for a particular task than traditional metrics such as time. </div> <div> <B>Methods</B></div> <div> Twenty-four medical students (third year), 19 surgical residents (PG1-5), and 3 attending surgeons were asked to perform four different tasks 3 times in a hybrid computer-based physical laparoscopic trainer (ProMIS, Haptica Inc., Dublin). The 4 tasks in order of complexity were laparoscopic orientation (Task 1), object positioning (Task 2), sharp dissection (Task 3), and intra-corporeal knot tying (Task 4). Metrics recorded were time, path length, and smoothness. Laparoscopic operative experience for each user was quantified using case logs. Correlations were determined using regression analysis and ANOVA. </div> <div> <B>Results</B></div> <div> A statistically significant correlation was observed between experience and performance for all three metrics for tasks 2-4 (all p< 0.01). Smoothness was the only metric to correlate in the laparoscopic orientation task. Within tasks, time and smoothness correlate much more strongly with experience and to a similar degree. The strongest correlation was observed for the knot tying task (r2=0.60 for time and 0.59 smoothness). An r2=1.0 would represent a perfect correlation between experience and the specified metric. </div> <div> <B>Conclusions</B></div> <div> The computer-derived metrics measured by the hybrid trainer correlate with laparoscopic experience. Further study is necessary to determine if specific metrics are better indicators of actual skill. </div> <div> __________________________________________</div> <bR> <div> SAGES 2007 Education/Outcomes - P284</div> <div> <I><B>14.  ABSTRACT VIRTUAL REALITY TRAINING DEVELOPS CORE LAPAROSCOPIC SKILLS COMPARABLE TO EXPERIENCED LAPAROSCOPIC SURGEONS: RESULTS OF A PROSPECTIVE RANDOMIZED TRIAL COMPARING TWO VIRTUAL REALITY TRAINERS</B></I></div> <div> E. Matt Ritter MD, Elisabeth A Pimentel BA, Ryan E Earnest BS, Randy S Haluck MD, Mark W Bowyer MD, National Capital Area Medical Simulation Center, Uniformed Services University, Bethesda, Maryland / Department of Surgery, Pennsylvania State College of Medicine, Hershey, Pennsylvania </div> <div> <B>Introduction</B></div> <div> While simulation is becoming more widely accepted in surgical training, comparative trials on the training effectiveness of these simulators are lacking. We sought to compare the effectiveness of two abstract virtual reality trainers to train laparoscopic skills as assessed by the Fundamentals of Laparoscopic Surgery (FLS). We then compared the post training performance of the novice subjects with a group of experienced surgeons. </div> <div> <B>Methods and Results</B></div> <div> 20 novice medical students were recruited. Each subject performed a pre-test consisting of 3 FLS tasks - Peg Transfer (PT), Pattern Cut (PC) and Intracorporeal Suture (IS) - placed in the ProMIS augmented reality simulator (Haptica, Ireland). They were then randomized to train to predetermined levels of proficiency on 3 tasks of the Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) (Mentice, Sweden) or the Rapid Fire/Smart Tutor (RFST) (Verefi, Elizabethtown, PA). After reaching the proficiency levels, both groups then took a post test consisting of 3 trials of the same tasks used for the pre-test. Post test performance by both groups was then compared to a control group, composed of 10 experienced surgeons who had completed the same post test. </div> <div> MIST-VR and RFST groups demonstrated statistically significant improvement from the pre-test to the post test on all 3 FLS tasks (p < 0.0001). There was no significant difference in post test performance between the MIST-VR and RFST groups. When the simulation trained groups were compared to experienced controls there was no significant difference in performance with respect to PT. The experienced controls did significantly outperformed the MIST-VR group in PC (p<0.01) and IS (p<0.05), but differences between the experienced controls and the RFST group did not reach statistical significance. </div> <div> <B>Conclusion</B></div> <div> Simulation based training on either the MISTVR or the RFST simulator improves the skill level of novices as assessed by FLS. The post training skill level of these novices compares favorably with a group of experienced surgeons. Virtual Reality trainers, such as RFST and MISTVR, train fundamental laparoscopic skills equally and to a level comparable to a group of experienced practicing surgeons. </div> <div> __________________________________________</div> <bR> <div> SAGES07 Ergonomics/Instrumentation P307</div> <div> <I><B>15.  A COMPUTERIZED ANALYSIS OF STANDARD VERSUS HIGH DEXTERITY LAPAROSCOPIC INSTRUMENTATION IN TASK PERFORMANCE</B></I></div> <div> V K Narula MD,K M Reavis MD,D R Renton MD,D J Mikami MD,B J Needleman MD,J W Hazey MD,K E Hinshaw BS,W S Melvin MD, THE OHIO STATE UNIVERSITY HOSPITAL, CENTER FOR MINIMALLY INVASIVE SURGERY </div> <div> <B>Introduction</B></div> <div> Minimally invasive surgery is becoming the standard of care for the majority of abdominal procedures. Laparoscopic instrumentation is constantly undergoing improvements to give surgeons an advantage. Articulated instrumentation provides a distinct advantage in the field of robotic surgery. Applying the same principles to standard laparoscopic instrumentation could offer increased degrees of freedom to make complex laparoscopic tasks easier to perform. We utilized a novel computerized assessment system to objectively evaluate task performance comparing Standard and High Dexterity (HD) laparoscopic instrumentation.</div> <div> <B>Methods</B></div> <div> Advanced laparoscopic surgeons (2-12yrs experience) performed 3 unique task modules utilizing Standard and HD laparoscopic instrumentation (Novare Surgical Systems, Cupertino, CA). Performance was evaluated using a computerized assessment system (ProMIS, Dublin, Ireland) and results were recorded as time (sec), path (mm), and precision. Each surgeon had an initial training session followed by two testing sessions for each module. A Paired Student's T-Test was used to analyze the data. Results: Nine surgeons completed the study. Objective assessment of the data is presented in the table below. Module 1 was statistically significant, whereas Module 2 and 3 showed no difference in task performance with the HD instrumentation. </div> <div> <B>Conclusion</B></div> <div> HD instrumentation is in its infancy. Results showed no advantage using HD instrumentation. This could be due to the learning curve associated with new instrumentation and technology. With future developments in HD technology and training, the user interface will improve and may offer an advantage over standard laparoscopic instrumentation. </div> <div> __________________________________________</div> <bR> <div> SAGES 2006 Emerging Technologies P036 </div> <div> <I><B>16.  AUGMENTED REALITY SIMULATOR FOR HAND-ASSISTED LAPAROSCOPIC COLECTOMY </B></I></div> <div> Derek Young, Derek Cassidy, Fiona Slevin, Donncha Ryan, Haptica Ltd, Dublin, Ireland. </div> <div> Training in Hand-Assisted Laparoscopic Colectomy (HALC) has largely been done using cadavers and porcine models. These have drawbacks in terms of realism, logistics and lack of performance measurement. A Simulator would provide consistent instruction and practice and provide feedback on performance. However, given the range of instruments used in the procedure, and especially the use of a hand, pure virtual reality could not be considered as a solution.</div> <div> The ProMIS Augmented Reality simulator platform - by combining physical and virtual reality - enables interaction and tracking of real instruments with a physical model. And because of its technological approach (vision-tracking) also enables the hand to be tracked. In the new ProMIS HALC simulator, 3D models or graphical objects are overlaid on the physical model to provide instruction and guidance. For example, a 3D animation may be used to demonstrate how to complete a step; a graphical guideline `A - B' may be used to indicate a target area for dissection. ProMIS HALC measures surgical skill by gathering data on the movement of commercial laparoscopic instruments while completing a standardized task. The main performance metrics are time taken, total path length and economy of movement. Additionally metrics specific to a step are calculated to measure performance associated with a specific instruction in a specific region of the physical model. Following the simulated procedure, the user completes a self-assessment which contributes to the metrics for the full procedure. A full analysis is of performance is presented to the user on completion of the procedure and self-assessment. </div> <div> <B>Results</B></div> <div> Initial trials of the ProMIS HALC simulator indicate that practice on the simulator improves performance as measured by the metrics gathered by the simulator. While detailed validation studies remain to be done, initial indicators are that the HALC simulator represents an “unparalleled opportunity to practice, step by step, a Hand-assisted laparoscopic sigmoid resection” and “a huge step forward in surgical training”. </div> <div> __________________________________________</div> <bR> <div> May 2006 North of England Surgical Society Annual Registrar's Meeting <FONT SIZE="1" COLOR="#000000" FACE="Arial" ><I>Winner of the George Feggeter Gold Medal</I></FONT></div> <div> <I><B>17.  Laparoscopic Skills Acquisition: Is Psychometric Motion Analysis a Valid Assessment Tool?</B></I></div> <div> Pellen MGC<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1;2</FONT>, Barton JR<FONT SIZE="1" COLOR="#000000" FACE="Arial" >2</FONT>, Horgan LF<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1</FONT>, Attwood SE<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1</FONT> Northumbria Upper Gastrointestinal Team of Surgeons<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1</FONT>; University of Newcastle upon Tyne<FONT SIZE="1" COLOR="#000000" FACE="Arial" >2</FONT>, Newcastle upon Tyne, United Kingdom</div> <div> <B>Aims</B></div> <div> In an evolving climate of competency-based assessment, reliable and validated methods of objective skills assessment are required for trainee surgeons. We aimed to assess whether the ProMIS Simulator (Haptica, Dublin, Ireland) offers a method of assessing laparoscopic psychomotor performance. </div> <div> <B>Methods</B></div> <div> Volunteers comprising 17 experienced laparoscopists ( >100 laparoscopic cholecystectomies) and 38 medical students novices (no laparoscopic experience) performed 3 simulated tasks comprising virtual reality camera navigation, object  transfer and sharp dissection task (glove over balloon). A further group of 28 basic surgical trainees (experience limited to 1<FONT SIZE="1" COLOR="#000000" FACE="Arial" >st</FONT> assistant) attending BSS Courses were assessed on the same tasks before and after training in laparoscopic skills. Data metrics of time, smoothness and path length were measured via optical tracking of instrument movement. Objective observations of specific errors were also recorded.</div> <div> <B>Results</B></div> <div> Non-parametric analysis demonstrated experienced laparoscopists performed all 3 tasks significantly faster, smoother and with more economy of movement (p<0.05), excluding camera navigation path length. Experienced participants performed sharp dissection more accurately (p<0.01) although no difference in balloon puncture was seen. Repeat assessment of BSS Course Trainees showed significant improvements in simulator metrics (Paired T test, P<0.05), although smaller yet significant improvements in “untrained” student performance was also seen. </div> <div> <B>Conclusions</B></div> <div> Gross analysis of these metrics can distinguish between experience levels supporting construct validity of these simulator tasks. These results suggest potential for objectively measuring baseline skill level and response to training. Further work will examine the effect of interface familiarisation and defining target levels of performance in simulated tasks.</div> <div> __________________________________________</div> <bR> <div> EAES 2006, Poster</div> <div> <I><B>18.  Developing Psychometric Assessment of Laparoscopic Skills using the ProMIS Simulator</B></I></div> <div> Pellen MGC<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1;2</FONT>, Barton JR<FONT SIZE="1" COLOR="#000000" FACE="Arial" >2</FONT>, Horgan LF<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1</FONT>, Attwood SE<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1 </FONT>Northumbria Upper Gastrointestinal Team of Surgeons<FONT SIZE="1" COLOR="#000000" FACE="Arial" >1</FONT>; University of Newcastle upon Tyne<FONT SIZE="1" COLOR="#000000" FACE="Arial" >2</FONT>, United Kingdom</div> <div> <B>Aims</B></div> <div> Reliable and validated methods of objective skills training and assessment are required for trainee surgeons. The ProMIS Simulator (Haptica, Ireland) potentially offers a method of assessing laparoscopic psychomotor performance. We present initial data from our Centre and Royal College of Surgeons Basic Surgical Skills (BSS) Courses. </div> <div> <B>Methods and results</B></div> <div> Volunteers comprising 17 experienced laparoscopists (>100 laparoscopic cholecystectomies) and 38 medical students novices (no laparoscopic experience) were assessed on a complex sharp dissection task (glove over balloon). A further group of 28 basic surgical trainees (experience limited to 1<FONT SIZE="1" COLOR="#000000" FACE="Arial" >st</FONT> assistant) attending BSS Courses were assessed on the same task before and after training in laparoscopic skills.  Data metrics of time, smoothness and path length were measured via optical tracking of instrum-ent movement as well as observations of specific errors. </div> <div> Data analysis (ANOVA) demonstrated experienced laparoscopists performed target dissection at least 50% faster, smoother and with more economy of instrument movement than students (p<0.05). Experienced participants performed sharp dissection more accurately (p<0.01) although no difference in balloon puncture frequency was seen. Similarly significantly better performance over trainees was demonstrated. Trainees showed only significantly smoother instrument handling when compared to students, possibly reflecting greater baseline dexterity in this selective group. Repeat assessment following course training showed significant improvements in all metrics by 32-40% (Paired T test, P<0.05). Whilst significant improvements were also demonstrated in repeat assessment of the untrained student group, these were less marked (15-18%).  </div> <div> <B>Conclusions</B></div> <div> The gross analysis of these metrics can distinguish between experience levels supporting the construct validity of this simulator task. These results suggest a potential role for objectively measuring baseline skill level and response to training in distinct psychomotor challenges. Further work in progress is examining the effect of interface familiarisation and repeated task performance on novice learning curves and defining target levels of performance in a range of simulated tasks. </div> <div> __________________________________________</div> <bR> <div> Surg Endosc (May 2006) 20: 900-904</div> <div> <I><B>19.  Construct validation of a novel hybrid surgical simulator</B></I></div> <div> D. Broe, P. F. Ridgway, S. Johnson, S. Tierney, K. C. Conlon Department of Surgery, Professorial Surgical Unit, Level 4, The Adelaide and Meath Hospital, incorporating the National Children's Hospital, Tallaght, Dublin 24, Ireland</div> <div> <B>Background</B></div> <div> Simulated minimal access surgery has improved recently as both a learning and assessment tool. The construct validation of a novel simulator, ProMis, is described for use by residents in training.</div> <div> <B>Methods</B></div> <div> ProMis is a surgical simulator that can design tasks in both virtual and actual reality. A pilot group of surgical residents ranging from novice to expert completed three standardized tasks: orientation, dissection, and basic suturing. The tasks were tested for construct validity. Two experienced surgeons examined the recorded tasks in a blinded fashion using an objective structured assessment of technical skills format (OSATS: task-specific checklist and global rating score) as well as metrics delivered by the simulator.</div> <div> <B>Results</B></div> <div> The findings showed excellent interrater reliability (Cronbach<FONT SIZE="2" COLOR="#000000" FACE="AdvPSMP1" >_</FONT>s <FONT SIZE="2" COLOR="#000000" FACE="AdvPSMP10" >a </FONT>of 0.88 for the checklist and 0.93 for the global rating). The median scores in the experience groups were statistically different in both the global rating and the task-specific checklists (<FONT SIZE="2" COLOR="#000000" FACE="AdvTimes-i" >p </FONT>< 0.05). The scores for the orientation task alone did not reach significance (<FONT SIZE="2" COLOR="#000000" FACE="AdvTimes-i" >p </FONT>= 0.1), suggesting that modification is required before ProMis could be used in isolation as an assessment tool.</div> <div> <B>Conclusions</B></div> <div> The three simulated tasks in combination are construct valid for differentiating experience levels among surgeons in training. This hybrid simulator has potential added benefits of marrying the virtual with actual, and of combining simple box traits and advanced virtual reality simulation.</div> <div>  <FONT SIZE="2" COLOR="#336699" FACE="Arial" >__________________________________________</FONT></div> <bR> <div> Surgical endoscopy ISSN: 0930-2794 (Paper) 1432-2218 (Online)</div> <div> <I><B>20.  Construct validation of the ProMIS simulator using a novel laparoscopic suturing task</B></I></div> <div> K. R. Van Sickle1, D. A. McClusky III1, A. G. Gallagher and C. D. Smith1 </div> <div> <B>Background</B></div> <div> The use of simulation for minimally invasive surgery (MIS) skills training has many advantages over current traditional methods. One advantage of simulation is that it enables an objective assessment of technical performance. The purpose of this study was to determine whether the ProMIS augmented reality simulator could objectively distinguish between levels of performance skills on a complex laparoscopic suturing task.</div> <div> <B>Methodology</B></div> <div> Ten subjects - five laparoscopic experts and five laparoscopic novices - were assessed for baseline perceptual, visio-spatial, and psychomotor abilities using validated tests. After three trials of a novel laparoscopic suturing task were performed on the simulator, measures for time, smoothness of movement, and path distance were analyzed for each trial. Accuracy and errors were evaluated separately by two blinded reviewers to an interrater reliability of >0.8. Comparisons of mean performance measures were made between the two groups using a Mann-Whitney U test. Internal consistency of Pro<FONT SIZE="2" COLOR="#000000" FACE="Trebuchet MS Greek" >MIS measures was assessed with coefficient á.</FONT></div> <div> <B>Results</B></div> <div> The psychomotor performance of the experts was superior at baseline assessment (p < 0.001). On the laparoscopic suturing task, the experts performed significantly better than the novices across all three trials (p < 0.001). They performed the tasks between three and four times faster (p < 0.0001), had three times shorter instrument path length (p < 0.0001), and had four times greater smoothness of instrument movement (p < 0.009). Experts also showed greater consistency in their performance, as demonstrated by SDs across all measures, which were four times smaller than the novice group. Observed internal consi<FONT SIZE="2" COLOR="#000000" FACE="Trebuchet MS Greek" >stency of ProMIS measures was high (á = 0.95, p < 0.00001).</FONT></div> <div> <B>Conclusions</B></div> <div> Preliminary results of construct validation efforts of the ProMIS simulator show that it can distinguish between experts and novices and has promising psychometric properties. The attractive feature of ProMIS is that a wide variety of MIS tasks can be used to train and assess technical skills.</div> <div> __________________________________________</div> <bR> <div> EAES 2004 Abstract nr.: O207 </div> <div> <I><B>21.  </B></I><I><B>Relationship Between Motion Analysis, Time, Accuracy, and Errors During Performance of a Laparoscopic Suturing Task on an Augmented Reality Simulator</B></I></div> <div> Author: D.A.M. McClusky, Emory University School of Medicine, Atlanta Georgia, United States of America. Co-author(s): K. Van Sickle, Emory University School of Medicine, Atlanta Georgia, United States of America</div> <div> A.G. Gallagher, Emory University School of Medicine, Atlanta Georgia, United States of America</div> <div> <B>Background</B></div> <div> Time, efficient movement, accuracy, and safety are reliable and discriminative metrics of proficiency during virtual reality and box-trainer based minimally invasive surgical (MIS) training. The role these metrics may serve during more advanced skills training are not well understood. Using a novel augmented reality simulator, we sought to gain an understanding of the relationship between these metrics during an advanced MIS suturing task. Methods: Eleven subjects completed 3 trials of a suturing task designed for a box-trainer and adapted for the ProMIS (Haptica, Dublin, Ireland) simulator. Time, tool path, and smoothness of movement were assessed using computer algorithms. Measures of accuracy during suture placement and errors in performance were assessed by two blinded reviewers trained to assess performance with inter-rater reliability > 0.8. A Pearson's correlation coefficient was used to assess the strength of the relationship between ProMIS metrics and suturing task performance. Results: Of the ProMIS metrics, time correlated with tool path distance and smoothness of movement in three trials (range 0.914 - 0.957, p < 0.0001). When the suturing task was analyzed, accuracy and error score demonstrated an equally strong relationship (range -0.726 - -0.84, all p < 0.0001). Combining all metrics, path distance correlated strongest with accuracy (2 trials significant, range -0.67 - -0.93), and error score (3 trials significant, range 0.54 - 0.61). Smoothness of movement significantly correlated with accuracy in 2 trials (range -0.63 - -0.88), and time correlated with error score in 2 trials (range 0.56 - 0.60). </div> <div> <B>Conclusion</B></div> <div> Metrics based on movement efficiency and time, and those based on task accuracy and error scores strongly correlate when grouped independently. At this time, a proficiency curriculum should incorporate both forms of analysis, however further validation work is needed to replicate these findings and give further insight into how ProMIS metrics relate to real-world performance. </div> <div> __________________________________________</div> <bR> <div> EAES 2004 abstract nr.: O208 </div> <div> <I><B>22.  </B></I><I><B>Psychomotor Skills Assessment in Practicing Surgeons Performing Advanced Laparoscopic Procedures II: Demographics and Performance Profiles</B></I></div> <div> Author: A.G. Gallagher, Emory University, Atlanta, United States of America. Co-author(s): C.D. Smith, Emory University, Atlanta, United States of AmericaR.M. Satava, University of Washington, Seattle, United States of America</div> <div> <B>Background</B></div> <div> This study reports on the objectively assessed psychomotor performance of minimally invasive surgeons on a box-trainer and a virtual reality (VR) task as a function of handedness, gender, sight corrected status, and age. Methods: Two hundred and ten surgeons attending the 2001 annual meeting of the American College of Surgeons (ACS) in New Orleans who reported having completed more than 50 laparoscopic procedures participated. Subjects completed a box-trainer laparoscopic cutting task and a similar virtual reality task twice. Demographic and laparoscopic experience data was also collected. Results: There were no significant differences between subjects performance on either tasks in terms of handedness, gender or whether they were sight corrected or not. A clear and consistent linear trend emerged in terms of age. Older subjects (ages 60 - 69) performed significantly worse than younger subjects (ages 30 - 39, 40 - 49) on the box-trainer task for correct incisions (13.1 Vs 19.3, p < 0.008) and incorrect incisions (12.3 Vs 2.5, p > 0.05). They also performed worse on the VR task for time (132 Vs 71, p < 0.05), error (99 Vs 41, p < 0.05) and economy of movement (22.8 Vs 11.7, p < 0.05). Conclusions: Increasing age was found to be associated with a decline in objectively assessed psychomotor performance on two well validated laparoscopic tasks. </div> <div> <B>Acknowledgements </B></div> <div> This study was supported with grants from the ACS, SAGES, SLS, TATRC, and Emory University Endosurgery Unit. </div> <div> __________________________________________</div> <div> <I><B> </B></I></div> <div> Society of Laparoendoscopic Surgeons 2004 Abstract no. 4111MUL </div> <div> <I><B>23.  Evaluation of a laparoscopic video trainer with in-built measures of performance. </B></I></div> <div> Authors Hance J (MBBS), Aggarwal R (MBBS), Undre S (MBBS), Patel H, Selvapatt N, Darzi A (MD).</div> <div> <B>Background</B></div> <div> It is widely accepted that training in laparoscopic surgery must begin outside the operating theatre. Objective assessment of performance is fundamental in order to provide formative feedback during training, though current laparoscopic video trainers lack this ability. The ProMIS surgical simulator retains the benefits of a video trainer, by using original laparoscopic instruments and tactile tasks, but in addition generates objective measures of performance. The aim of this study is to evaluate whether this new simulator can assess surgical skill.</div> <div> <B>Methods</B></div> <div> A total of 32 subjects of differing laparoscopic experience completed three tasks on the simulator. The subjects were 12 novice surgeons (Laparoscopic Cholecystectomy, LC=0), 11 trainees (LC=5-50) and 9 experts (LC>100). The three tasks completed were `object positioning', `clip and cut' and `sharp dissection'. Using video tracking technology the simulator generates objective measures of performance, namely instrument tip `path-length', `smoothness' and time. Data was analysed using non-parametric tests.</div> <div> <B>Results</B></div> <div> There were significant differences between the performance of all three groups for all parameters on the `object positioning' (p=0.001) and `sharp dissection' (p=0.001) tasks. For the `clip and cut' exercise there was no significant difference between experts and trainees (p=0.489), although novices were significantly worse (p=0.005).</div> <div> <B>Conclusions</B></div> <div> ProMIS demonstrates the ability to provide objective measures of laparoscopic performance on a video trainer, using standardized tasks. These measures can be used to provide objective feedback to junior surgeons and indicate when individuals can progress on to the next stage of their training. </div> <div> __________________________________________</div> <bR> <div> EAES 2004 abstract nr.: O012 </div> <div> <I><B>24.  Validation of a Novel Hybrid Surgical Simulator</B></I></div> <div> Author: D. Broe, AMNCH, Dublin, Ireland. Co-author(s): P.F. Ridgway, S. Johnson, C. Tierney, K.C. Conlon</div> <div> <B>Introduction </B></div> <div> Simulated minimal access surgery has improved both as a learning and assessment tool in recent times. We describe the pilot validation of a novel hybrid simulator, Haptica TM, for use in assessment of residents in training. </div> <div> <B>Methodology </B></div> <div> The device allows both virtual (VR) and actual reality laparoscopic tasks to be designed. A pilot group of surgical residents ranging from novice to expert completed three standardised tasks (Orientation, dissection and complex pick/ place). There was no antecedent training on the model. Tasks were tested for construct validity (ability of the test to predict experience level). Two experienced surgeons examined the recorded tasks in a blinded fashion using the Objective structured assessment of technical skills format* (OSATS: Task specific checklist and global rating). </div> <div> <B>Results </B></div> <div> There was good inter examiner reliability (Cronbach's alpha: 0.88 checklist and 0.97 global rating). The median scores in the experience groups where statistically different in both global rating (97.14 versus 32.85; p=0.008 Mann-Whitney U) and task (dissection and pick/place) specific checklists (p<0.05). Scores for the orientation task alone did not reach significance (p=0.1) suggesting it required modification before it could be used in isolation as an assessment tool. </div> <div> <B>Conclusions </B></div> <div> The three simulated tasks in combination are construct valid for differentiating experience levels among surgeons in training. This hybrid simulator has added benefits to models available as it seamlessly marries the virtual with actual, combining traits from simple box and advanced VR simulation. </div> <bR> <bR> <div>  <A NAME="sages_2008_p155"><IMG BORDER="0" SRC="1x1.gif" HEIGHT="1" ALIGN="bottom" WIDTH="1" HSPACE="0" VSPACE="0" ></A>SAGES 2008 P155</div> <div> <I><B>VALIDATION OF LAPAROSCOPIC SURGICAL SKILLS TRAINING OUTSIDE THE OPERATING ROOM: A LONG ROAD,</B></I></div> <div> Nancy J Hogle MS,Lily Chang MD,Vivian E Strong MD,Akuezunkpa O Ude MD,Mika Sinaan MD,Robert Bailey MD,Dennis L Fowler MD, College of Physicians and Surgeons, Columbia University, New York, NY, Virginia Mason Medical Center, Seattle, WA, Memorial Sloan-Kettering Cancer Center, New York, NY, University of Washington, Seattle, WA, University of Miami Hospital, Miami, FL</div> <div> <B>Introduction</B></div> <div> Surgical skills training outside the OR is beneficial.The best methods have yet to be identified. In a series of three studies, we attempted unsuccessfully to document</div> <div> predictive validity of simulation training.</div> <div> <B>Methods</B></div> <div> In each study we used Global Operative Assessment of Laparoscopic Skills (GOALS) to evaluate operative performance 1.) Prospective, randomized, multicenter trial to assess performance of junior residents in the OR after training to criteria or not training on a laparoscopic simulator [LapSim Surgical Science Ltd., Goteborg, Sweden]). 2.) Retrospective review of operative performance of junior residents operating as primary surgeon before and after implementation of a laparoscopic skills training curriculum. 3.) Intern operative performance of laparoscopic cholecystectomy in a porcine model before and after training on a simulator.</div> <div> <B>Results</B></div> <div> We failed to document predictive validity in all three studies. In each study, no difference was found between trained and untrained residents in GOALS domains of depth perception, bimanual dexterity, efficiency, tissue handling, and autonomy.</div> <div> <B>Conclusions</B></div> <div> We found a lack of correlation between three types of training outside the operating room and improved operative performance. Possible explanations include.) Too few subjects; 2.) Introducing training too late on the learning curve; and 3.) Training criteria too easy. Awareness of these failures can improve design of future studies.</div> <bR> <div> SAGES 2008 P169</div> <div> <I><B>SPEED-ACCURACY TRADEOFF IN VR SIMULATOR WITH HAPTIC FEEDBACK</B></I></div> <div> Audrey Bell BS,Ethan Mandelup,Steven D Schwaitzberg MD,Caroline G Cao PhD, Tufts University, Cambridge Health Alliance</div> <div> Virtual reality (VR) technology is expected to play an increasingly important role in laparoscopic surgical training in the future. Haptic feedback in VR training remains a debated topic amongst researchers and practitioners. This study was designed to test the hypothesis that haptics would improve the control of force application in a virtual pick-and-place task, especially when the task index of difficulty (ID) is high. Using a laparoscopic tool connected to a haptic device, subjects transported a ball into a box of varying sizes (W) at different distances (A) in a virtual environment. Three indices of difficulty (ID) were used: ID1 (easy), ID2 (moderate), and ID3 (difficult). ID was determined using Fitts' Law (log2 (2A/W)). Ten naive subjects participated in a 2-haptics x 3-ID repeated measures design experiment, performing three trials per condition. Dependent measures were time, maximum and minimum applied force, and smoothness of tool motion (force and smoothness were tracked in three dimensions). Data analysis results based on analysis of variance and t-tests showed that 1) significantly lower forces were applied when performing tasks with haptic feedback than without (p < 0.001); 2) significantly lower forces were applied in the ID1 condition than in the ID3 condition in the y and z dimensions (p=0.025, p < 0.001, respectively); 3) maximum force application in the z dimension was significantly lower with haptic feedback, the difference being more pronounced in the easy task than in the other more difficult tasks (p = 0.015); 4) tool motion was significantly smoother in the easy task (ID1) than in the difficult task (ID3) (p < 0.001). This was true for all three dimensions, with or without haptics. Time to task completion was not significantly different, except in the transport phase of the difficult task (ID3). These results indicate that subjects were better able to control force application with haptics. </div> <div> The similarity in the time measure among task conditions suggests a strategy favoring speed over force control in the absence of haptic feedback. Therefore, it can be concluded that haptic feedback is important in improving the accuracy of force application while maintaining the speed (or efficiency) of task performance. The benefit of haptic feedback in VR training simulators for surgical skill acquisition remains to be investigated.</div> <bR> <div> SAGES 2008 P170</div> <div> <I><B>LEARNING CURVE IN LAPAROSCOPIC SUTURING WITH HAPTIC FEEDBACK</B></I></div> <div> Mi Zhou MS,Susan Tse BS,Alex Derevianko MD,Daniel B Jones MD,Steven D Schwaitzberg MD,Caroline G Cao PhD, Tufts University, Beth Israel Deaconess Medical Center, Cambridge Health Alliance </div> <div> Simulators with haptic feedback may not be justified if its benefit is not applicable in surgical skills training. We hypothesized that trainees would reach the first plateau in their learning curve earlier when trained using simulators with haptic feedback, compared with those who trained without haptic feedback. Fifteen novices were randomly assigned to either the haptics group (n=7) or the no-haptics group (n=8). They learned to execute an intracorporeal suture with a square knot over a period of three weeks (1 hour-long session per day, 6 days a week, for 3 consecutive weeks). Learning curves were plotted using task completion time. A trend analysis was conducted based on a predicted mathematical model (S=Smax(1-e^(-slope x Session#)) of maximum speed (Smax) and slope (slope) of the learning curve for each successive session (Session#). Subjective workload assessment for each session was obtained using a standardized NASA TLX workload questionnaire and analyzed using t-test. Results showed that performance variability in each session for subjects in the haptics condition was significantly lower (p<0.02) than in the no-haptics condition. Subjects were significantly faster in the haptics condition than the no haptics condition (p<0.001 for last finished trial in each session; p<0.001 for all trials). Predicted maximum speed was also significantly faster (p<0.04 for last finished trial in each session, p<0.03 for all trials) with haptics than without haptics. However, the slope of the learning curves in the two haptic conditions were not significantly different (p<0.63 for last finished trial in each session, p<0.58 for all trials). On average, both groups reached the first plateau after 5 days of training. Workload was not different with or without haptics. Even though the durations of the skill acquisition phase in learning a complex surgical task were similar, those who learned with haptics were able to perform the task faster and more consistently. </div> <div> Our results support incorporating haptics in the early stages of surgical skills training. </div> <bR> <div> SAGES 2008  P173</div> <div> <I><B>PROFICIENCY-BASED FLS SKILLS TRAINING RESULTS IN DURABLE PERFORMANCE IMPROVEMENT AND UNIFORM CERTIFICATION PASS RATE</B></I></div> <div> Mouza T Goova MD,E. Matt Ritter MD,Seifu T Tesfay MS,Elisabeth A Pimentel BA,Robert Hartzler BS,Daniel J Scott MD, Southwestern Center for Minimally Invasive Surgery, UT Southwestern Medical center, Dalls, Texas. Uniformed Services University, Bethesda, Maryland </div> <div> <B>Introduction: </B></div> <div> We have previously documented a 100% certification pass rate immediately following a proficiency-based skills training curriculum for the Fundamentals of Laparoscopic Surgery (FLS) program. The purpose of this study was to determine the durability of skills acquired after initial training. </div> <div> <B>Methods: </B></div> <div> Novice medical students (n=21) were enrolled in IRB-approved protocols at 2 institutions. As previously reported, all participants successfully completed a structured proficiency-based training curriculum by practicing the 5 FLS tasks in a distributed fashion over a 2-month period. Pre and post-testing were conducted and standard FLS testing metrics were used. Participants returned for repeat posttesting at 6 and 13 months following initial curriculum completion. Participants had no additional skills lab training and had minimal clinical laparoscopic exposure. Data are reported as mean ± s.d. (p< 0.05 considered significant). </div> <div> <B>Results: </B></div> <div> n= Score Pass Rate Pre-Test 21 125.8 ± 75.1 0% Post-Test 21 468.4 ± 24.3 100% 6 Months 19 436.1 ± 28.7 100% 13 Months 15 425.5 ± 34.9 100% Trainee performance significantly improved (p <0.001) after initial training. Participants retained a very high level of performance at 6 months (93% retention of post-test score) and 13 months (91% retention of post-test score). Performance at both retention testing intervals was sufficient to pass the certification exam (270 cutoff score for passing) with a comfortable margin. There were no significant differences in performance at any time points between the two institutions. </div> <div> <B>Conclusions: </B></div> <div> The proficiency-based FLS skills curriculum reliably results in a high level of skill retention, even in the absence of ongoing simulator-based training or clinical experience. This curriculum is suitable for widespread implementation.  </div> <bR> <div> <A HREF="promis.htm" TARGET="_top" TITLE="ProMIS surgical simulator">Back to ProMIS main page >>></A></div> <bR> </td> </tr></table></body>