1.3
Simulation in Urology
Hamid Abboudi1,2, Muhammad Shamim Khan1,2,3, Prokar Dasgupta1,2, and Kamran Ahmed1,2
1 MRC Centre for Transplantation, King’s College London, King’s Health Partners, London, UK
2 Department of Urology, Guy’s & St. Thomas’ Hospital, London, UK
3 Guy’s & St. Thomas’ Hospital, NHS Foundation Trust, King’s College, London, UK
Abstract
Surgical simulation can be traced as far back as ancient India, where the first notion of using cadavers and synthetic models for practice prior to surgery spawned. In modern surgical practice, the advancements in science and technology have led to the introduction of new surgical technologies as well as more challenging surgical procedures for a more complex patient population. Limitations in surgeons’ working hours and increased emphasis on utilising costly resources more efficiently have posed challenges to surgical educators. Fundamentally and most importantly, simulation training provides a platform for safe surgical education without compromising patient safety. Simulation has the potential to provide a time‐efficient, cost‐effective, and safe method of training. Furthermore simulated scenarios provide a means to develop nontechnical, safe, and effective teamwork skills that have been shown to influence patient outcomes. As such, there is growing interest in establishing simulation curriculums and assessment methodology with associated integration within urology training programmes to aid in the acquisition of both technical and nontechnical skills required for safe, modern urological practice.
Keywords: simulation training; virtual‐reality simulation
1.3.1 Introduction
The traditional Halsted model of surgical training, ‘see one, do one, teach one’, has become increasingly difficult to implement. The introduction of new surgical technologies has been accompanied by more challenging surgical procedures for a more complex patient population. In addition, reduction in working hours, fear of litigation, outcome reporting, and pressures to utilise costly operating room (OR) resources more efficiently have led to significant changes in the curricula for surgical training.
Recognising modern changes to surgical practice, surgical simulation has evolved over the last two decades as an education tool that seeks to bridge the gap between the initial phases of a procedural learning curve and the OR. The aim of simulation is to aid the acquisition of specific skills and improve both competence and confidence. Simulation training provides a safe environment for repetitive practice of different surgical skills free from risk of harm to patients and a means of providing trainees with immediate feedback and the opportunity to train to a predetermined expert proficiency. This is exemplified by the theory of ‘deliberate practice’ [1], which dictates that to achieve ‘expertise’, multiple repetitions of a skill and the provision of constructive feedback to ensure the skill is being learned correctly is required. Surgical simulation provides the trainee with an opportunity to repeatedly perform a specific skill, or set of skills, in a low‐risk environment away from patients, thus allowing for a safe environment where mistakes can occur. The trainee is then able to learn how to correct mistakes, problem solve, and develop the confidence to deal with complications when they occur in real‐world settings without it being experienced for the first time. Furthermore, simulation allows the trainer to provide timely and constructive formative or summative feedback based on trainee performance, ensuring acquisition of competency.
1.3.2 History of Simulation in Medicine
Surgical simulation has been around long before complex, high‐fidelity, virtual‐reality devices were developed. Traditional basic surgical skill training was done on oranges to practise suturing skills or on a table leg for surgical knots. Currently, synthetic models have greatly improved in look, feel, and tactile behaviours, and many medical device companies now produce a range of general and specialist surgical skills models from high‐fidelity materials. However, the earliest contribution to the field of surgical simulation was by the ancient Indian surgeon, Sushruta Samhita [2]. He advocated the use of cadavers and synthetic models for practice prior to surgery. He taught technical skills using experimental models that included probing on worm‐eaten wood and incising vegetables, such as watermelons and cucumbers. The first modern skills ‘simulator’ was produced in 1928 when Edwin Link created a pilot trainer, which allowed trainee pilots to learn important aeronautical manoeuvres whilst remaining on the ground, making learning to fly cheaper, safer, and more accessible. By the end of World War II in 1945, there were 6271 Link Trainer simulators in circulation [3]. One of the earliest examples of medical simulation was a foetal and human pelvis bone model used by Madame du Coudray to train midwives in eighteenth‐century France [4]. In 1960, the first medical simulator, a resuscitation training manikin was developed when a girl drowned in the River Seine in Paris. This model, initially created by the toy maker Asmund S. Laerdal, has now become the Resusci Anne, which is still widely used today by first aid and life support courses to teach mouth to mouth resuscitation and airway and breathing assessments. In the 1960s, Abrahamson and Denson developed the Sim One, resuscitation trainer. This could simulate breathing, heart rate, pulses, and blood pressure and had eyes and a mouth that could open and close. Further developments in high‐fidelity simulators had to wait until the 1980s. The Comprehensive Anaesthetic Simulation Environment (CASE) and the Gainesville Anaesthetic Simulator (GAS) utilised concepts developed by the aviation industry to teach anaesthetic crisis management using team‐based simulated scenarios. Modern high‐fidelity training focuses on the technical and nontechnical skills (NTS) required to make a safe, effective surgeon and also add a realistic environment for simulations.
In 1989, the US computer scientist Jaron Lanier took the concept of virtual, describing something that appeared to exist without actually existing, and combined it with the term ‘reality’, meaning something that is observable and measurable as opposed to the imaginary or delusional [5]. Virtual‐reality (VR) surgical simulators began to grow in the 1990s beginning with lower limb reconstruction simulator (Delph et al.) to the late 1990s where a range of operations could be catered for by this technology. Evidence for its safety and effectiveness began to emerge; in particular, Seymour et al. showed that trainees could significantly reduce their procedure times to perform a cholecystectomy by 29% and were five times less likely to injure the gallbladder after using a VR trainer [6]. In 2004, the US Food and Drug Administration approved the introduction of the VR carotid stent procedure into their training programme. For the first time, trainees had to prove themselves on a competency‐based task to be allowed to perform a human procedure [7]. With the development of web‐based simulations, such teaching modalities are becoming cheaper and more accessible. However, the future of surgical education does not solely lie with one form of simulation, but in the combination of different modalities integrated into the curriculum.
1.3.3 Simulation Modalities
Surgical simulator training can be separated into four broad categories: physical (mechanical) simulators, animal, cadaveric, and virtual reality (VR). Physical simulator models range from part task trainers (e.g. laparoscopic box trainers) to procedure‐specific trainers (e.g. ureteroscopy trainers). These are often more suitable to novice or intermediate trainees. Because of the lack of clinical variability and the inability to provide individualised proficiency‐based variations in complexity, inanimate surgical simulation models often decrease in utility for advanced‐level training. Animal and cadaveric surgical training has more significant cost and ethical and legislative implications; however, this is offset by providing improved contextual fidelity and opportunities to practise a whole procedure. To maximise this finite resource, it is paramount that animal‐model training forms part of a well‐designed, comprehensive curriculum with clear learning objectives.
VR simulators have been the mainstay of training in aviation and nuclear industries for decades, being utilised for teaching, evaluation, certification, and recertification purposes. Tasks are performed on a computer‐based platform and artificially generated virtual environment. Improvements in computer processing have led to more realistic and sensitive VR simulators, which are now capable of providing statistical feedback on the surgeon’s performance, a quality that is not shared by mechanical or cadaveric simulator trainers. This feature also eliminates the need for expert faculty present during the entirety of training and promotes a self‐directed learning environment. The improvements in modern software allow for clinical variations to be built into the simulator, improving the training content delivered. VR simulators have been validated as a training tool with shown educational impact and content, context, and construct validity [8]. The main challenge to programme directors has been justifying the steep costs because little data currently shows any cost effectiveness.
Simulation training also encompasses simulated clinical scenarios such as mock OR team‐training sessions. These high‐fidelity facilities are used in the development and assessment of crisis management skills and nontechnical skills (NTS) training such as leadership, management, and communication. With the majority of surgical errors as a result of poor communication [9], increasing attention has been directed towards simulated OR facilities to improve pre‐, intra‐, and postoperative communication between the surgical multidisciplinary team.
1.3.4 Simulation in Urology
Urology remains at the forefront of surgical innovation, particularly in the field of minimally invasive surgery, which includes endoscopy, laparoscopy, and robotics. As the nature of practice is changing, so are the definitions and requirements of competency [10]. Fundamental to this change in practise is patient safety, which must not be compromised. A competent urologist must have a plethora of attributes to practice in the modern surgical environment including excellent technical, team‐working, communication, leadership, and decision‐making skills. Urology has embraced simulation training, both in its development and application.
1.3.5 Endourology Simulation
Over the last few decades, the applications of endoscopic techniques have expanded. The development of ureterorenoscopy (URS) and percutaneous nephrolithotomy (PCNL), superseding open stone surgery, and novel bladder outflow obstruction (BOO) techniques challenging the gold standard transurethral resection of the prostate (TURP), such as Holium enucleation and Green light laser, are growing. This has necessitated the concomitant need for training and qualification in the established and newer techniques. In response to this demand, there have been a range of endourological simulators focusing on core procedures, including urethrocystoscopy, URS, transurethral resection of a bladder tumour (TURBT), TURP, and percutaneous access (Table 1.3.1). The most extensively studied models are the URO Mentor (Simbionix); a computer‐based VR model offering semi‐rigid and flexible urethrocystoscopy and URS modules and the Uro‐Scopic trainer (Limbs & Things, Bristol UK), a high‐fidelity bench model permitting use of real OR instruments in urethrocystoscopy and URS. Three prospective randomised controlled trials have supported these two ureteroscopy simulators in terms of face, content, and construct validity, with training on the URO Mentor showing improvements in surgical performance in the OR [11–14]. There are only two available simulators for TURBT training, with few validation studies. This may be related to difficulties in developing simulators that can encompass the variation of tumours in patients. Eight TURP, nine URS, eight urethrocystoscopy, and nine percutaneous access models have been developed. Universities have developed the majority of these, in particular the TURP and percutaneous access models, while ureteroscopy simulation has received greater interest from the industry.
Table 1.3.1 Endourology simulators and there validation.
Procedure | Simulator type | Description of simulator | Face, content, and construct validity demonstrated | Evidence of skills transfer to the OR |
Urethrocystoscopy | VR | URO Mentor (Symbionix) | Yes | |
URO trainer | Yes | |||
George Washington University Medical Center, Washington, DC, US | Content only | |||
Storz | Content and Construct | |||
Bench | Limbs & Things | No | ||
Mediskills | No | |||
Organic: pumpkin, green pepper | Southern Illinois University School of Medicine, Springfield, IL, US | No | ||
Animal, porcine liver in pumpkin | Southern Illinois University School of Medicine, Springfield, IL, US | No | ||
Ureterorenoscopy | VR | URO mentor (Symbionix) | Yes | Yes |
Duke University Medical Center, Durham, NC, US | ||||
Bench | Limbs and things | Yes | ||
Mediskills | Yes | |||
University Toronto model | Yes | |||
Animal, porcine kidney | Klinikum Coburg, Germany | No | ||
Southern Illinois University School of Medicine, Springfield, IL, US | No | |||
University of California, Irvine, CA, US | No | |||
Human, uterus | University Medical Center, Shreveport, LA, US | No | ||
TURP | VR | Pelvic vision | Yes | Yes |
UW TURP trainer | Yes | |||
University College, London, UK | Yes | |||
University of Washington, Seattle, WA, US | No | |||
University Hospital Linkoping, Sweden | Content and Construct | |||
Imperial College of Science, Medicine, London, UK | No | |||
Bench | Limbs & Things | No | ||
Animal, canine cadaver | University of California, CA, US | No | ||
TURBT | VR | URO Mentor (Symbionix) | Yes | |
Storz | Content and Construct | |||
Bench | Limbs and things | No | ||
PCNL | VR | PERC mentor (Symbionix) | Yes | |
Bench | Limbs and things | No | ||
Mediskills | No | |||
University Hospital of Tours, Tours, France | No | |||
Animal, porcine kidney | Petropolis School of Medicine, Rio de Janeiro, Brazil | No | ||
Klinikum Coburg, Germany | No | |||
Southern Illinois University School of Medicine, Springfield, IL, US | No | |||
Medicine, Springfield, IL, US | No | |||
University Hospital Mannheim, Germany | No |
PCNL, percutaneous nephrolithotomy; TURBT, transurethral resection of a bladder tumour; TURP, transurethral resection of the prostate; VR, virtual reality.
Limitations in endourological‐simulator validity studies relate to small participant numbers and the use of time as the objective parameter to assess competency. Although time is an easy parameter to measure, it is not necessarily a sign of competency, and more focus is required to objectively measure outcomes and the various steps within a procedure. For example, TURP parameters should include volume of prostate resected, blood loss, and associated injuries such as ureteric orifice or sphincter damage. With the majority of endourological simulator literature in the form of descriptive texts as opposed to validity studies, coupled with the lack of randomised controlled studies, none of the urology training models described and researched can be said to have proven validity for use in specialty training. There is also a lack in the development and research of simulation devices that focus on novel bladder outflow obstruction procedures such as Green Light Laser and Holium Enucleation.
1.3.6 Laparoscopy Simulation
There are two laparoscopic urology simulators that have undergone validation studies: Procedieus MIST VR (Mentice) and the LAP mentor (Symbionix) (Table 1.3.2). Currently these platforms have only been developed and researched for training in laparoscopic nephrectomy (LN). Wijn et al. assessed the construct validity of the LN VR simulator (Mentice, Gothenburg, Sweden) [15].
Table 1.3.2 Laparoscopy simulators and there validation.
Procedure | Simulator type | Description of simulator | Face, content, and construct validity demonstrated | Evidence of skills transfer to the OR |
Laparoscopic nephrectomy | VR | Procedicus MIST (Mentice) | Yes | |
LAP mentor (Symbionix) | Yes | |||
General laparoscopic skills | Physical | eoSim Laparoscopic simulator (eo Surgical) | Yes | |
LapSim (Limbs and Things) | Yes | |||
Pyxus HD (inovus) | No | |||
iSIm2 (iSurgicals) | No | |||
FLS Laparoscopic Trainer (Venture Technologies Inc.) | Yes |
VR, virtual reality.
A total of 21 novice, 32 intermediate, and 10 experienced urologists performed the same retroperitoneal task on the LN VR three times each. In regard to outcomes, including time, blood loss, path length (distance of displacement of all instruments in m), and total score, the LN VR simulator could not distinguish between intermediate and experienced participants. Wijn et al. concluded that the simulator did not have sufficient construct validity. Brewin et al. evaluated the face, content, and construct validity of the Procedicus MIST™ nephrectomy simulator [16]. Face validity was established by eight expert laparoscopic surgeons, who believed that the simulator was a good training tool. Content validity was established by the experts, who rated the simulator as above average for realism. Construct validity was verified after performance results were analysed. Experts completed nephrectomy simulation significantly faster than novices with fewer errors and less haemorrhage and tool travel (Mann–Whitney U test p < 0.05).
The European Association of Urology (EAU) has also recognised the importance of simulation training in overcoming the initial learning curve in laparoscopic surgery and consequently developed the programme for laparoscopic urological skills (PLUS). The PLUS examination offers quality criteria and time criteria for the completion of basic laparoscopic tasks including peg transfer, cutting a circle, single knot tying, clip and cut, and needle guidance. This programme has been proven for face, content, and construct validity by a cohort of laparoscopic experts, intermediates, and novices in the Netherlands [17]. As a result, PLUS has been implemented at a national level in the Netherlands as a basic laparoscopy examination. The EAU have extended the programme further by developing the European Basic Laparoscopic Urology Skills Course [18], a combination of theory and practical assessments that aim to provide a basic level of laparoscopic competency prior to embarking on live training. Full validation is still awaited.
Similar to endoscopic simulators, there is a lack of prospective randomised controlled trials investigating the translational benefit of laparoscopic simulators to the OR. The majority of laparoscopic simulation research has focussed on LN procedures. With the expanding role of laparoscopic partial nephrectomy, pyeloplasty, cystectomy, and prostatectomy, future laparoscopic simulation development and research must keep pace.
1.3.7 Robotic Surgery Simulation
Since its first application in 2000 by Binder and Kramer, the uptake of robotic surgery has been rapid. This ever‐evolving surgical technique has cemented itself as the gold standard for the removal of the prostate gland. There are six commercially available robotic simulators all of which are VR platforms (Table 1.3.3): Robotic Surgical Simulator (RoSS™), SimSurgery Educational Platform (SEP), ProMIS®, Mimic dV‐Trainer (MdVT), Surgical SIM RSS, and the da Vinci® Skills Simulator™ (dVSS) (Figure 1.3.1). The biomechanics lab at the University of Nebraska at Omaha have also developed their own VR simulator. These platforms aim to establish the basic principles of robotic operative techniques, namely, endowrist manipulation, camera and clutching, needle control and driving, energy, and dissection control. In terms of validation, all of the simulators except RoSS have demonstrated face, content, and construct validity, but the numbers in these studies remain small. Educational impact has been shown in studies and in all commercially available simulators except SEP. Evidence of criterion validity, such as predictive or concurrent validity, is sparse. Other parameters, such as inter‐rater and inter‐item reliability, feasibility, acceptability, and cost effectiveness of the simulation platforms were not evaluated by any of the studies. Similarly no group has validated the use of animal models and freshly frozen cadavers and structured skills training based on observation for robotic surgery. There is a distinct lack of comparative studies between the different simulators. The current body of evidence does not identify any one simulator being more effective in training the next generation of robotic surgeons than another. Each platform has the capability to train and assess a range of different robotic skills fundamental to the technique. Unlike the dVSS, the MdVT and RoSS platforms have user interfaces that are similar to, but not exact duplicates of, the dVSS console used in clinical practice. The ProMIS simulator enables virtual and physical reality to be used together and has been investigated in the laparoscopic setting previously. A randomised control trial by Feifer et al. represents the highest level of evidence for any of the simulators currently available [19]. Their study showed that the use of ProMIS and LapSim simulators in conjunction with each other could improve robotic console performance. Interestingly, the LapSim group showed no improvement, and it was therefore not clear what contribution LapSim had on the overall improvement seen when both simulators were used in conjunction. Despite SEP’s level of validation, its biggest disadvantage lies in the fact that the images are not three dimensional (3D), a fundamental concept pertaining to robotic compared with laparoscopic surgery. Further studies or perhaps even hardware upgrades to convert the two‐dimensional (2D) simulator into a 3D platform are therefore warranted.
Table 1.3.3 Robotic simulators and their validation.
Procedure | Simulator type | Description of simulator | Face, content, and construct validity demonstrated | Evidence of skills transfer to the OR |
Robotic surgery: generic skills and prostatectomy | VR | dV‐trainer (Mimic) | Yes | Tissue |
dVSS (Intuitive Surgical) | Yes | |||
RoSS (simulated surgical systems) | Face and Content | |||
SEP (Sim Surgery) | Yes | |||
ProMIS (CAE healthcare) | Yes | |||
Surgical SIM RSS (METI) | Yes | |||
Partial Nephrectomy | Bench | No |
VR, virtual reality.
The majority of robotic surgical simulators focus on prostatectomy training. There is an ever‐growing role for robotic cystectomy, radical nephrectomy, and partial nephrectomy. As such, current simulators will require software updates, and in future simulator, developers will need to consider these advances in their designs.
1.3.8 Nontechnical Skills Simulation
NTS can be classified into cognitive factors (e.g. decision making, situational awareness), social factors (e.g. communication, teamwork, leadership), and personal resource factors (e.g. ability to cope with stress and fatigue) [20]. These NTS and related behaviours specifically affect performance in the OR. Studies have also developed validated scoring systems such as The Oxford Non‐Technical Skills score (NOTECHS) and Non‐technical Skills for Surgeons (NOTSS) (Table 1.3.4) that can be used for research, to provide feedback during training, and to provide an educational framework to describe these skills to surgeons [21, 22]. Research in medicine and other safety critical industries (e.g. aviation, nuclear power, and military) has shown that training can improve NTS and team performance in the workplace. Several strategies have been used to improve NTS, but in surgery, as in other industries, simulation‐based team training has emerged as one of the best ways to achieve this.
Table 1.3.4 Operating theatre team NOTECHS assessment tool.
Leadership and management | |||
Leadership | Involves or reflects on suggestions; visible; accessible; inspires; motivates; coaches | ||
Maintenance of standards | Subscribes to standards; monitors compliance to standards; intervenes if deviation; deviates with team approval; demonstrates desire to achieve high standards | ||
Planning and preparation | Team participation in planning; plan is shared; understanding confirmed; projects; changes in consultation | ||
Workload management | Distributes tasks; monitors; reviews; tasks are prioritised; allots adequate time; responds to stress | ||
Authority and assertiveness | Advocates position; values team input; takes control; persistent; appropriate assertiveness | ||
Teamwork and cooperation | |||
Team building/maintaining | Relaxed; supportive; open; inclusive; polite; friendly; use of humour; does not compete | ||
Support of others | Helps others; offers assistance; gives feedback | ||
Understanding team needs | Listens to others; recognises ability of team; condition of others considered; gives personal feedback | ||
Conflict solving | Keeps calm in conflicts; suggests conflict solutions; concentrates on what is right | ||
Problem solving and decision making | |||
Definition and diagnosis | Uses all resources; analytical decision making; reviews factors with team | ||
Option generation | Suggests alternative options; asks for options; reviews outcomes; confirms options | ||
Risk assessment | Estimates risks; considers risk in terms of team capabilities; estimates patient outcome | ||
Outcome review | Reviews outcomes; reviews new options; objective, constructive, and timely reviews; makes time for review; seeks feedback from others; conducts post‐treatment review | ||
Situation awareness | |||
Notice | Considers all team elements; asks for or shares information; aware of available of resources; encourages vigilance; checks and reports changes in team; requests reports; updates | ||
Understand | Knows capabilities; cross‐checks above; shares mental models; speaks up when unsure; updates other team members; discusses team constraints | ||
Think ahead | Identifies future problems; discusses contingencies; anticipates requirements | ||
Below standard = 1 | Basic standard = 2 | Standard = 3 | Excellent = 4 |
Behaviour directly compromises patient safety and effective teamwork | Behaviour in other conditions could directly compromise patient safety and effective teamwork | Behaviour maintains an effective level of patient safety and teamwork | Behaviour enhances patient safety and teamwork; a model for all other teams |
NOTECHS, Nontechnical Skills.
Team‐based training typically uses high‐fidelity simulated environments to represent clinical scenarios. Simulated OR scenarios can be developed by combining a high‐fidelity human patient manikin with a part task surgical trainer in a simulated OR environment (Figure 1.3.2). Modern manikins can reproduce realistic patient physiology, while a part‐task surgical simulator simulates a technical aspect of the procedure. Careful attention to scenario design, training as a multidisciplinary team, and use of video feedback can help maximise learning in these contexts. However the post‐scenario debriefing has consistently been identified as the most important aspect of the learning experience [23]. A skilled facilitator can provide feedback, encourage learners to analyse specific behaviours and NTS, create a safe learning environment, and help learners to apply their knowledge to work‐based settings [20].
Recognising the value of simulation and the fact that the majority of errors in surgery are the result of deficiencies in NTS, the SIMULATE programme has been developed in the United Kingdom [24]. It is a team‐based simulation training programme in addition to workplace training. The programme provides training in both technical and NTS. Technical skills including TURP, TURBT, rigid and flexible URS, PCNL, laparoscopy, and robotics are taught using a combination of bench top and virtual reality simulators, whereas NTS are developed using an interactive human patient mannequin in a high‐fidelity simulation environment. Trainees are provided with colleague support appropriate to the scenario (e.g. anaesthetist and nurse), and a video relay to the debriefing room allows full observation and feedback by an experienced faculty. The aim is to create an environment which is realistic enough for the surgical team to participate in the simulation and promote realistic team behaviours that can be discussed an analysed in debriefing sessions. NTS training has also been adopted by the Australasian College of Surgeons, the American College of Surgeons Association of Programme Directors in Surgery (ACS‐APSD), and the Team Strategies and Tools to Enhance Performance and Patient Safety (team STEPS) [25, 26]. Further strengthening the argument to integrate such high‐fidelity NTS training within urology curricula are the results of several well‐designed studies by Lee et al. and Gettman et al. which have supported this type of NTS training in urology [27, 28].
1.3.9 Simulation Training Curriculums
A well‐structured performance‐based curriculum with well‐defined end points and benchmark criteria remains the holy grail of simulation training. A number of barriers exist to developing such an ideal, namely cost, agreement on choice of simulator, and types of procedures requiring simulation training. Furthermore, integration of simulation programmes into training curriculum vary across different regions with each institution developing its own curriculum content and design. Much of the education literature supports the need to create a national standardised curriculum with proficiency‐based strategies [29, 30]. It has been suggested that surgeons acquire technical skills through a series of steps, beginning with cognitive orientation, followed by repetition, and then demonstration of autonomy with that skill. A validated simulation programme can be used to support and implement such a proficiency‐based curriculum because it has built‐in objective measures of assessment from which proficiency levels can be derived [29]. The work by Aggarwal et al. has shown this through the design of a proficiency‐based curriculum for basic laparoscopic skills acquisition [30]. Performance criteria were preset by pooling the scores of expert surgeons on the simulator. Thus benchmark proficiency criteria were defined for endpoints for each task such as time taken, number of errors, and path lengths. This created the basis of proficiency‐based curriculum with clearly defined endpoints. Progression along the curriculum could be charted by passing set performance criteria so proficiency can be defined and competency ensured through end‐of‐module examinations. This curriculum has set the blueprint for ongoing research into the most effective simulation curriculum. Such performance‐based simulated criteria has led to effective skills acquisition and improved operating theatre performance. For example, proficiency‐based simulator training in laparoscopic suturing and knot tying improved performance in the operating room [31, 32]. Furthermore, a proficiency‐based simulator curriculum ensures uniform skills acquisition by ensuring all trainees reach a set level of competency.
1.3.10 Assessment of Trainees
Before surgical simulation‐based training methods can be used to assess the competency of surgeons, they must undergo initial testing across a variety of validity and reliability parameters (Table 1.3.5). This would include the assessment of face validity, which examines the realism of the simulator or simulated scenario; construct validity which differentiates novice from experienced operators; context validity examines whether the device can teach what it is supposed to teach; concurrent validity is the extent to which the results of the test correlate with the gold standard known to measure the same domain; and predictive validity is the extent to which an assessment will predict future performance [33]. Reliability relates to the reproducibility of an assessment and is linked to the assessment’s validity. It allows educators to quantify the amount of random error in the measured data, facilitating valid interpretations and the usage of trainee assessment scores. Correlation coefficients such as Cronbach’s alpha coefficients are used to demonstrate internal consistency with reliability scores of at least 0.90 for high‐stake assessments such as certification examinations [34].
Table 1.3.5 Definitions of terms related to competence, training and assessment.
Face validity | Relates to realism of simulator. Does simulator represent what it is supposed to represent? |
Concurrent validity | Degree to which simulator, as assessment tool, correlates with gold standard. |
Construct validity | Refers to demonstrating difference in task performance based on experience level. Construct validity accounts for variance in test performance. |
Content validity | Judgement of suitability of simulator as teaching modality involving formal evaluation by experts in subject matter of training device. This answers the question of whether a simulator can realistically teach what it is supposed to teach. |
Predictive validity | Ability of test to predict future performance in operating theatre. |
As well as validity and reliability evidence, the application of simulation‐based tools for assessment rely on establishing credible, defensible, and acceptable standards. Standards are categorised as either norm‐based standards, which determine competency relative to the performance of a well‐defined group (e.g. laparoscopic expert surgeons) or criterion‐based standards, which determine competency based on a predetermined absolute level of performance. Criterion‐based standards are preferred to assess competency because they imply a certain level of mastery of a skill [35].
1.3.11 Future of Simulation
One of the challenges for surgical educators is deciding which of the simulators to integrate into urological curriculums. With the rapid developments in technology, new simulators are constantly being developed, making research and validation of these tools challenging. Additionally more than one simulator exists for a given procedure, with a lack of comparative studies adding to the difficulties faced by programme directors. Often such decisions are based on results of research on older simulators, which have a better evidence base and on expert judgement. Compounding matters is that there are no universally accepted criteria on how to exactly validate a simulator, leading to methodological variations between studies, with a real need for a consensus regarding validation methodologies. Importantly the VR TURP trainer and the VR Uromentor (Symbionix) have been shown to improve surgical performance in the OR. Further studies with the aim of identifying this relationship are required. Although such studies are essential, they are intensive, time consuming, and pose numerous ethical conundrums. Showing this translation requires a large number of patients and training sessions and are therefore highly resource intensive. Ethical issues relate to patient consent. Patients want the best possible treatment and may withhold consent if participation entails a relative increase in their risk of complications. As such, these studies are best performed in well‐organised, multicentre settings with approval from local ethics committees.
The ideal simulation programme would have a mix of high‐ and low‐fidelity simulators to match various learning requirements. However, in reality, very few centres can afford the high‐fidelity VR simulators. One suggestion has been to develop a hub‐and‐spoke service model where a larger, centralised simulation centre with better investments and a larger repertoire of simulators can liaise with peripheral centres to share resources [36].