A High-Resolution Virtual Reality-Based Simulator to Enhance Perioperative Echocardiography Training

Abstract

Perioperative echocardiography requires an advanced understanding of the complex human cardiac anatomy. Currently, conventional training simulators rely on handcrafted heart models that lack accuracy and details and undermine the complexities of the cardiac anatomy, both actual and relative. These simulators are expensive and difficult to transport, creating barriers to widespread implementation. In this report, the authors describe a realistic, virtual reality simulator using high-resolution human heart scans that accurately represent the healthy and pathologic cardiac anatomies in ways that can be standardized and made accessible to a wide range of learners at the cost of a virtual reality headset. Herein, the authors present a description, including the design of the transesophageal echocardiography and transthoracic echocardiography simulator, and their initial experiences during cardiac fellowship training. Perioperative echocardiography is an essential tool in the management of surgical patients, and is used to assist decision-making along the entire perioperative pathway.1.Barber RL Fletcher SN. A review of echocardiography in anaesthetic and peri-operative practice. Part 1: Impact and utility.Anaesthesia. 2014; 69: 764-769Crossref PubMed Scopus (41) Google Scholar, 2.Practice guidelines for perioperative transesophageal echocardiography. An updated report by the American Society of Anesthesiologists and the Society of Cardiovascular Anesthesiologists Task Force on Transesophageal Echocardiography.Anesthesiology. 2010; 112: 1084-1096Crossref PubMed Scopus (391) Google Scholar, 3.Cowie B. Focused cardiovascular ultrasound performed by anesthesiologists in the perioperative period: Feasible and alters patient management.J Cardiothorac Vasc Anesth. 2009; 23: 450-456Abstract Full Text Full Text PDF PubMed Scopus (84) Google Scholar Perioperative echocardiography is also critical during many interventional procedures. Combined, the need for expertise in echocardiography is growing.4.Patel H Raisinghani A DeMaria A. Echocardiography in transcatheter structural heart disease interventions.Progr Cardiovasc Dis. 2018; 61: 4233-4236Crossref Scopus (7) Google Scholar Yet, one of the limiting factors for the routine use of echocardiography in the perioperative setting is the availability of trained operators.1.Barber RL Fletcher SN. A review of echocardiography in anaesthetic and peri-operative practice. Part 1: Impact and utility.Anaesthesia. 2014; 69: 764-769Crossref PubMed Scopus (41) Google Scholar Echocardiography is a complex technical skill that requires extensive training.5.Ryan T Berlacher K Lindner JR et al.COCATS 4 Task Force 5: Training in echocardiography.J Am Coll Cardiol. 2015; 65: 1786-1799Crossref PubMed Scopus (49) Google Scholar The American Society of Echocardiography has recommended, as a part of basic training requirements for competence in echocardiography, a thorough understanding of varied human cardiac anatomy and the manifestations of pathology in both acquired and congenital heart disease.6.Hahn RT Abraham T Adams MS et al.Guidelines for performing a comprehensive transesophageal echocardiographic examination: Recommendations from the American Society of Echocardiography and the Society of Cardiovascular Anesthesiologists.J Am Soc Echocardiogr. 2013; 26: 921-964Abstract Full Text Full Text PDF PubMed Scopus (723) Google Scholar To meet this requirement, simulation has become a popular and effective training tool in recent years.7.Jujo S Nakahira A Kataoka Y et al.Transesophageal echocardiography simulator training: A systematic review and meta-analysis of randomized controlled trials.Simul Healthc. 2021; 16: 341-352Crossref PubMed Scopus (10) Google Scholar, 8.Vegas A Meineri M Jerath A et al.Impact of online transesophageal echocardiographic simulation on learning to navigate the 20 standard views.J Cardiothorac Vasc Anesth. 2013; 27: 531-535Abstract Full Text Full Text PDF PubMed Scopus (24) Google Scholar, 9.Sidhu HS Olubaniyi BO Bhatnagar G et al.Role of simulation-based education in ultrasound practice training.J Ultrasound Med. 2012; 31: 785-791Crossref PubMed Scopus (52) Google Scholar In general, simulation-based learning is a trainee-focused environment away from the pressures of the clinical setting that offers a risk-free platform from which trainees can gain expertise in obtaining quality echocardiographic images in a low-stress environment.10.Dreyfus J Donal E Pezel T Moving into a new era for echocardiography education with simulation and workshop-based training.JACC Case Rep. 2020; 2: 505-507Crossref PubMed Google Scholar Although advancements in hands-on and online simulation have disrupted how echocardiography is taught, both fall short on several fronts: (1) three-dimensional (3D) heart models are either reconstructed from low-resolution scans or handcrafted to look animated. Either way, models lack the accuracy and detail for learning advanced cardiac anatomy, especially as it relates to atrioventricular valves and subvalvar complex8.Vegas A Meineri M Jerath A et al.Impact of online transesophageal echocardiographic simulation on learning to navigate the 20 standard views.J Cardiothorac Vasc Anesth. 2013; 27: 531-535Abstract Full Text Full Text PDF PubMed Scopus (24) Google Scholar,11.Bose R Matyal R Panzica P et al.Transesophageal echocardiography simulator: A new learning tool.J Cardiothorac Vasc Anesth. 2009; 23: 544-548Abstract Full Text Full Text PDF PubMed Scopus (47) Google Scholar; (2) conventional simulators use flat screens, undermining the complexities inherent to cardiac anatomy best understood in 3D8.Vegas A Meineri M Jerath A et al.Impact of online transesophageal echocardiographic simulation on learning to navigate the 20 standard views.J Cardiothorac Vasc Anesth. 2013; 27: 531-535Abstract Full Text Full Text PDF PubMed Scopus (24) Google Scholar,12.Deng S Wheeler G Toussaint N et al.A virtual reality system for improved image-based planning of complex cardiac procedures.J Imaging. 2021; 7: 151Crossref PubMed Scopus (4) Google Scholar; and (3) the cost of a transthoracic echocardiography (TTE)/transesophageal echocardiography (TEE) simulator system is approximately $100,000,10.Dreyfus J Donal E Pezel T Moving into a new era for echocardiography education with simulation and workshop-based training.JACC Case Rep. 2020; 2: 505-507Crossref PubMed Google Scholar a known barrier to widespread implementation and restricting access to major academic institutions. To bridge these deficiencies and further enhance echocardiography training, the authors have developed a realistic, low-cost, virtual reality simulator using high-resolution human heart scans that accurately represent the healthy and pathologic cardiac anatomies in ways that can be standardized and made accessible to a wide range of learners. Herein, the authors present a description, including the design of the TEE and TTE simulator. The study protocol for this research conformed to the ethical guidelines of the 1975 Declaration of Helsinki, and was reviewed and approved by the Institution Review Board at the University of Minnesota. After consent, fresh human heart specimens not suitable for orthotopic transplantation were procured (in collaboration with LifeSource, Minneapolis, MN) and donated to the Visible Heart Laboratories. Explanted hearts were cannulated through the great vessels and perfusion-fixed with 10% formalin for at least 24 hours to preserve an approximation of the end-diastolic state.13.Anderson SE Quill JL Iaizzo PA. Venous valves within left ventricular coronary veins.J Interv Card Electrophysiol. 2008; 23: 95-99Crossref PubMed Scopus (21) Google Scholar Microcomputed tomography images were obtained (X3000; NorthStar Imaging, Rogers, MN) with a voxel resolution of 90-to-120 μm, (resolutions that cannot be obtained readily during in vivo imaging).14Arango S, Diaz-Gomez JL, Iaizzo P, et al. A high-fidelity three-dimensional computational model of a patient with hypertrophic cardiomyopathy. CASE. 2022;6(8):350–4.Google Scholar Anonymized datasets were imported into Mimics Innovation Suite (Materialise NV, Leuven, Belgium) software for generation of the different cardiac 3D models. To generate the chest and heart model, a dataset of the extracardiac anatomy within the chest cavity was derived from a contrast-enhanced computed tomography of the donor while still alive. The dataset was imported into the Mimics Innovation Suite for segmenting the organs and generating the 3D model. Subsequently, the matching high-resolution cardiac computational model from the same donor was superimposed onto the chest dataset using anatomic landmarks, and then merged as a single geometry using a software called Geomagic Design X (3D Systems, Rock Hill, SC).15.Gosnell J Pietila T Samuel BP et al.Integration of computed tomography and three-dimensional echocardiography for hybrid three-dimensional printing in congenital heart disease.J Digit Imaging. 2016; 29: 665-669Crossref PubMed Scopus (68) Google Scholar The resulting models were exported using surface tessellation language format to Blender (Blender Foundation, Amsterdam, The Netherlands), Geomagic Design X, and Meshmixer (Autodesk, Inc, San Rafael, CA) to smooth the surface of the anatomy to reduce pixelation.15.Gosnell J Pietila T Samuel BP et al.Integration of computed tomography and three-dimensional echocardiography for hybrid three-dimensional printing in congenital heart disease.J Digit Imaging. 2016; 29: 665-669Crossref PubMed Scopus (68) Google Scholar Using the Unity game engine, the authors developed an Open XR VR simulator compatible with any of the currently available commercial headsets. Any basic commercially available virtual reality simulation system consists of hardware, namely a headset and 2 controllers, one on each hand, and the simulation software (Fig 1). The headset is head-mounted device that acts as a display, allowing the user to look around the 3D environment. The controllers are used to manipulate the virtual objects in the display and select various configurations on the menu. The simulator the authors describe is compatible with most commercially available virtual reality devices. The authors selected to use the Oculus Quest 2 due to its availability, cost ($299 USD; including the headset and the 2 controllers), and efficiency at the time they were developing their program. The software, an Open XR (compatible with any virtual reality [VR] headset) application, was developed on a Windows-10 64-bit (Microsoft, Corp, Redmond, Washington) computer using Unity 3D (Unity Technologies, San Francisco, CA). To access the VR application, the user can either download the application onto a commercially available VR-compatible desktop or laptop computer or connect the VR headset to the internet and launch the application through one of the cloud-based portals available. If the first option is chosen, and the application has been downloaded onto a computer, the VR headset connected via a type C cable and the system is ready to use. If the second option is chosen, a cloud-based portal service must be downloaded for the VR system to be used. To date, 9 heart models have been generated and used for both TEE and TTE VR-based simulation. One of the 7 hearts represents normal anatomy, 2 have a patent foramen ovale, 1 has an atrial septal defect, 2 have valve replacements, and 2 with hypertrophic cardiomyopathy and with medical devices like cannulae and implantable defibrillators. Once the user has launched the VR Echo Simulator application, a video tutorial is automatically launched to explain to the user how to control the simulator using the hand-held controllers. The tutorial demonstrates visually how to use the controllers to access the system's menu and control the central display. By using the various buttons located on each of the 2 controllers, the learner can directly access central and secondary menus to choose different heart models and the echocardiographic modes (TEE or TTE), use the built-in laser pointer, rotate the central display along 3 planes at varying speeds, change the size of the central display by “bringing it closer” or “pushing it farther away,” and slice the central display along multiple planes. The simulator the authors have developed currently has options to enter either a TEE or TTE mode. In the TEE simulator mode, the user controls the “probe” from inside the esophagus and adjust the image by rotating, advancing, withdrawing, anteflexing, and retroflexing using one of the 2 controllers (Fig 2). The authors included a menu of options from which to choose any number of preprogrammed American Society of Echocardiography-recommended views, and a laser pointer for pointing out structures while instructing learners (Figs. 3 and 4; Video 1). In the TTE simulator mode, the user can hold and control the “probe” from the surface of the chest (Fig 5). In the TTE mode, users are able to gain only images of the heart by positioning the virtual probe between rib spaces (ie, ribs and lung tissue will “block” cardiac imaging similar to a real TTE). Both modes allow the user to visualize an image in 3D while viewing a contiguous 2-dimensional representation of the ultrasound image as they would on an ultrasound machine (Fig 6). In their initial applications, the authors were able to standardize the experience for multiple users by feeding the VR application directly into the headset as well as an external display (Fig 1; Video 2).Fig 3(A) A midesophageal 4-chamber view of the heart as seen from the esophagus (bottom) and the corresponding echocardiographic image (upper). (B) A transgastric short-axis view of the heart as seen from the esophagus (bottom) and the corresponding echocardiographic image (upper). In the authors’ simulator, both the virtual reality view and corresponding echocardiographic image are seen simultaneously by the learner. To further facilitate conceptualization, scan planes are demarcated with red and green lines that correspond to the left and right side of the echo image, respectively.View Large Image Figure ViewerDownload Hi-res image Download (PPT)Fig 4(A) Midesophageal bicaval view in a heart model with a patent foramen ovale as identified by the red laser pointer. (B) Midesophageal long-axis view of a heart model with a mechanical aortic (red arrow) and mitral valve (green arrow). AO, aorta; LA, left atrium; LV, left ventricle; LVOT, left ventricular outflow tract; RA, right atrium; RV, right ventricle; SVC, superior vena cava.View Large Image Figure ViewerDownload Hi-res image Download (PPT)Fig 5View of a high-resolution 3-dimensional chest model containing the heart and the surrounding structures. The user is able to hold and manipulate a digital probe, as seen to the right of the chest model, to create sections through the heart model and corresponding echocardiographic images, as described in Figure 5.View Large Image Figure ViewerDownload Hi-res image Download (PPT)Fig 6In the transthoracic echocardiography mode, the 3-dimensional model of the chest (right) can be visualized simultaneously with the scan planes that generate the 2-dimensional images (left). The reference plane in (A) is the parasternal short-axis view and in (B) the parasternal long-axis view.View Large Image Figure ViewerDownload Hi-res image Download (PPT) The simulator described herein is free and can be downloaded using a VR-compatible computer tethered to a virtual reality headset. The system also can be accessed using a cloud-based streaming platform using only a headset. The cost to use the system is limited to the cost of a commercially available headset that ranges from $300 to $900. The authors implemented the VR simulator described herein as part of this years’ echocardiography training curriculum for their current adult Cardiothoracic Anesthesiology (aCTA) Fellows. These sessions are held once a week and last for an hour each. In the authors’ experience and informal assessment, using the VR simulator has facilitated an understanding of cardiac anatomy and aided their aCTA Fellows with proper TEE image orientation and recognition, and allowed for shared experiences, accessibility, and standardization during the learning process. In developing and implementing a VR-based simulator as a tool to train aCTA Fellows to learn perioperative echocardiography, the authors identified several limitations. Some participants may experience motion sickness while wearing the headset. Published studies have reported dropout rates of users due to VR sickness as high as 15.6%, an issue that is more likely to occur when participants make quick and/or fast head movements.16.Saredakis D Szpak A Birckhead B et al.Factors associated with virtual reality sickness in head-mounted displays: A systematic review and meta-analysis.Front Hum Neurosci. 2020; 14: 96Crossref PubMed Scopus (196) Google Scholar The authors found that adequate participation using their system requires minimal head movements, and can be done while seated, both limiting the risk of motion sickness. The myocardium and valves can be visualized only in static positions. The authors are currently working with animators to create a dynamic component to their heart models whereby myocardium will contract, thicken, and relax similar to systole and diastole, respectively, and valves will open and close in concordance with the cardiac cycle. The authors’ VR environment does not currently allow learners to simulate TEE probe insertion or probe manipulation. In the absence of this feature, learners must use either available simulators or gain these experiences during clinical care. Finally, the authors understand that the assessment of the system as a tool for teaching their aCTA Fellows is informal and, therefore, limited. Although trialing iterations of this tool has been valuable for improving utility and usability, the authors have not quantified the value of the tool in improving understanding when compared with traditional methods for teaching perioperative echocardiography. Future work will compare a VR-based curriculum with traditional methods, including lectures and web-based learning tools. The authors successfully developed and implemented a VR-based echocardiography simulator, using a human heart, with unparalleled resolution, anatomic detail, and accuracy as a means of standardizing perioperative echocardiography training that is both accessible and cost-conscious for their aCTA Fellows. Future efforts will quantify the value and usability of this tool while the authors expand their program to include anesthesia, surgical, and emergency medicine residents and fellows and faculty.

Topics & Concepts