A New Virtual World? The Future of Immersive Environments in Anesthesiology
Fahad Alam, Clyde Matava
Abstract
See Article, page 220 Immersive virtual environments (IVEs), including virtual reality (VR) and augmented reality (AR), have gained immense popularity in medical education health care.1 Over the last 10 years, the expansion of gaming and development platforms, such as Unreal Engine and Unity, along with head-mounted displays (HMDs), has been fueled by major commercial interests, such as Meta, Microsoft, and Google, with huge investments in this immersive reality market. The immersive reality market is currently valued at 6.1 billion US dollars (USD) and is predicted to reach >20 billion USD by 2025.2 This commercial expansion has led to the rapid advancement of affordable, accessible, high-powered VR/AR technology and development tools supporting IVEs in medical education including anesthesia. IVEs date back to the early 19th century with the advent of the stereoscope. This ingenious device allowed 2 separate images, taken at slightly different angles, to be viewed as a single image with depth. The stereoscope led to the creation of a children’s toy: the View-Master (View-Master Ideal Group, Inc), a pioneer for modern-day VR. Subsequently, the invention of the microchip represented a significant leap, leading to a myriad of devices and technologies that make up IVEs today. Virtual environments have been described in 2 domains: immersive (HMD-based) and nonimmersive (desktop-based). Nonimmersive, desktop-based virtual environments do not use an HMD, but instead, the participant controls and manipulates the virtual environment on a screen with a traditional keyboard and mouse or a joystick.3,4 Playing a video game on one’s television screen using popular hardware such as the Sony PlayStation or the Microsoft Xbox is a basic example of desktop-based VR. In medical education, the American Society of Anesthesiologists (ASA) recently partnered with CAE Healthcare to create desktop-based virtual environments for managing anesthesia emergencies,5 which, when completed, fulfill the American Board of Anesthesiology Maintenance of Certification in Anesthesiology (MOCA) 2.0 Part 2 and 4 requirements.6 Although nonimmersive desktop-based virtual environments have been popular for decades and are an important part of our current medical education system, the focus of this article is on HMD-based IVEs. HMD-based IVEs have been described as “typically multi-modal in nature by providing a sense of immersion in the environment through 360° visuals by aid of an HMD, auditory stimulation through the use of earphones, and increasing the proprioception of limbs by way of controllers and tracking.”4 Using HMDs, IVEs “transport” users to alternate simulated environments while creating a sense of immersion. Imagine playing a video game, but instead of watching it on a 2-dimensional screen, actually being transported into the game itself, akin to the experience of the characters in the movie “Jumanji.” The HMDs used vary in complexity from simple cardboard-based stereoscopic lenses that pair with a mobile device to expensive helmets and goggles with advanced haptics. A tenet of IVE is presence. Presence is the extent to which a participant feels that they are submerged within the simulated IVE. Presence is a key term that describes the difference between VR and AR (Table). VR is the process of being completely immersed in a “new” environment (filmed or computer-generated), while AR overlays digital elements on real-life surroundings. As an example, VR would completely alter the sensory input of users via an HMD and transport them into a virtual trauma room, while AR would project, again via an HMD, a virtual patient into an actual real-world operating room (OR). Table. - Virtual Reality Versus Augmented Reality: Differences and Similarities Virtual Reality Augmented Reality Creates entire virtual world Creates a combination of virtual and real world Immersive Holographic Computer generated or filmed using 360° camera elements Computer-generated elements that are superimposed onto real life Requires a headset Requires multiple devices Immersed completely into the projected environment using a head-mounted display that completely suppresses the real world (eg, Google Cardboard and Oculus Rift) Immersed partially from screen-based devices with cameras (ie, smartphones, laptop, or tablet) using a head-mounted display with see-through lenses, allowing for visualization of the real world and virtual elements overlaid into it (eg, Microsoft Hololens) Object manipulation Object manipulation Only digital objects in virtual world Combines physical simulation with virtual objects Pricing Pricing Cost for creation and equipment can vary from a few hundred to thousands of dollars, depending on the level of immersion and complexity. Cost for creation and equipment can vary from a few hundred to thousands of dollars, depending on the level of immersion and complexity. In the same manner that IVEs have evolved from the stereoscope, Anesthesia & Analgesia has evolved over the past 100 years, from being the world’s first anesthesia journal to one that is a leading source for innovation and education. Using lessons learned from other health care specialties and the non-health care industry, this article serves as a futuristic report that describes how IVEs, specifically VR and AR, can enhance medical training and subsequent delivery of patient care within the field of anesthesia. We suggest future directions and next steps in the evolution of this technology within the specialty of anesthesia. Technical aspects on “how-to” create and develop immersive environments are beyond the scope of this article; however, we hope to create excitement around the potential for the use of IVEs in anesthesia. IVEs AND ANESTHESIA EDUCATION Before the coronavirus disease 2019 (COVID-19) pandemic, the staple discourse in medical education included 3 prevalent themes. The first dealt with health care worker fatigue and working hour restrictions implemented to prevent human errors and to maintain learning.7 The second reflected the seismic shift in medical curriculums that have moved toward competency-based rather than time-based training.8 The third dominant theme revolved around the growing need for continuing education programs, especially considering recent retrospective literature, which has suggested correlations of increasing physician age and duration from graduation with patient complications.9,10 The result of these significant issues has tasked educators with finding innovative and effective methods to enhance learning curriculums. Despite this need, health care delivery continues to fall behind other service industries with regard to its investment into, and adoption of, new disruptive technologies. Furthermore, the COVID-19 pandemic has only accelerated the need for innovative educational approaches and emphasized the need for technology, including Zoom and Teams. IVE is a technology that is likely to grow beyond the pandemic.11,12 Discovering and using adjuncts to trainee curriculums and creating pathways to extend the reach of health care beyond the physical boundaries of the hospital are some of the new themes of medical education.11–13 Technical Skills Training for a career in anesthesiology requires expertise in both technical and nontechnical skills. The field of anesthesiology has led the way in creating and adapting mannequin-based simulation as a gold standard for training without risk to patients.14 However, to date, procedural dominant specialties such as surgery have pioneered the use of IVEs to teach and practice surgical skills and to instruct anatomy in an interactive manner.15,16 IVE-based anesthesia training has been reported for procedural areas such as bronchoscopy, central line insertion, and regional anesthesia simulators.17–20 Studies investigating such procedural training have shown that practicing technical skills in virtual environments can lead to effective knowledge acquisition and retention with subsequent successful clinical application.15,17 Furthermore, IVEs can be easily manipulated, allowing iterative skill-building within evolving virtual scenarios. If a trainee typically takes 50 attempts at intubation to become proficient, that goal may be achieved more quickly using a VR simulator that evolves in difficulty and variance. The learning curve would be enhanced because the scenarios change (as they would in the real world). However, they do so repeatedly, over a shorter time period compared to a traditional training system that would have the trainee wait hours for each surgery to complete before having a chance to perform another intubation procedure. Nontechnical Skills IVEs may have significant potential in teaching and assessing nontechnical skills, such as empathy, communication, and decision-making. At a basic level, observational learning, in which a learner watches a teacher demonstrate or model “correct” behavior or steps, has been used as a teaching tool for decades.21 However, with busier clinical environments, reduced trainee working hours, and amid pandemic-based restrictions, direct observation opportunities are also becoming limited.22,23 One solution to supplement observation learning is the use of VR-360 video (Figure 1). Asynchronously, we can use VR-360 cameras to film core concepts of a teaching curriculum for learners to view using HMDs, allowing them to feel as though they are physically present24 in the learning environment, learning from the teacher, when not there.25,26 Synchronously, such technology has been used to broadcast live surgeries to trainees when not present in the OR. VR-360 streams do not need advanced HMDs; instead, they can be viewed using simple HMDs such as the Google Cardboard combined with a user’s mobile device.Figure 1.: VR-360 video of an operating room. This is a fisheye perspective, as it is not being viewed through an HMD (source: www.CHISIL.ca). HMD indicates head-mounted display.Filming core concepts of a teaching curriculum for learners to watch is not new. VR-360 videos have been used to create IVEs to raise the participation factor and improve the learning experience.25,27–30 Existing examples are already in place, such as the Oxford Medical Education lab (www.oxfordmedicaleducation.com) and the Collaborative Human ImmerSive Interaction Laboratory (CHISIL; www.chisil.ca) at the University of Toronto. These sites have released teaching programs in an effort to improve decision-making skills around topics such as advanced life support, trauma management, and cardiac arrest/code blue scenarios using VR-360 video simulations. Fully immersive computer-generated video game software mimicking clinical environments can also be used to effectively teach decision-making around uncommon crises, such as OR fires and malignant hyperthermia, and more common scenarios, such as trauma management, anaphylaxis, and obstetrical emergencies.31 Unlike passive VR-360 video, these advanced immersive crisis management “video games” require advanced HMDs to allow users to interact with their environment and the virtual components to respond to their actions. In these virtual environments, scenarios can change quickly or adapt to suit learners’ needs or specific objectives. Virtual patients will improve or deteriorate in real time, depending on the actions taken by learners (Figure 2A). Like a video game, learners can “restart” and manage crises repeatedly with feedback processes built into the software. Furthermore, these scenarios can be recorded for future viewing or even observed live by a facilitator to provide coaching or assessment remotely (ie, from another location).Figure 2.: IVEs for clinical education. A, An advanced IVE depicting a trauma crisis with interactive avatars and responsive patients (source: www.CHISIL.ca). B, Dr Julian Wiegelmann demonstrates the use of augmented reality by using an HMD, converting the operating room into a crisis scenario with a virtual patient and responsive hemodynamics broadcast live on anesthesia monitors (source: www.Holosim.ca). HMD indicates head-mounted display; IVE, immersive virtual environment.Similarly, augmented virtual environments can make any physical space a virtual learning environment. For instance, at our local site, the onset of the pandemic required training and implementation of multidisciplinary COVID intubation teams within a short time frame. Using proprietary, locally developed AR software (www.HOLOSIM.ca), we converted our ORs and native monitoring equipment into virtual COVID care rooms with virtual patients requiring intubation and hemodynamic management (Figure 2B). Their simulated vitals were broadcast on our live OR monitors in response to the actions of the team. VALIDATING IVEs A significant challenge with IVEs is that their implementation may be outpacing the ability of the education community to validate them as effective teaching tools. Validation of IVEs for anesthesia by medical educators is vital to ensure they offer teaching interventions that align with curriculum goals using relevant educational frameworks. The current literature has shown that research in this field has been heterogeneous and lacks a basis in foundational learning theory.32,33 Opportunity exists to test and validate the use of IVEs for anesthesia education, including improved patient outcomes. To help advance appropriate validation when implementing a VR teaching program, medical educators should consider sound learning theory and learning outcomes as the structural foundation of the project. Learning theory, at its basic level, can be thought of as a foundation that helps achieve learning outcomes. Similar to biological research, it can be thought of as “biological plausibility.” Like all new technologies, the urge to just “build an app” without really considering the learning outcomes and how intelligent design can best facilitate achieving these outcomes is a common setup in which this “just do” mentality will fail. Learning theory is a complex topic, but for this article, we highlight that one must consider normative and cognitive learning theories when designing immersive virtual applications. Normative theories emphasize that the educator must consider the type of learner they are targeting, and cognitive theories “are concerned with how the brain actively processes information to produce effective learning.”34 Common theories in technology are multimedia design learning and cognitive load, meaning one must consider how learners process the new 3-dimensional audio and visual stimuli in immersive environments and how easily the content can be filtered by a learner and hence, absorbed.34 Our laboratory has developed a design framework called “LOOP” to be applied to every project. The designers are obliged to create based on sound learning theory, considering what learning outcomes the educator is aiming for, and to shape and guide the actual immersive software output (Figure 3).Figure 3.: LOOP framework. Design framework developed to advance appropriate validation when implementing IVE teaching programs by considering learning theory, desired learning outcomes and the actual immersive software AR indicates augmented IVE, immersive virtual virtual directions of training with immersive environments and educational areas for development teaching skills, and and Using live broadcast of VR-360 video, a teacher or may be of a health care in a by an HMD and with their The teacher or can in a digital with or decision-making clinical crises to specific clinical needs or offer learning is to the of coaching using learning theory in the of these potential can be learned from and can be applied to immersive For instance, used 2-dimensional screen-based software to teach regional anesthesia skills. the theories of observational learning and practice into their design to have a learning on their This can be taken as an example, developed into a virtual immersive teaching tool with which and learners different can be immersed in the using VR. This the learner can the the and practice their direct This is currently being with at our of the University of Toronto. Furthermore, teaching skills using immersive and interactive virtual environments can be a For instance, in one learners and practicing care with and in simulated were in some of the care they were with simulated this can be applied to anesthesia teaching curriculums. Virtual patients can be by live in or respond to live learners when practicing such as or However, even beyond technology is evolving so that patients can and respond to a live with the appropriate An important and in the future of using immersive environments for teaching is a of in simulation training and the of training as a The literature has that simulation technology has been in the of in the patient The use of in which an and can be easily may allow to and the in which they A simple but example would be learning in a The potential for immersive VR-360 videos or interactive environments is significant in The and the sense of immersion that these technologies provide may allow for a more for the learner can more in the of an from a different of it has been that and part in immersive VR which they were to characters that were a different than their while Furthermore, medical have used VR to allow their trainees to experience what their patients when it to their and how they the health This is a of training to medical with information to the health of through 360° creating digital an HMD, and on the of and health care can to steps toward what these EDUCATION In to medical education for health care IVEs are to clinical anesthesia management and health care delivery patient and education. IVEs as interventions to and and provide to Studies have shown that IVEs viewed through HMDs can be an effective supplement to of and patients in management, in early for insertion, requiring such as and for Using IVEs for management has been based on the of through the sense of in another or have the on using are more than in this and the of how to develop tools To achieve complete the patient should feel in the simulated IVE they are viewing with the of how best to achieve this for instance, computer-generated environments real-world VR-360 camera to be In the of IVE to the sense of and of the In other how is the IVE of the so it to its actual and within the new reality to as with any the for have to be and how their will vary with different must also be Furthermore, of these and research is to the for using IVEs and their in the IVE software tools have been developed to but is a of literature and research in this specific that needs to be to management are the of and to of patients experience the and this has been with in the in has already that IVEs can be used to and as as to IVEs have been applied to the experience as a patient education The literature has shown them to be an effective to and for patients This has and educators with a for interventions using and IVEs have been used as a of by patients their hospital experience with virtual of ORs or of the before they even in a have used IVEs as a of to patient hospital and while to demonstrate these in and may into improved clinical outcomes will be an and so it an Using the of and virtual described future of IVEs in clinical management can the of boundaries and expertise to The ability to a physician with a patient and the immersive experience that traditional 2-dimensional video offer can change the practice of anesthesia in hospital can use HMDs to patients for surgery without requiring to patient education and to care is another in which VR may be of Immersive virtual can allow health care to part in a new manner of for patients or care with are not easily A can be immersed in the environment to the patient in their environment and an may not have to all the time while in the as a or teacher can be transported into their virtual to offer coaching and teaching to help with a of such as a specific clinical an or clinical as as an clinical multidisciplinary teams with from both and as as from the virtual software development and are to on the in education and can the to create and these for In IVEs may in the of brain health in the IVEs have been used in and of and the literature has the of IVEs on and cognitive with age or IVEs have also been implemented as to aid A recent that VR can be a tool for cognitive brain these to of brain health in the the of is another is a and that has a significant on and in a significant to in of patient of and IVEs are being as part of a in for and cognitive as as in care as these will be to use IVE software in their to risk patients and their and a with IVEs as a structural to prevent or Imagine a patient back into their they can interact with their native environment and present IVEs have the potential to be a disruptive technology in the anesthesia education and health care delivery is at a rapid fueled by the commercial To ensure the development of IVEs in the of education, medical educators must guide this process using as its foundation sound learning theory and allowing for appropriate by our the We Julian and as as and for their help with our research We are to for advanced of the This developed the and the This advanced