Virtual Reality Simulation for Disaster Preparedness Training in Hospitals: Integrated Review

Introduction

Emergency preparedness in hospitals for disasters is unexpectedly required. Disasters could occur anywhere and at any time [1,2]. Disasters include not only accidents in hospitals (eg, an outbreak of fire) but also natural and human-made catastrophic events occurring outside of hospitals. In-hospital disaster preparedness must be performed fully on the spot, and health care professionals are the front line of disaster preparedness by controlling disaster situation and caring for victim patients. Human history has demonstrated the serious effects of poor disaster preparedness [3]. In the case of the aftereffect of Hurricane Katrina, the emergency evacuation at the Memorial Medical Center in New Orleans was unexpectedly conducted and poor evacuation management resulted in tremendous loss of life [4].

A fundamental component of effective in-hospital disaster preparedness is appropriate, repetitive experiential education and training of health care professionals with live disaster situations [5]. Disasters, however, are rarely encountered clinically, making it ideal for simulation. Live drills enable trainees to practice in a realistic simulation, where they can experience mock-up scenarios and learn sequences of actions and tasks to deal with the situations. Live drills are available when disaster simulations are set through the process of implementing everything realistically, including participants, infrastructures, and medical devices and tools. One of the live drill examples is the timely and effective evacuation practice of neonates during an outbreak of fire [6]. Here, fast exit routing by newborn intensive care unit workers while caring for mannequins of neonates had to be developed. Live drills have a great advantage in that they can take effect in a real disaster situation. Cost restraints and logistic constraints, however, make implementing simulations and scenarios difficult, and training opportunities with live drills may be severely limited [7,8].

Having lectures and practice with learning materials is another approach. Its advantages are that it is simple and does not require time and cost on a large scale. The simulations and instructions, however, may be fragmentary (eg, via verbal description or simple figures), and the learning outcomes may not be so effective in a real disaster situation [9].

Emerging evidence suggests that virtual reality simulation (VRS) is a viable alternative for workforce training in a variety of industry disciplines, with positive effects of increasing confidence and gaining necessary knowledge [10-13]. VR is a cutting-edge technology that can generate virtual environments of different training scenarios and combine with multisource data, stereoscopic vision, and intuitive interaction interfaces [14]. VR enables trainees to take part in virtual scenarios using avatars, in which they can simulate the sequences of actions and tasks they may make and reflect on the consequences of their choices [15,16]. Its realistic and interactive characteristics can greatly enhance trainees’ perception level and interest in learning [17]. The current generation of VR technologies is now equipped with head-mounted displays, where the wearers can obtain fully immersive experience of VRS [18]. There are various advantages of the use of VRS in education and training as follows: (1) repeatability, where training practices can be repetitive until goals are achieved [19]; (2) dynamic expansion and update of training scenarios due to technical advances to reconstruct a variety of training scenarios in a realistic manner [20,21]; and (3) just-in-time training, where training can be conducted anywhere and at any time if VR devices are available [22].

The purpose of this paper is to conduct an integrative review of peer-reviewed literature that applies VRS for in-hospital disaster preparedness training. The research problems this integrated review identified are follows: For use in in-hospital disaster preparedness, (1) what kinds of VRS training programs have been presented?; (2) are there any studies that demonstrate the usefulness of the VRS training in a quantitatively or qualitatively manner, and does this usefulness analysis include technological feasibility, training effects, and cost saving of VRS?; and (3) are there any studies that identify the pros and cons of VRS training when compared to conventional alternatives (eg, live drills or material-based lectures)?

This new integrated review builds upon and complements a prior integrated review publication by Miller et al [23]. In 2013, they analyzed peer-reviewed literature published during the period of 2005-2012, with findings that the efficacy of VRS for its use in disaster preparedness training was identified but such that the findings were difficult to be generalized due to the small volume of the literature. This new integrated review analyzed state-of-the-art literature published in the past 15 years, where there have been massive technological advances in VR engineering [17]. This motivated us to investigate the current progress in disaster preparedness training programs using VRS in terms of its wide adoption, practical usages, and efficiency. Compared to the prior publication [23], this new integrated review narrowed down the application domains and was only limited to in-hospital events and scenarios (eg, emergency departments) where health care professionals should equip patient triage and treatment skills for mass casualty incidents (MCIs) [24], which are critical and should be timely and carefully managed.

Methods

Study Design

This study followed the well-known methodology suggested by Whittemore et al [25] to conduct an integrative review. This review methodology consists of (1) problem identification, (2) a literature search and inclusion criteria definition, (3) validation of searched studies, and (4) presentation of findings. Two investigators—an assistant professor (author YJ) and a senior student of computer science—were involved in this study.

Problem Identification

The investigators agreed on the purpose and boundaries of this study at the initial stage of the research meeting. It seems that health care professionals do not have enough opportunities for in-hospital disaster preparedness training [7,8]. The investigators also agreed on recent advances of VRS for its use in education and training in a variety of application industry domains [10-13]. The research problems mentioned in the Introduction section were then derived based on their agreement.

Literature Search and Inclusion Criteria Definition

The literature search was performed by the investigators independently. They used 3 keywords: (1) “virtual reality,” which represents a key technology to be used for implementation of in-hospital disaster preparedness training programs; (2) “disaster” or “fire,” which represents catastrophic events to be simulated for training programs; and (2) “health care” and “hospital,” which represent major responders when the training programs are executed. The combination of these keywords can cover the volume of relevant literature for this integrated review. Note that the chosen keywords—“disaster” and “fire”—sufficiently covered different possible disaster types, where “disaster” represents natural and human-made catastrophic events (eg, earthquake and MCIs) and “fire” represents accidents the majority of the literature considers as catastrophic events occurring in hospitals.

A computerized search approach was used with 3 websites: Web of Science (WOS), PubMed (PMD), and Embase (EMB). WOS is an established publication repository for VR technologies (technology domains), and the other 2 are related to the medicine application domain. Such inclusion may decrease the possibility of missing relevant literature. The advanced search function of each publication repository was used as a search option as follows:

• The WOS search with its Web of Science Core Collection database and without any restrictions on document types
• The PMD search with all fields and without any other options or filters
• The EMB search with its default options

Inclusion criteria were as follows:

• Peer-reviewed empiric studies published from 2006 to 2020 and written in English
• All VRS training programs related to disaster preparedness
• Participants of training programs being all types of health care professionals
• Place of training programs anywhere in hospitals

Validation of Searched Studies

The final studies to be included were determined through a 2-step validation procedure: (1) Abstracts of searched studies were initially reviewed to confirm whether they met the inclusion criteria, and (2) full papers of the abstract-filtered studies were printed and read in their entirety for inclusion. Many of the studies have contributed to the development of VRS training programs, and there was a lack of evaluating and analyzing their usefulness and efficacy in real practice scenarios. These were excluded since the purpose of this integrated review was to suggest the scientific evidence pertaining to the practical values of VRS training programs. The investigators independently performed the validation procedure. Discrepancies were resolved through active discussion among the investigators.

Presentation of Findings

A matrix of the presentation for validated final studies was used to reach an integrated result. The matrix consisted of authors and dates of publication, study design, disaster type, participant details, contents, and outcomes. The investigators believed that this categorization would be useful to capture and promote the understanding of features of VRS disaster preparedness training programs and their usefulness and practical values. For quasi-experimental studies, it is important to identify differences and similarities, describe variables, and clarify intervention factors to make logical connections. All of these were carefully considered in this integrated review.

Results

Study Selection

There were 99 results (studies) from the literature search of the 3 publication repositories, as shown in Table 1. After study screening using the inclusion criteria and redundancy removal, 12 (12%) studies in total were included in this integrated review: 4 (24%) studies were included from 17 studies in WOS, 6 (26%) from 23 studies in PMD, and 2 (3%) from 59 studies in EMB.

Study Review

The summarization for each of the 12 studies [6,24,26-35] is shown in Table 2 in ascending order of publication year. Here, there were 4 categories: authors and publication years, experiment, content, and outcomes. In the experiment category, data including the status of the randomized controlled trial (RCT) and disaster type and study design were described. In addition, the number of participants was provided, with a distribution of their role—either the experiment group (EG) with VRS training or the control group (CG) without VRS training—if a quasi-experimental study was conducted. Prior to experiments, both groups were taught through a conventional training approach (eg, lecture with learning materials) and an additional VRS training session was delivered only to the EG.

In Table 3, statistical characteristics of the 12 studies [6,24,26-35] are shown according to predefined categories.

Discussion

Principal Findings

All the 12 studies suggested positive outcomes of VRS from its use for in-hospital disaster preparedness training. Since the outcomes varied, the interpretation for the studies was multidimensional, focusing on 3 major themes: VRS training’s (1) realism, (2) learning effect, and (3) cost saving.

Realism of VRS Training

The definition of realism was how the health care participants felt that the training environment provided by VRS was like their actual clinical environment and medical practice. There were 4 (33%) of 12 studies [27,28,30,33] where the analysis and evaluation of VRS training programs with regard to realism was explicitly conducted. These studies [27,28,30,33] demonstrated the capabilities of VR technology in realistically simulating clinical scenarios and medical practices related to disasters. The confirmatory factorial analysis by Loverglio et al [33] suggested that such the replicability is the main contributing factor in the sense of realism. Here, the health care participants could interact with sequences of actions and tasks required to deal with the in-hospital disaster scenarios [27,28,30]. These findings may indicate that VR technology enables immersive training and learning in the clinical domain. This is consistently found in prior integrated reviews in medicine [23] as well as those in other nonmedical industry domains [12,13].

In a prior integrated review by Miller et al [23], triage and treatment training from human-made MCIs was only for application practices. This limited applicability was mitigated in the current integrated review by including new programs of building evacuation safety training from natural catastrophic events, such as earthquakes [33]. The technological advances in VRS during the past 15 years may be 1 of the contributing factors in the application to clinical scenarios that require sophisticated modeling and environment depiction.

There were complains about the VR interface from an early study by Heinrich et al [27] in 2008, but no such complaints were discovered in recent studies until 2020. This might imply that the technological maturity of VRS is at a level that includes consideration of extraneous variables, such as technological sophistication of the participants and ease of navigation.

Learning Effects of VRS Training

In total, 10 (83%) of 12 studies [6,24,26-32,34] (2 studies by Ferra et al [35] and Lovreglio et al [33]) evaluated VRS for its learning effects on in-hospital disaster preparedness training. Learning effects on increased confidence and enhanced knowledge acquisition were captured in all the 10 studies [6,24,26-32,34], and such learning effects were consistently found in previous integrated reviews in medicine [23] as well as other nonmedical industry domains [12].

In a prior integrated review by Miller et al [23], the learning effects were limited in triage and treatment knowledge and skills because all included studies focused on those practices. The larger variation of applied practices was found in our integrated review; it included not only triage practices but also training programs on fire prevention and safety, and emergency evacuation planning and execution. Through this new integrated review, it was found that the learning effects from the use of VRS can be generalized to a variation of medical training practices.

There were 4 (33%) quasi-experimental studies [6,31,32,34] where EG participants with VRS training were analyzed and directly compared with their CG counterparts without VRS training. These studies [6,31,32,34] consistently showed no significant difference between the 2 groups in posttest evaluation on self-efficacy or knowledge acquisition. The EG participants, however, showed statistical or clinical performance enhancement in gained knowledge and task completion efficiency when evaluated in final live drills (eg, performing the correct sequence of steps in extinguishing a simulated fire) [32]. This result suggested the practical usefulness of VRS and its learning effects on real disaster scenarios that health care professionals may encounter in the future.

One of the important learning effects is increased learning retention [36,37]. The prior integrated review by Miller et al [23], however, could not derive any findings on it due to a lack of studies. In this integrated review, there was 1 (8%) study [31] that took the learning retention effect into consideration. The study [31] conducted a longitudinal experiment with repeated measures taken during a year, and the results showed improved knowledge gaining as training progressed. This statement, however, is somewhat difficult to be generalized due to the insufficient volume of longitudinal experimental studies. This insufficiency may be, in part, attributed to the fact that inputs and efforts, including participants, facilities, and program management, are much greater and have to be maintained for a longer period when compared to 1-shot studies.

Cost Saving of VRS Training

Cost analysis of the training program implementation and operation is 1 of the important factors training bodies have to consider, but it was not part of the previous integrated review by Miller et al [23]. In this integrated review, 2 (16%) of 12 relevant studies [26,35] suggested consistent findings for the cost advantage in implementing and operating VRS training programs over the live drill alternative. The larger initial investment from VRS training was undeniable, but it could be scalable and compensated for its longer-term use with the large number of participants. A comparative numerical study by Farra et al [35] found that the initial additional cost from VRS training was $97.99 when compared to the live drill alternative, and this initial investment provided a benefit of $114.65 after 3 years of program operation. Farra et al [35] attributed this to the repeatability feature of VRS anywhere and anytime, enabling just-in-time training.

Other Aspects of VRS Training

This integrated review of 12 studies [6,24,26-35] showed the applicability of VRS to different training scenarios of in-hospital disaster preparedness, including triage and transport of victim patients, emergency treatment, fire extinguishing, and building evacuation. These, however, do not entirely cover all disaster scenarios that could happen in hospitals. Investigating VRS with other challenging clinical scenarios that require sophisticated environment depiction and complicated training tasks (eg, patient protection and evacuation in scenarios of water entering a hospital due to floods) is an interesting future direction for effective in-hospital disaster management. The recent maturity of VR technologies in modeling and rendering and the development tools would be able to address the complexity of different clinical scenarios.

The number of VRS studies on in-hospital disaster preparedness training programs is still limited. Compared to the previous integrated review by Miller et al [23], the number increased from 10 to 12. Considering that Miller et al [23] collected all the VRS studies, including training scenarios occurring outside hospitals and in-hospital events, the increase in VRS studies on in-hospital training programs is noticeable. In addition, it was observed that the number of studies published during the past 5 years (from 2016 to 2020) was greater than that published in the older but longer period between 2006 and 2015 (ie, 7 [29-35] vs 5 studies [6,24,26-28]).

The scale of participants is 1 of the important variables to draw conclusive findings from studies. There were 4 (33%) of 12 studies [6,31,33,35] that considered the scale of participants by conducting VRS training programs with more than 50 participants. This increased from 2 in the previous integrated review by Miller et al [23] and was double. In addition, all 4 studies [6,31,33,35] were conducted in the recent 5 years. These observations suggest that future studies would put more emphasis on this important variable and the potential outcomes can be more conclusive.

It was observed that 7 (58%) of 12 studies [29-35] since the past 5 years used consistent development platforms, such as the Virtual Electrosurgical Skill Trainer (VEST) and Unity, for the clinical implementation of VRS training programs. The use of established development platforms enables rapid implementation, easy scenario replication, and, therefore, increased feasibility in conducting VRS-based clinical training studies. This observation can somewhat explain the increasing trends in the number of VRS studies and studies with large-scale participants.

It was also observed that the details of evaluation processes were not fully described. There were only 3 (25%) of 12 studies [24,28,30] that explicitly stated the skill levels and experiences of health care participants, and 1 (8%) study [26] even lacked details of the participant number. Demographic details can be 1 of the influential variables to allow in-depth interpretation of outcomes from studies (eg, how learning effects work for different levels of expertise, as shown in the study by Pucher et al [28]). Similarly, qualitative analysis in the participants’ experience was described using Liker-type surveys, postexperience interviews, and focus group open discussions, where there were little details on the reliability and validity of coding of qualitative data from interviews or focus groups. Such a lack was consistently found in the previous integrated review by Miller et al [23]. This integrated review, therefore, suggests that process details need to be sufficiently described in future studies, which will facilitate future investigators to draw deep insights into the value of VRS.

It was noted that the baseline performance comparisons of different studies highly varied: 3 (25%) of 12 studies compared VRS training programs among participants where some had previous disaster preparedness training experiences prior to study enrollment; 6 (50%) studies [6,26,27,30-32,34,35] compared VRS training effects against the experiences of live drills; and the remaining 3 (25%) studies [28,29,33] did not explicitly state the comparison approach. This may suggest that some efforts on developing the standard of VRS training program comparison protocols are meaningful, and this will allow for objective and quantitative comparison among different studies and draw general insights that are applicable to different disaster training programs.

Conclusion

The findings from this integrated review suggest that VRS could be a viable, cost-effective approach for health care professional training in in-hospital disaster preparedness. The reproducibility, just-in-time training, and repeatability features of VRS, along with its low cost of clinical implementation, suggest that VRS potentially represents a competitive adjunct to existing training approaches.

As VR continues to evolve in all technological aspects, it is anticipated that studies using VRS can become more vitalized in the clinical domain, while addressing currently unsolved issues. As an example of issues, studies with a massive number of participants with sophisticated assessment tools can be performed with more detailed and rigorous interventions and measurement of long-term retention.