Background: Immersive virtual reality (IVR) applications are gaining popularity in health care education. They provide an uninterrupted, scaled environment capable of simulating the full magnitude of sensory stimuli present in busy health care settings and increase students’ competence and confidence by providing them with accessible and repeatable learning opportunities in a fail-safe environment.
Objective: This systematic review aimed to evaluate the effects of IVR teaching on the learning outcomes and experiences of undergraduate health care students compared with other teaching methods.
Methods: MEDLINE, Embase, PubMed, and Scopus were searched (last search on May 2022) for randomized controlled trials (RCTs) or quasi-experimental studies published in English between January 2000 and March 2022. The inclusion criteria were studies involving undergraduate students majoring in health care, IVR teaching, and evaluations of students’ learning outcomes and experiences. The methodological validity of the studies was examined using the Joanna Briggs Institute standard critical appraisal instruments for RCTs or quasi-experimental studies. The findings were synthesized without a meta-analysis using vote counting as the synthesis metric. A binomial test with P<.05 was used to test for statistical significance using SPSS (version 28; IBM Corp). The overall quality of evidence was evaluated using the Grading of Recommendations Assessment, Development, and Evaluation tool.
Results: A total of 17 articles from 16 studies totaling 1787 participants conducted between 2007 and 2021 were included. The undergraduate students in the studies majored in medicine, nursing, rehabilitation, pharmacy, biomedicine, radiography, audiology, or stomatology. The IVR teaching domains included procedural training (13/16, 81%), anatomical knowledge (2/16, 12%), and orientation to the operating room setting (1/16, 6%). The quality of the 75% (12/16) of RCT studies was poor, with unclear descriptions of randomization, allocation concealment, and outcome assessor blinding procedures. The overall risk of bias was relatively low in the 25% (4/16) of quasi-experimental studies. A vote count showed that 60% (9/15; 95% CI 16.3%-67.7%; P=.61) of the studies identified similar learning outcomes between IVR teaching and other teaching approaches regardless of teaching domains. The vote count showed that 62% (8/13) of the studies favored using IVR as a teaching medium. The results of the binomial test (95% CI 34.9%-90%; P=.59) did not show a statistically significant difference. Low-level evidence was identified based on the Grading of Recommendations Assessment, Development, and Evaluation tool.
Conclusions: This review found that undergraduate students had positive learning outcomes and experiences after engaging with IVR teaching, although the effects may be similar to those of other forms of virtual reality or conventional teaching methods. Given the identification of risk of bias and low level of the overall evidence, more studies with a larger sample size and robust study design are required to evaluate the effects of IVR teaching.
Trial Registration: International prospective register of systematic reviews (PROSPERO) CRD42022313706; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=313706
Clinical competency is essential for all health care professionals. Health care students, particularly undergraduate students with limited clinical experience, must incorporate theories into their skills with a good professional attitude to attain clinical competency. In addition to conventional classroom teaching for theoretical knowledge inputs and laboratory skill practice, the clinical practicum is crucial in undergraduate health care education to develop the clinical competency of students . However, clinically based degree programs such as medicine, nursing, and physiotherapy are facing faculty shortages and increasing demands on clinical venues for student clinical placements [ ]. In addition, clinical practicums for undergraduate health care students have been suspended in many countries because of the COVID-19 pandemic, further diminishing learning opportunities for students [ ]. Therefore, there is an ongoing need for accessible, cost-effective, and high-quality methods of education to overcome these resource limitations [ ].
Simulation-based learning is useful for teaching in all domains (ie, knowledge, skills, and attitudes) relevant to the education of health professionals . It is a well-applied teaching and learning strategy [ ] for increasing training opportunities and enhancing learning efficiency [ ]. Simulation-based learning facilitates a deeper understanding of theoretical knowledge, relationships between different concepts, advanced inquiry, problem-solving, and decision-making [ ]. It allows health care students to practice procedural skills without compromising patient safety and improves the quality of their patient care [ ]. Many studies have shown that simulation-based learning effectively advances students’ competencies in acquiring diagnostic and psychomotor skills [ , ]. However, it is not aimed at replacing the clinical practicum but at better engaging and preparing students for it.
Immersive virtual reality (IVR) applications for simulation-based training are gaining popularity in health care professional education [, ]. They provide an immersive learning experience with a first-person viewpoint in a 3D virtual environment supported by head-mounted displays or Cave Automatic Virtual Environments (room-sized cube virtual reality [VR] environments) [ , ]. They provide direct sensory feedback or reactions to users based on their physical actions.
Compared with nonimmersive VR (such as computer-based simulation games) or other forms of traditional simulation-based training (such as skill laboratories using mannequins and simulators), only IVR provides learners with the perception of being physically present in a synthetic world . It provides an uninterrupted, scaled environment capable of simulating the full magnitude of sensory stimuli present in busy health care settings such as operating rooms, emergency departments, and surgical or medical wards [ ]. Owing to the uniquely high level of immersion that IVR offers, students can practice a particular procedure or rehearse a specific action with a high level of physical and psychological fidelity, with immediate and standardized feedback provided corresponding to the users’ actions. IVR can increase the competence and confidence of students by providing them with accessible and repeatable learning opportunities in a fail-safe environment [ ].
The COVID-19 pandemic has led to a paradigm shift in all clinically based undergraduate programs, prompting adaptations through better use of educational technology, in which IVR is a promising alternative . It is believed that the use of IVR-based simulation learning in health care professional training will persist beyond the pandemic [ ]. A small number of systematic reviews have reported the efficacy of VR-based simulation learning in improving the knowledge and skill competence of students compared with conventional simulation-based training [ , , ]. However, the learning approaches included in these reviews were mainly non-IVR approaches. Recently, a systematic review of 17 studies involving 307 surgical trainees reported significantly improved procedural completion times and greater postintervention scores on procedural checklists compared with conventional teaching methods such as watching standardized surgical training videos [ ]. As previously mentioned, simulation-based learning, including the use of IVR, should be useful for learning in different domains (knowledge, skills, and attitudes) [ ]. However, the extent to which IVR applications are used for teaching in different learning domains and their effectiveness in undergraduate health care professional education remain unknown [ ].
Therefore, the aim of this review was to evaluate the effects of IVR applications in improving the learning outcomes and experiences of undergraduate health care students compared with other teaching methods. Two review questions were devised: (1) What are the effects of IVR applications on improving students’ learning outcomes in different teaching domains (such as knowledge, skills, and attitudes) compared with other teaching methods? (2) What are the effects of IVR applications on enhancing the learning experiences of students (such as their level of satisfaction, perception of IVR innovation, and self-perceived competence and confidence) compared with other teaching methods?
This systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses;) [ ] and SWiM (Synthesis Without Meta-analysis) [ ] guidelines. The review protocol was registered in PROSPERO (CRD42022313706).
We used the Population, Intervention, Comparison, Outcome, and Study Design (PICOS) structure to define the eligibility criteria ().
Inclusion and exclusion criteria.
- Inclusion criteria
- Population: the participants were undergraduate students majoring in professional health care fields such as medicine, nursing, and physiotherapy who were included without regard to sex or race.
- Intervention: the teaching methods used in the studies focused on immersive virtual reality (IVR) applications using either fully immersive 360° IVR, such as head-mounted display products, or Cave Automatic Virtual Environment systems, such as HTC VIVE, Samsung Oculus Rift, or other similar programs.
- Comparison: we aimed to determine whether other teaching methods would have similar effects to IVR applications in professional health care education. For the purpose of comparison, we classified the different teaching methods into either active or passive comparison groups. The active comparison groups included other forms of simulation-based training such as nonimmersive virtual reality (VR), computer screen–based simulators, and real patient simulation. The passive control groups included mainly conventional teaching methods such as face-to-face lectures, tutorial classes, role-play, and reading materials.
- Outcome: we determined the effects of IVR applications on the learning outcomes (ie, the primary outcome) and experiences (ie, the secondary outcome) of students. The primary outcome refers to the change in the students’ theoretical knowledge and procedural skills, as reflected by any form of written examination or clinical skills such as communication skills, completion time, and error rate in a specific clinical procedure. The secondary outcome was the students’ learning experience, as assessed by their satisfaction levels, perception of IVR innovations, and self-perceived competence and confidence after receiving IVR.
- Study design: this review included randomized controlled trials and quasi-experimental studies published in English between January 2000 and March 2022 as IVR technology has developed enough to be used in health education since 2000 [ ].
- Exclusion criteria
- Studies were excluded if their target population was postgraduate health care students or students of other subjects unrelated to health care. Studies that involved only other forms of VR, such as nonimmersive VR, or other types of simulation-based technologies, such as augmented reality and manikins, were excluded. Pilot and feasibility studies were also excluded.
A combination of Medical Subject Headings and free-text terms was used to search through 4 databases, namely, MEDLINE, Embase, PubMed, and Scopus, for potentially relevant abstracts. In addition, hand searches were conducted by reviewing the reference lists of all papers selected for inclusion in this review from the electronic databases, Google Scholar, and hard copies in university libraries to identify any articles missed by the database search. An alert for updated articles in each database was set to avoid missing potential up-to-date studies.
Search strategies were developed according to the 2 primary concepts of this review: the use of IVR applications in health professional undergraduate education and their effectiveness in enhancing the learning outcomes and experiences of students. To identify studies that used IVR applications, we used search terms such as “Immersive Virtual Reality” or “Simulated environment” and “Simulation” and “Healthcare” and “Students” or “Undergraduates” or “Trainees” (). These terms were revised appropriately for different databases.
The search results were imported into the EndNote bibliographic software (version 20; Clarivate Analytics), and duplicate studies were removed. The titles and abstracts of all the identified studies were screened independently by 2 researchers (JYWL and YHY) to identify potentially relevant papers based on the review criteria. Both researchers compared the preliminary results of the review to reach an agreement. Full-text articles were then obtained and screened independently. Any disagreements between the reviewers were then discussed among the members of the research team to reach a consensus.
Data Collection Process and Data Items
A specific data extraction matrix was created to collect information from each included study, including the author, year, title, country of origin, demographic data of the participants (age, sex, and type of health profession), methodological data (aims of the study, sample size, study design, educational innovation, and comparison groups), and outcome data (primary outcome, eg, examination scores; secondary outcome, eg, self-perceived competence and confidence). If any original study had been published in more than one paper, the information was extracted as 1 study based on the study protocol number. The authors of the primary studies were contacted when clarifications were required or if any information was missing. Data extraction was conducted independently by the same 2 researchers. Once complete, the results were compared, and discrepancies were resolved through discussion. A final extraction table was developed.
Study Risk-of-Bias Assessment
The studies selected for this review were independently assessed for methodological validity by the reviewers involved in their selection using the Joanna Briggs Institute (JBI; University of Adelaide, Australia) standard critical appraisal instruments for randomized controlled trials (RCTs) or quasi-experimental studies.
We assessed the included RCTs based on the following 12 appraisal items: methods of randomization; treatment allocation and concealment; similarity of characteristics between groups at baseline; blinding procedures for participants, interventionists, and outcome assessors; whether the comparison groups were treated identically other than in the intervention of interest (ie, IVR applications); completeness of the follow-up (ie, if there was any bias because of missing data); consistency and reliability of the outcome measurements; and appropriateness of the statistical analysis and trial design (JBI). For example, to determine if there were any biases because of missing data, we checked whether there were differences between groups with regard to the loss to follow-up (numbers or proportions, reasons, any analysis of patterns of loss to follow-up, and their impact on the internal validity of the study). We assessed the included quasi-experimental studies based on the following 9 appraisal items: clarity of dependent and independent variables, similarity of characteristics between groups at baseline, whether comparison groups were treated identically other than in the intervention of interest (ie, IVR applications), any comparison group, any pre- and postoutcome measurements, completeness of the follow-up (ie, whether there was any bias because of missing data), consistency and reliability of the outcome measurements, and appropriateness of the statistical analysis.
Working independently to assess for risk of bias, the same 2 assessors rated each item as yes, no, unclear, or not applicable. Any disagreements on the results of the bias assessment were then reviewed and discussed by the research team until a consensus was reached.
The aim of this assessment of the risk of bias was to determine the quality of each study. However, the risk of bias was not used as a criterion for the inclusion of a study in this review. A trial was judged to be at a low risk of bias overall when all items were rated as yes. Conversely, a study was judged to be at a high risk of bias when it reported on a procedure that could be judged as being a no or unclear in any item. Owing to the nature of the intervention, it was impossible to blind the participants; thus, we did not include the blinding procedures for participants item when determining a study’s overall risk of bias .
Criteria for Grouping Studies
Following the review questions, studies for synthesis were grouped according to the study outcomes (ie, learning outcomes for research question 1 and learning experience for research question 2). In addition, studies with the same teaching domains (ie, procedural skills vs theoretical knowledge) were grouped for the analysis of the 2 outcomes.
Standardized and Synthesis Metrics and Method
The direction of the effects (learning outcomes and learning experiences) was used as the standardized metric as there was a lack of precision, which was specific to the effects of the intervention (IVR teaching) and control on the results presented by different studies. This did not allow for the calculation of summary statistics . In addition, the clinical and methodological characteristics (such as populations, intervention components, and the choice of outcome measurements and study designs) of each study were used to evaluate the heterogeneity based on the Cochrane Handbook for Systematic Reviews of Interventions [ ]. The included studies were highly heterogeneous, with diverse teaching aims and outcome assessments to fit the needs of students from different health care professions. Different IVR features were adopted, with different frequencies and durations. In addition, most of the included studies were rated as having a high risk of bias. As a result, we felt that it was not appropriate to conduct a meta-analysis. Therefore, vote counting was the best match for synthesizing the results. A binomial test using SPSS (version 28; IBM Corp) was used to indicate whether there was evidence of an effect [ ]. The quality of the evidence generated by different studies was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) tool [ ].
A total of 982 articles were found by searching the databases; after the removal of duplicates (n=344, 35%), 638 (65%) were left. After screening the titles and abstracts, 72.6% (463/638) of the articles were excluded, leaving 175 to be retrieved for a full-text screening. Following the full-text screening, a further 90.9% (159/175) of the articles were excluded, leaving 17 articles [- ]. Of the 17 articles, 2 (12%) [ , ] were from the same study. Therefore, a total of 16 studies were included in this systematic review. Details of the selection process and the reasons for the exclusion of articles are presented in the PRISMA flowchart ( ).
The characteristics of the included studies are shown in. The 16 studies were conducted between 2007 and 2021. An RCT design was adopted in 75% (12/16) of the studies [ , , - , - , - , ], and a quasi-experimental design was adopted in 25% (4/16) of the studies [ , , , ]. All 12 RCTs were 2-armed, among which were 8% (1/12) [ ] that had a crossover design. Of the 4 quasi-experimental trials, 1 (25%) [ ] was a 4-armed study, 1 (25%) [ ] was a 3-armed study, and the other 2 (50%) were 2-armed studies [ , ].
A total of 1787 participants were involved in the included studies, with the number ranging from 25 to 289 in the RCTs and from 29 to 197 in the quasi-experimental studies. In total, 44% (7/16) of the studies involved sample sizes of <60. Female students accounted for 71.73%, with 25% (4/16) of the studies [, , , ] not disclosing the sex distribution. The participants ranged in age from 19 to 35 years, whereas the age distribution was not discussed in 25% (4/16) of the studies [ , , , ]. The types of participants included undergraduate students majoring in medicine, nursing, rehabilitation, pharmacy, biomedicine, radiography, audiology, or stomatology.
IVR teaching was compared with other forms of VR such as desktop-based VR (ie, the active control) in 25% (4/16) of the studies [, , , ]. In 50% (8/16) of the studies [ , , - , ], IVR teaching was compared with conventional teaching methods such as verbal didactic instruction and hard-copy teaching materials (ie, the passive control). In total, 12% (2/16) of the studies [ , ] used passive and active controls. A total of 6% (1/16) of the studies [ ] adopted role-play, and another study (1/16, 6%) [ ] used a clinical practicum as their comparison group. These teaching approaches did not involve any use of VR, and we considered the role-play and clinical practicum groups as passive controls.
|Author, year, and country||Design||Sample size (percentage of female participants), N||Age (years)||Participants||Intervention (IVRa)||Comparator||Side effects|
|Gutiérrez-Maldonado et al , 2015, Spain||2-armed RCTb||Total: 52 (N/Ac); EGd: 26; CGe: 26||N/A||Undergraduate students (without a specific discipline)||N/A|
|Harrington et al [, ], 2018, Ireland||2-armed crossover RCT||Total: 40 (50); EG: 20; CG: 20||23||Preclinical undergraduate students (medicine, physical therapy, pharmacy, and biomedical sciences)||Self-reported levels of nausea were low (0.7 out of 10) but varied substantially from 0 to 8 among participants.|
|Smith et al , 2018, United States||Quasi-experimental trial||Total: 197 (88.2); EG: 59; active CG: 58; passive CG: 55||18-25 (73.3%); 26-34 (15.5%); 35-50 (11.2%)||Senior baccalaureate-degree nursing students||None|
|Zackoff et al , 2020, United States||2-armed RCT||Total: 168 (48.8); EG: 78; CG: 90||20-24 (34.5%); 25-29 (60.1%); 30-34 (4.2%); 35-39 (0.6%)||Year-3 medical students||None|
|Francis et al , 2020, United States||2-armed RCT||Total: 52 (89); EG: 26; CG: 26||≤23 (40.4%); ≥24 (59.6%)||Year-2 preclinical physician assistant students||N/A|
|Kurul et al , 2020, Turkey||2-armed RCT||Total: 72 (72.2); EG: 36; CG: 36||19||Year-1 physical therapy students||Vision discomfort (n=7), eyestrain (n=6), and general discomfort (n=6)|
|Gutiérrez et al , 2007, and Pierce et al [ ], 2008, United States||2-armed RCT||Total: 25 (N/A); EG: 13; CG: 12||N/A||Year-1 medical students||None|
|Sapkaroski et al , 2019, Australia||2-armed RCT||Total: 76 (75); EG: 38; CG: 38||21||Radiography students||None|
|Stepan et al , 2017, United States||2-armed RCT||Total: 66 (50); EG: 33; CG: 33||21-25 (89.4%); 26-30 (9.1%); 31-35 (1.5%)||Year-1 and year-2 medical students||None|
|Bakhos et al , 2020, France||Crossover quasi-experimental trial||Total: 29 (N/A); EG: 15; CG: 14||EG: 22; CG: 20||Year-1 audiology students (without beginning an internship)||None|
|Chao et al , 2021, Taiwan||2-armed RCT||Total: 45 (86.7); EG: 22; CG: 23||24||Nursing students (aged ≥20 years) who had never acquired the skills of NGk tube feeding||A total of 5 students (23%) reported feeling slightly dizzy, but this did not affect their ability to watch the video.|
|Berg and Steinsbekk , 2020, Norway||2-armed RCT||Total: 289 (78.5); EG: 149; CG: 140||<20 (23.8%); 20-24 (64.6%); >25 (11.6%)||Year-1 medical and nursing students (who had started their studies no later than 2 months before this study)||N/A|
|Berg and Steinsbekk , 2021, Norway||2-armed RCT||Total: 289 (84.6); EG: 146; CG: 143||<20 (29.6%); 20-24 (59.3%); >25 (11.1%)||Year-1 medical and nursing students (who had started their studies no later than 2 months before this study)||N/A|
|Collaço et al , 2021, Brazil||Quasi-experimental trial||Total: 163 (N/A); full: 42; NPm: 40; NTn: 40; NHo: 41||N/A||Clinical dental students||Most reported experiencing no side effects (87%) and “slight” (9%), “moderate” (3%), and “severe” (1%) symptoms.|
|Ros et al , 2020, France||2-armed RCT||Total: 173 (52); EG: 85; CG: 88||N/A||Year-4 medical students (2 years before residency)||N/A|
|Yu et al , 2021, Korea||Quasi-experimental trial||Total: 51 (92); EG: 26; CG: 25||22.4||Senior nursing students||N/A|
aIVR: immersive virtual reality.
bRCT: randomized controlled trial.
cN/A: not applicable.
dEG: experimental group.
eCG: control group.
fVR: virtual reality.
gLED: light-emitting diode.
hHMD: head-mounted display.
jCESTOL: Clinical Education Training Solution.
lABCDE: airways, breathing, circulation, disability, exposure.
oNH: nonhaptic feedback.
IVR Teaching Characteristics
A total of 81% (13/16) of the studies used IVR to train students in skills, including in techniques for diagnosing eating disorders  and respiratory distress in infants [ ], laparoscopic cholecystectomy [ ], decontamination skills [ ], physical examinations for patients with traumatic head injuries [ ], correct positioning for x-ray imaging [ ], audiometry techniques [ ], nasogastric tube feeding [ ], immediate assessment and treatment of patients who are critically ill [ , ], dental anesthesia skills [ ], external ventricular drainage [ ], and neonatal infection control [ ]. Only in 12% (2/16) of the studies was IVR used to teach students anatomical knowledge [ , ], whereas in 6% (1/16) of the studies, IVR was used to orient the students to the setting of the surgical operating room [ ].
Most studies (12/16, 75%) featured only a single IVR experience for the students. The exceptions were the study by Bakhos et al , which had 3 different IVR cases, and the study by Collaço et al [ ], who provided 2 IVR training episodes. Berg and Steinsbekk [ , ] allowed students to self-practice the skills but did not mention the number of sessions. The duration for students to experience IVR learning was typically short, ranging from 5 to 30 minutes in 56% (9/16) of the studies. In total, 38% (6/16) of the studies [ , , , , , ] allowed students to be exposed to IVR environments as long as they needed to complete the specific tasks or procedures. A total of 6% (1/16) of the studies [ ] did not state the exact duration of the students’ IVR learning session.
The IVR products used in the included studies were Oculus VR, Samsung Gear VR, Clinical Education Training Solution VR Clinic, HTC VIVE, and a university-created platform.
Side effects were reported in 25% (4/16) of the studies. Nearly half of the students in the study by Kurul et al  reported different forms of slight discomfort (ie, vision discomfort, eyestrain, and general discomfort). In the study by Chao et al [ ], 23% of the students reported feeling slightly dizzy, but this did not affect their viewing activities. In contrast, most students in another 12% (2/16) of the studies [ , ] did not experience any side effects.
Learning theory provides a framework to guide the development of teaching activities to help students imbibe, process, and retain the knowledge and skills that they have learned [, ]. When applied to educational IVR, a learning theory should provide a pedagogical framework and foundation for designing IVR-related teaching and learning strategies. However, most of the included studies (15/16, 94%) did not mention any theoretical approaches underpinning the development of IVR teaching. The exception was the study by Smith et al [ ], who used the National League for Nurses Jeffries Simulation Theory [ ] as their theoretical basis to guide the design of their teaching innovation. Most studies (15/16, 94%) supplemented the IVR lessons and tutorials by providing additional pedagogical practices or materials to encourage learning. In total, 25% (4/16) of the studies [ , , , ] included web-based modules and reading materials in addition to the IVR experience. The provision of either introduction or prebriefing or debriefing sessions before and after IVR learning was mentioned in 44% (7/16) of the studies [ , , , , , , ]. A total of 31% (5/16) of the studies [ , , , , ] used IVR as the sole method of learning.
Among the included RCTs, a risk of bias was identified in most domains, with the exception of the domains Q7, Q8, and Q10 to Q13 (). Only 25% (3/12) of the studies [ , , , ] gave clear details on the randomization procedure, whereas other studies simply briefly stated that the design was randomized without providing further information. Allocation concealment was not implemented or was unclear in 75% (9/12) of the studies [ , , - , , , , ]. In total, 33% (4/12) of the studies [ , , , ] did not report the baseline comparison between the groups, and in 8% (1/12) of the studies [ ], there were differences in age and practical experience between the groups. The blinding of participants, intervention providers, and outcome assessors was another major concern with the RCTs. In none of the studies were the participants blinded, and in only 17% (2/12) of the studies [ , ] were both the assessors and intervention providers blinded. Among all 16 evaluated items in the JBI checklist, there were 25% (3/12) of the studies in which 1 to 3 items were viewed as having a low risk of bias and 75% (9/12) of the studies in which 4 to 6 items were viewed as having a moderate risk of bias ( ).
For the quasi-experimental studies, the overall risk of bias was low (). However, in 25% (1/4) of the studies [ ], 13% of the participants failed to complete all the tests, and no detailed explanations were given about which groups were involved and how this issue was handled in the study. In addition, in 50% (2/4) of the studies [ , ], the baseline difference between the groups was not reported.
|Gutiérrez-Maldonado et al , 2015||Un||No||U||N||U||U||Yp||Y||Y||Y||Y||Y||Y|
|Harrington et al [, ], 2018||Y||U||U||N||U||U||Y||Y||Y||Y||Y||Y||Y|
|Zackoff et al , 2020||U||U||Y||N||N||U||Y||Y||Y||Y||Y||Y||Y|
|Francis et al , 2020||U||U||Y||N||U||U||Y||Y||Y||Y||Y||Y||Y|
|Kurul et al , 2020||U||U||Y||N||—q||Y||Y||Y||Y||Y||Y||Y||Y|
|Gutiérrez et al , 2007||U||N||Y||N||—||U||Y||Y||Y||Y||Y||Y||Y|
|Pierce et al , 2008||U||N||Y||N||—||U||Y||Y||Y||Y||Y||Y||Y|
|Sapkaroski et al , 2019||U||Y||U||N||U||Y||Y||Y||Y||Y||Y||Y||Y|
|Stepan et al , 2017||U||U||Y||N||—||U||Y||Y||Y||Y||Y||Y||Y|
|Chao et al , 2021||U||N||Y||N||N||U||Y||Y||Y||Y||Y||Y||Y|
|Berg and Steinsbekk , 2020||Y||Y||Y||N||Y||Y||Y||Y||Y||Y||Y||Y||Y|
|Berg and Steinsbekk , 2021||Y||Y||N||N||Y||Y||Y||Y||Y||Y||Y||Y||Y|
|Ros et al , 2020||U||U||U||N||U||U||Y||Y||Y||Y||Y||Y||Y|
aQ1: was true randomization used for assigning participants to treatment groups?
bQ2: was the allocation to the treatment groups concealed?
cQ3: were the treatment groups similar at baseline?
dQ4: were participants blinded to the treatment assignment?
eQ5: were those delivering treatment blinded to the treatment assignment?
fQ6: were outcome assessors blinded to the treatment assignment?
gQ7: were the treatment groups treated identically other than in the intervention of interest?
hQ8: was follow-up complete, and if not, were strategies used to address incomplete follow-ups (ie, was there an analysis of patterns of those lost to follow-up)?
iQ9: were participants analyzed in the groups to which they were randomized?
jQ10: were the outcomes measured in the same way for the treatment groups?
kQ11: were the outcomes measured in a reliable way?
lQ12: was an appropriate statistical analysis used?
mQ13: was the trial design appropriate, and were any deviations from the standard RCT design (individual randomization and parallel groups) accounted for in the conduct and analysis of the trial?
|Smith et al , 2018||Yj||Y||Y||Y||Y||Uk||Y||Y||Y|
|Bakhos et al , 2020||Y||U||Y||Y||Y||Y||Y||Y||Y|
|Collaço et al , 2021||Y||U||Y||Y||Y||Y||Y||Y||Y|
|Yu et al , 2021||Y||Y||Y||Y||Y||Y||Y||Y||Y|
aQ1: are the cause and the effect made clear in the study (ie, there was no confusion about which variable came first)?
bQ2: were the participants who were included in any comparisons similar to each other in any way?
cQ3: were the participants who were included in any comparisons receiving similar treatment or care other than the exposure or intervention of interest?
dQ4: was there a control group?
eQ5: were there multiple measurements of the outcome both before and after the intervention or exposure?
fQ6: was the follow-up complete, and if not, were the differences between the groups in terms of their follow-up adequately described and analyzed?
gQ7: were the outcomes of the participants included in any comparisons measured in the same way?
hQ8: were the outcomes measured in a reliable way?
iQ9: was an appropriate statistical analysis used?
Effects of IVR Teaching on Student Learning Outcomes
Of the 16 included studies (reported in 17 papers), 1 (6%) study using IVR for orientation did not include any assessment of student learning outcomes . Therefore, 94% (15/16) of the studies were retained to assess the primary objective (ie, the students’ learning outcomes) based on vote counting and a binomial probability test. Assuming that the true probability of favoring either IVR teaching or non-IVR teaching was equivalent to 0.05 under the null hypothesis (IVR teaching=non-IVR teaching on student learning outcomes), the results showed that 40% (6/15) of the studies [ , , , , , ] favored IVR teaching (95% CI 16.3%-67.7%; P=.61). The remaining 60% (9/15) of the studies [ - , - , ] showed similar effects between IVR and other teaching methods. These figures are below the expected binomial probability mean of 1.60 (SD 0.51) votes. Thus, we need to accept the null hypothesis.
Procedural Skill Outcomes
In total, 87% (13/15) of the studies adopted IVR to teach procedural skills. A total of 62% (8/13) of the studies [, , - , , ] showed that the improvement in the students’ acquisition of procedural skills was similar whether they were in the IVR groups or in the groups that used other teaching methods. In other words, 38% (5/13) of the studies [ , , , , ] indicated that students who received IVR teaching performed significantly better (95% CI 13.9%-68.4%; P=.58) than those who received other forms of teaching. These figures are also below the binomial probability mean of 1.62 (SD 0.51) votes; thus, the null hypothesis is accepted.
Theoretical Knowledge and Other Outcomes
A total of 13% (2/15) of the studies evaluated the effects of IVR on teaching theoretical knowledge [, ]. IVR teaching was found to have a greater effect on the acquisition of theoretical knowledge compared with the passive control in 50% (1/2) of the studies [ ], but in another study [ ], the effects of IVR were shown to be similar to those of the passive control. Owing to the small number of studies (only 2 studies), we did not run the binomial test. One study [ ] used IVR for orienting students to the operating room, but the students’ knowledge was not tested afterward. In none of the studies was IVR used to enhance the students’ professional attitudes.
Effects of IVR Teaching on the Students’ Learning Experience
Of the 15 studies, 2 (13%) did not measure students’ learning experience [, ], 1 (7%) explored students’ satisfaction and experiences with IVR through a focus group (only qualitative data were collected) [ ], and 1 (7%) only measured the students’ learning experience in the IVR group but not in the control group [ ]. These studies were excluded from the vote counting and binomial test. Of the 12 studies, 8 (67%) favored IVR teaching (95% CI 34.9%-90%; P=.59), whereas the remaining 4 (33%) showed similar effects on the students’ learning experience between IVR and other methods of teaching.
In total, 80% (12/15) of the studies [, , , , , - , ] evaluated the learning experiences of students from both the IVR and control teaching groups. Overall, the students said that they had a more positive experience learning with IVR than with other teaching methods ( ). A total of 67% (8/12) of the studies [ , , , - , ] showed that the students favored IVR teaching, and 33% (4/12) of the studies [ , , , ] reported that the learning experiences of students who learned with IVR were similar to those of either the active or passive controls (95% CI 34.9%-90.1%; P=.39). These figures are also below the binomial probability mean of 1.33 (SD 0.49) votes; thus, the null hypothesis is accepted. In 17% (2/12) of the studies [ , ], only the experimental group was evaluated, and both studies showed that students liked the approach of using IVR to learn. The effect direction plot of the different studies, together with the associated risk of bias, is shown in .
|Study, year, and country or region||Type of IVR||Outcome measurements||Major findings||Learning outcomes favor group||Learning experience favor group|
|Gutiérrez-Maldonado et al , 2015, Spain||Oculus Rift DK1||Primary outcome: a diagnostic interview skill test; secondary outcome (usability): SUMIa||Primary and secondary outcome: the mean score of the EGb was higher than that of the CGc, but the difference between the 2 groups was nonsignificant (P=.23; P=.89).||Similar||Similar|
|Harrington et al [, ], 2018, Ireland||Samsung Gear VRd||Primary outcome: knowledge retention (8-point multiple choice); secondary outcomes: (1) attentiveness (engagement) and (2) appraisals||Knowledge retention: no significant variances (P=.14). Attentiveness (engagement): a crossover analysis revealed a significantly higher level in the EG (P<.001) and across periods (P<.001) with no significant carryover effect (P=.97). Appraisals: two-thirds of participants reported choosing EG.||Similar||EG|
|Smith et al , 2018, United States||Oculus Rift Developer Kit 2||Primary outcomes: (1) decontamination knowledge gain, (2) skill performance, and (3) time spent by students completing the procedure; secondary outcome: qualitative focus group||Primary outcome: decontamination knowledge gain—no significant difference among the 3 groups at all 3 time points; skill performance—no significant difference among the 3 groups at the 2 time points; performance time—no significant difference among the 3 groups at all 3 time points; secondary outcome: high levels of satisfaction in the EG||Similar||Only measured the EG|
|Zackoff et al , 2020, United States||Oculus Rift||Primary outcomes: recognition or interpretation of key examination findings, assignment of an appropriate respiratory status assessment, and recognition of the need to escalate care for patients with impending respiratory failure by using a video-based assessment; secondary outcome: self-assessed competence||Primary outcome: significantly higher in the EG than in the CG; secondary outcome: 81% of the EG demonstrated an improvement in self-assessed competency.||EG||EG|
|Francis et al , 2020, United States||Oculus VR||Self-efficacy of the participants||Self-efficacy significantly improved in the EG (P=.007) but not for the CG (P=.30).||N/Ae||EG|
|Kurul et al , 2020, Turkey||3D Organon Anatomy (Medis Media) and 3D glasses (Oculus Rift; Oculus VR)||Primary outcome: written examination; secondary outcome: participants’ perceptions, including enjoyment and learning efficiency||Primary outcome: both the EG and CG had significantly higher posttest scores, but the difference between the pretest results was found to be significantly higher in favor of the EG (P<.001). Secondary outcome: 88.8% of students “enjoyed studying anatomy with IVR”; 83.3% of students felt that “it is easy to understand the location of structures with VR.”||EG||Only measured the EG|
|Gutiérrez et al , 2007, and Pierce et al [ ], 2008, United States||A platform created at the University of New Mexico||Primary outcome: written examination; secondary outcomes: the efficiency of the user interface and satisfaction with it||Primary outcome: posttest scores were significantly higher than pretest scores in both the EG and CG (within-group). There was a significant interaction between groups and time for the EG. Secondary outcome: there was no overall significant difference in efficiency and satisfaction between the groups.||EG||Similar|
|Sapkaroski et al , 2019, Australia||CESTOLf VR Clinic||Primary outcome: students’ performance in a skill test involving taking images of hand positions with an x-ray machine||The EG performed, on average, 36% (P<.001) better in digit separation, 11% (P<.001) better in palm flatness, and 23% (P<.05) better in central ray positioning onto the third metacarpal. There was no significant difference (P=.17) in positioning between the 2 groups.||EG||N/A|
|Stepan et al , 2017, United States||Oculus Rift||Primary outcome: anatomy knowledge quizzes; secondary outcome: participants’ subjective user experience via interview||Primary outcome: no significant difference was found between the EG and CG in the postintervention quiz (P=.87) or the retention quiz (P=.47). Secondary outcome: subjective learner experience survey—EG were more engaged (P<.01), felt more enjoyment (P<.01), and thought it was more useful for learning (P<.01); IMMSg survey—the EG scored higher in the total IMMS (P<.01) and the subscales for attention (P<.01), confidence (P<.01), and satisfaction (P<.01).||Similar||EG|
|Bakhos et al , 2020, France||Oculus Rift 1||Primary outcome: examination of 20 questions; secondary outcome: posttraining satisfaction and self-confidence||Primary outcome: mean posttraining test scores showed greater improvement in the EG, but the difference between the CG and EG was not significant. Secondary outcome: satisfaction and self-confidence ratings were significantly higher for the EG than for the CG.||Similar||EG|
|Chao et al , 2021, Taiwan||HTC VIVE||Primary outcome: nasogastric tube feeding quiz; secondary outcomes: (1) confidence scale and (2) satisfaction||There were no differences in knowledge (P=.84) and confidence (P=.96) between the 2 groups. Within groups, the scores on knowledge and confidence improved significantly in both the EG and CG immediately and 1 month after the intervention (P<.01). There was a significant difference in satisfaction levels between the intervention and comparison groups (t=2.30; P=.03).||Similar||EG|
|Berg and Steinsbekk , 2020, Norway||Oculus Rift S or Oculus Quest||Primary outcome: participants’ knowledge and performance of the ABCDEh approach (skill test); secondary outcome: students’ experiences||Primary outcome: noninferiority of the individual IVR conducting all observations in the correct order (EG vs CG: 24.8% vs 27.1%; absolute difference: 2.3% points, one-sided 95% CI 2.3%-10.8%); secondary outcomes were similar between the groups, but more students in the EG reported liking the way they practiced and stated that it was a good way to learn. The EG also scored high on the System Usability Scale.||Similar||EG|
|Berg and Steinsbekk , 2021, Norway||Oculus Rift S or Oculus Quest||Primary outcome: participants’ knowledge and performance of the ABCDE approach (skill test); secondary outcome: participants’ experience||Primary outcome: 29 (20%) participants in the EG and 30 (21%) participants in the CG answered everything correctly. Knowledge and performance of the ABCDE approach were similar in the 2 groups of students, except that the EG performed better in the report on respiratory frequency and in the usability test. Secondary outcome: the EG were more displeased about the learning experience, but the difference was not significant.||Similar||Similar|
|Collaço et al , 2021, Brazil||Samsung||Primary outcome: skill test; secondary outcome: the participants’ perceptions||Primary outcome: for the execution time—NPi and NTj took significantly longer than the full (EG) and NHk (P<.001) groups. The full (EG) and NH groups were more accurate in needle insertion than the NP and NT (P<.001). There were no significant differences among the groups in needle angle (P=.44) or needle depth (P=.24). Secondary outcome: no significant differences were found among the groups for factor 1 and factor 3. For factor 2, the NT group reported significantly (P<.001) more difficulty than the other groups (factor 1: tactile realism; factor 2: syringe control; factor 3: ease of performance).||EG||Similar|
|Ros et al , 2020, France||Samsung Gear VR||Primary outcome: written test about indications, patient management, and preparation until the incision||The EG had significantly better results (P=.01) in answering the questionnaire compared with the CG. The results were similar at 6 months (the scores in the EG were higher than in the CG but were nonsignificant).||EG||N/A|
|Yu et al , 2021, Korea||HTC VIVE||Student knowledge (HirNICCSl), self-efficacy (10-point scale), and satisfaction (5-point scale)||There was no significant difference between the EG and CG (P=.21) in knowledge. The EG showed a greater increase in self-efficacy than the CG(P=.02). The EG had a higher satisfaction score than the CG (P<.001).||Similar||EG|
aSUMI: Software Usability Measurement Inventory.
bEG: experimental group.
cCG: control group.
dVR: virtual reality.
eN/A: not applicable.
fCESTOL: Clinical Education Training Solution.
gIMMS: Instructional Materials Motivation Survey.
hABCDE: airways, breathing, circulation, disability, exposure.
kNH: nonhaptic feedback.
lHirNICCS: High-Risk Neonatal Infection Control Competency Scale Knowledge.
Certainty of Evidence
The generated GRADE evidence profile was used to present a synthesis of the findings regarding objective 1 (ie, students’ learning outcomes) and objective 2 (ie, students’ learning experiences) in. As there were serious concerns about most of the studies with regard to the study design, inconsistent results, and a strong suspected publication bias, all the evidence was considered to have a very low level of certainty.
|Outcome||Studies, n (%)||Study design||Risk of bias||Inconsistency||Indirectness||Other considerations||Summary of findings||Certainty|
|Students’ learning outcomes (assessed via examination and skill tests)||15 (94)||Randomized trials||Seriousc||Seriousd||Not serious||Publication bias strongly suspectede||6||9||EG<CG|
|Students’ learning on procedural skill outcomes (assessed via examination and skill tests)||13 (81)||Randomized trials||Seriousf||Seriousg||Not serious||Publication bias strongly suspectede||5||8||EG<CG|
|Students’ learning on theoretical knowledge and other outcomes (assessed via examination and skill tests)||2 (12)||Randomized trials||Serioush||Seriousi||Not serious||Publication bias strongly suspectede||1||1||EG=CG|
|Students’ learning experience (assessed via questionnaires)||12 (75)||Randomized trials||Seriousj||Seriousk||Not serious||Publication bias strongly suspectede||8||4||EG>CG|
aEG: experimental group.
bCG: control group.
cA total of 27% (4/15) of the studies that evaluated the students’ learning outcomes were quasi-experimental trials. According to the Joanna Brigs Institute critical appraisal checklist for randomized controlled trials, the remaining 11 randomized controlled trials had 1 to 6 items that were rated at no or unclear, which indicates that there were issues in the study design leading to a serious risk of bias.
dA total of 40% (6/15) of the studies favored immersive virtual reality teaching (P=.10 in a binomial probability test showing that the null hypothesis is accepted).
eNone of the papers registered their study protocol; therefore, we could not check if there was any publication bias.
fA total of 31% (4/13) of the studies were quasi-experimental trials. According to the Joanna Brigs Institute critical appraisal checklist for randomized controlled trials, the remaining 9 randomized controlled trials had 1 to 6 items that were rated at no or unclear, which indicates that there were issues in the study design leading to a serious risk of bias.
gOnly 38% (5/13) of the studies favored immersive virtual reality teaching (P=.58 in a binomial probability test showing that the null hypothesis is accepted).
hAccording to the Joanna Brigs Institute critical appraisal checklist for randomized controlled trials, these 3 randomized controlled trials have 3 to 5 items rated at no” or unclear, which indicates issues in the study design.
iA total of 50% (1/2) of the studies favored immersive virtual reality teaching. One study used IVR to orient the students to the operating room.
jA total of 17% (2/12) of the studies were quasi-experimental trials. According to the Joanna Brigs Institute critical appraisal checklist for randomized controlled trials, the remaining 10 randomized controlled trials had 1 to 5 items rated at no or unclear, which indicates issues in the study design leading to a serious risk of bias.
kA total of 62% (8/13) of the studies favored immersive virtual reality teaching (P=.39 in a binomial probability test showing that the null hypothesis is accepted).
The findings of this systematic review demonstrate that the use of IVR teaching in undergraduate health care education is effective in enhancing the procedural skills and knowledge acquisition of students. However, the effects on these learning outcomes were similar to those of other teaching approaches such as desktop-based VR or conventional classroom teaching. A vote count showed that 60% (9/15; 95% CI 16.3%-67.7%; P=.61) of the studies identified similar learning outcomes between IVR teaching and other teaching approaches regardless of the teaching domain.
In general, in the 15 studies, the students indicated that they had positive learning experiences with IVR teaching, including increased satisfaction, self-confidence, self-assessed competency, self-efficacy, and enjoyment with IVR teaching. When compared with those in the control group who received other methods of teaching, the vote count showed that, in 62% (8/13) of the studies, the participants were in favor of using IVR as a teaching medium. However, the results of the binomial test (95% CI 34.9%-90%; P=.59) did not indicate a statistically significant difference. Therefore, the results indicated similarly positive learning experiences between students who received IVR and those who received other teaching approaches. Considering the low-level evidence identified based on the GRADE tool, it is inconclusive whether IVR teaching is superior to other forms of VR and to conventional teaching methods with regard to students’ learning outcomes and learning experiences.
Comparison With Prior Work
Although the application of IVR in health care teaching has become more prevalent in recent years, only a limited number of systematic reviews have focused solely on evaluating its effects. Many reviews have included all forms of VR teaching (such as 2D computer games). Therefore, the effects of IVR on teaching are yet to be confirmed. This review is one of the few to provide additional evidence on the effects of IVR in health care teaching.
The inconclusive findings on the students’ learning outcomes identified in this review are similar to those of the systematic review of 29 articles by Hamilton et al . Their review also reported that the effects of IVR teaching on learning outcomes and attainment levels were inconsistent compared with those of conventional desktop-based VR or the original physical training scenario [ ]. These findings contradict those of the systematic review of 17 articles by Mao et al [ ]. Their review reported that medical students who received IVR training performed significantly faster in the time required to complete surgical procedures and had higher scores on procedural checklists than those who received other forms of training [ ]. A possible reason for the inconsistent findings may be the different target populations and objectives of the studies. Both our review and the review by Hamilton et al [ ] included studies targeting the teaching of theoretical knowledge and training in procedural skills, but the review by Mao et al [ ] focused mainly on training in surgical procedural skills. It is possible that IVR may be more effective for teaching procedural skills as it provides real-world simulations to give the students an immersive training experience and allows students to repeatedly practice the same procedures. However, when used in theoretical teaching, the instillation of knowledge appears to rely more on personal memorization and understanding [ ].
The lack of teaching theories and pedagogies to guide the integration of IVR into health care education may be another reason for the inconsistent teaching outcomes shown in this review. This finding is also similar to that of the review by Hamilton et al , which identified only 1 study with a pedagogical framework to guide the development of IVR teaching [ ]. Among all the included studies in this review, only the study by Smith et al [ ] adopted the National League for Nurses Jeffries Simulation Theory to guide the design of simulation teaching, which is that students learn information as part of a simulated experience [ ]. In general, the included studies indicated that their students’ exposure to IVR learning experiences was short. A total of 75% (12/16) of the studies arranged a single IVR learning experience ranging from 5 to 30 minutes for their students. To maximize the impacts of IVR teaching, it has been suggested that educators incorporate IVR teaching into their courses guided by a pedagogy to enrich the teaching contexts with actual experience, insightful reflections, realistic practices, and real-world connections [ ].
Although the learning outcomes from IVR teaching are comparable with those from other teaching approaches, IVR still has many advantages in undergraduate health care education. In particular, it can improve the students’ positive learning experiences. In total, 62% (10/16) of the studies in this review found that students favored learning with IVR, which can motivate them to learn actively rather than passively, although the results of the binomial test did not indicate a statistically significant difference. Similarly, the systematic review by Mao et al  reported that IVR could improve the self-confidence of medical trainees in performing surgical procedures compared with other training methods. In addition, IVR teaching can incorporate different scenarios of patients in critically ill and emergency situations, giving students valuable hands-on opportunities in a safe and controlled environment. In this review, 31% (5/16) of the studies [ - , ] simulated real-world scenarios to train students in clinical skills such as tube feeding and conducting physical assessments of infants with respiratory distress. This provided the students with a safe environment in which to avoid unnecessary adverse events, which health care students often experience.
Although there is evidence showing that IVR can be used to enhance the professional attitudes of students, such as in the areas of empathy, decision-making, or collaborative teamwork, none of the studies in this review focused on this domain of teaching. Most (13/16, 81%) used IVR to train students in procedural skills, 12% (2/16) used it to teach anatomy, and 6% (1/16) used it for orientation. A review of 178 medical studies showed that using IVR in teaching could effectively improve medical students’ understanding of the impacts of gerontological diseases on the daily life of older adults. Moreover, after learning through IVR, the students expressed more feelings of empathy toward older adults with sensory impairments and dementia . Another study showed that IVR teaching can create scenarios that closely resemble those in real health care settings, allowing students to immerse themselves in practicing clinical reasoning and learn how to deal with emergencies, which can boost their ability to make decisions and determine priorities [ ]. Another study tested the awareness and decision-making abilities of novice surgical residents by using IVR in surgical training. The results showed that the residents’ scores improved significantly in comparison with the scores of those who received conventional PowerPoint teaching [ ]. In addition, IVR teaching can help students build their communication skills. A study found that IVR teaching could significantly promote the communication skills of pediatricians to persuade parents of the merits of the influenza vaccine injection [ ]. Good communication skills are a core attribute of clinical competence, and IVR teaching can allow students to practice this skill in VR clinical settings and enable them to learn how to deal with different situations [ ].
This review has implications for both research and educational practice. First, the unique capabilities of IVR teaching are yet to be fully exploited. In addition to training in procedural skills and inputting theoretical knowledge, IVR teaching has great potential to be used for other teaching domains such as health care professional attitudes. Second, health educators need to align pedagogy with IVR teaching for successful integration into undergraduate health care education . Using pedagogy to guide the integration of IVR teaching into a course can maximize the impact on the students’ learning outcomes by enriching teaching contexts with robust instructions, realistic practices, and real-world connections [ ]. Third, it is extremely important in undergraduate health care training for skills and knowledge to be retained and applied in real clinical settings. Therefore, in addition to immediate skill tests or written examinations, other forms of assessment such as essays or group discussions can be considered for an in-depth evaluation of the students’ understanding of the course [ ]. It may be difficult to measure the differences between IVR and traditional teaching in terms of the transferability from knowledge to actual clinical practice. It is hard to tell whether students have transferred what they have learned from IVR into their clinical practice. Current evaluation methods focus on superficial and short-term effects. Clinically based observational studies conducted over a long period can be considered in the future. Finally, the quality of the current studies, especially the RCTs, is a concern. Standardized guidelines such as CONSORT (Consolidated Standards of Reporting Trials) [ ] should be referred to when designing a study.
Strengths and Limitations
Our review provides the most up-to-date evidence on the positive effects of IVR in undergraduate health professional education. We conducted a comprehensive search across different databases and followed the Cochrane gold-standard methodology together with a meta-synthesis while conducting this systematic review. This systematic review has some limitations. First, only articles published in English were included. It is possible that articles in other languages were overlooked. However, we adopted relatively broad eligibility criteria for the study types (RCTs and quasi-experimental trials), which led to sufficient findings. Second, the heterogeneity of the included studies prevented the data from being pooled for a meta-analysis, meaning that the effect size of the IVR on teaching outcomes could not be determined. Third, we did not analyze the learning experiences of students from data collected using qualitative methods. Although a qualitative method was only mentioned in 6% (1/16) of the included studies, it is possible that some findings related to the subjective learning experiences of students with IVR were missed. Fourth, we originally hypothesized that the types of equipment with different levels of immersive experiences used for IVR teaching would affect students’ learning outcomes and experiences. However, we found no studies that measured the level of immersive experience according to the type of VR equipment. Therefore, we could not perform any subgroup analyses to investigate these possible moderating effects on students’ learning outcomes and experiences. However, in our review, we found no evidence to support this hypothesis. The use of equipment played a small role in influencing teaching and learning outcomes. Finally, this review only focused on undergraduate health care students; therefore, the findings may not be transferable to students in other majors.
Despite the positive effects of IVR when adopted in undergraduate health care education in the digital era, robust assessments through high-quality, large-scale studies with long follow-up periods are still lacking. In addition, the true efficacy of IVR is best assessed through long-term integration into a real-world training guide with a pedagogical framework to maximize the effects of IVR on education. In addition to procedural training and theoretical knowledge, IVR has the potential to be used to train students in other desirable attributes of health care professionals (such as communication skills, decision-making skills, and feelings of empathy toward patients). Given its positive impacts on students’ learning outcomes and experiences, we recommend further investigation with rigorous studies focusing on important outcomes for students following long-term incorporation into different health care curricula in different teaching domains.
This systematic review demonstrates that the use of IVR teaching in undergraduate health care education is effective in enhancing the procedural skills and knowledge acquisition of students, although the effects on these learning outcomes were similar to those of other teaching approaches. IVR also has an advantage in enhancing the positive learning experiences of students. In total, 50% (8/16) of the studies in this review indicated that the students favored IVR teaching over other teaching methods. IVR teaching also has great potential to be used in other teaching domains such as enhancing the professional attitudes of students. Unfortunately, none of the included studies focused on those teaching domains, which are worth exploring in future studies. Finally, health educators should align pedagogy with IVR teaching for successful integration into undergraduate health care education to maximize the impact on students’ learning outcomes.
The research team would like to express their gratitude to the librarians at The Hong Kong Polytechnic University for recommending search strategies and to research associate Amy Cheung for formatting the manuscript.
JYWL conceived the idea for this review. JYWL and YHY searched for, reviewed, and analyzed the included studies as well as drafting the manuscript. PPKK and DSKC provided advice on methodology. Other authors critically commented on the selection of papers, the findings, and the manuscript.
Conflicts of Interest
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.DOCX File , 28 KB
Search strategy for different databases.DOCX File , 19 KB
- Rourke S. How does virtual reality simulation compare to simulated practice in the acquisition of clinical psychomotor skills for pre-registration student nurses? A systematic review. Int J Nurs Stud 2020 Feb;102:103466. [CrossRef] [Medline]
- Fealy S, Jones D, Hutton A, Graham K, McNeill L, Sweet L, et al. The integration of immersive virtual reality in tertiary nursing and midwifery education: a scoping review. Nurse Educ Today 2019 Aug;79:14-19. [CrossRef] [Medline]
- Chinelatto LA, Costa TR, Medeiros VM, Boog GH, Hojaij FC, Tempski PZ, et al. What you gain and what you lose in COVID-19: perception of medical students on their education. Clinics (Sao Paulo) 2020;75:e2133 [FREE Full text] [CrossRef] [Medline]
- Cook DA, Brydges R, Hamstra SJ, Zendejas B, Szostek JH, Wang AT, et al. Comparative effectiveness of technology-enhanced simulation versus other instructional methods: a systematic review and meta-analysis. Simul Healthc 2012 Oct;7(5):308-320. [CrossRef] [Medline]
- Hope A, Garside J, Prescott S. Rethinking theory and practice: pre-registration student nurses experiences of simulation teaching and learning in the acquisition of clinical skills in preparation for practice. Nurse Educ Today 2011 Oct;31(7):711-715. [CrossRef] [Medline]
- Al-Ghareeb AZ, Cooper SJ. Barriers and enablers to the use of high-fidelity patient simulation manikins in nurse education: an integrative review. Nurse Educ Today 2016 Jan;36:281-286. [CrossRef] [Medline]
- Chernikova O, Heitzmann N, Stadler M, Holzberger D, Seidel T, Fischer F. Simulation-based learning in higher education: a meta-analysis. Rev Educ Res 2020 Jun 15;90(4):499-541. [CrossRef]
- Sawyer T, White M, Zaveri P, Chang T, Ades A, French H, et al. Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine. Acad Med 2015 Aug;90(8):1025-1033. [CrossRef] [Medline]
- Cook DA. How much evidence does it take? A cumulative meta-analysis of outcomes of simulation-based education. Med Educ 2014 Aug;48(8):750-760. [CrossRef] [Medline]
- Hegland PA, Aarlie H, Strømme H, Jamtvedt G. Simulation-based training for nurses: systematic review and meta-analysis. Nurse Educ Today 2017 Jul;54:6-20. [CrossRef] [Medline]
- Mao RQ, Lan L, Kay J, Lohre R, Ayeni OR, Goel DP, et al. Immersive virtual reality for surgical training: a systematic review. J Surg Res 2021 Dec;268:40-58. [CrossRef] [Medline]
- Simpson R, LaViola J, Laidlaw D, Forsberg A, van Dam A. Immersive VR for scientific visualization: a progress report. IEEE Comput Grap Appl 2000;20(6):26-52. [CrossRef]
- Brooks F. What's real about virtual reality? IEEE Comput Grap Appl 1999;19(6):16-27. [CrossRef]
- Freina L, Ott M. A literature review on immersive virtual reality in education: state of the art and perspectives. In: Proceedings of the eLearning and Software for Education (eLSE). 2015 Presented at: eLearning and Software for Education (eLSE); Apr 23-24, 2015; Bucharest, Romania.
- Jeffries PR, Bushardt RL, DuBose-Morris R, Hood C, Kardong-Edgren S, Pintz C, et al. The role of technology in health professions education during the COVID-19 pandemic. Acad Med 2022 Mar 01;97(3S):S104-S109 [FREE Full text] [CrossRef] [Medline]
- McKechnie T, Levin M, Zhou K, Freedman B, Palter VN, Grantcharov TP. Virtual surgical training during COVID-19: operating room simulation platforms accessible from home. Ann Surg 2020 Aug;272(2):e153-e154 [FREE Full text] [CrossRef] [Medline]
- Bracq M, Michinov E, Jannin P. Virtual reality simulation in nontechnical skills training for healthcare professionals: a systematic review. Simul Healthc 2019 Jun;14(3):188-194 [FREE Full text] [CrossRef] [Medline]
- Rooney MK, Zhu F, Gillespie EF, Gunther JR, McKillip RP, Lineberry M, et al. Simulation as more than a treatment-planning tool: a systematic review of the literature on radiation oncology simulation-based medical education. Int J Radiat Oncol Biol Phys 2018 Oct 01;102(2):257-283 [FREE Full text] [CrossRef] [Medline]
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
- Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ 2020 Jan 16;368:l6890 [FREE Full text] [CrossRef] [Medline]
- Shrestha N, Kukkonen-Harjula KT, Verbeek JH, Ijaz S, Hermans V, Pedisic Z. Workplace interventions for reducing sitting at work. Cochrane Database Syst Rev 2018 Jun 20;6(6):CD010912 [FREE Full text] [CrossRef] [Medline]
- Higgins JL, Deeks J. Chapter 6 Choosing effect measures and computing estimates of effect. In: Cochrane Handbook for Systematic Reviews of Interventions. Hoboken, New Jersey: Wiley; Aug 24, 2011.
- McKenzie J, Brennan S. Chapter 12 Synthesizing and presenting findings using other methods. In: Cochrane Handbook for Systematic Reviews of Interventions, Second Edition. Hoboken, New Jersey: Wiley; 2019.
- Schunemann HH, Higgins J, Vist G, Glasziou P, Skoetz N, Akl E, et al. Chapter 14 Completing ‘Summary of findings’ tables and grading the certainty of the evidence. In: Cochrane Handbook for Systematic Reviews of Interventions, Second Edition. Hoboken, New Jersey: Wiley; 2019.
- Gutiérrez-Maldonado J, Ferrer-García M, Plasanjuanelo J, Andrés-Pueyo A, Talarn-Caparrós A. Virtual reality to train diagnostic skills in eating disorders. Comparison of two low cost systems. Stud Health Technol Inform 2015;219:75-81. [Medline]
- Harrington CM, Kavanagh DO, Wright Ballester G, Wright Ballester A, Dicker P, Traynor O, et al. 360° operative videos: a randomised cross-over study evaluating attentiveness and information retention. J Surg Educ 2018;75(4):993-1000. [CrossRef] [Medline]
- Smith SJ, Farra SL, Ulrich DL, Hodgson E, Nicely S, Mickle A. Effectiveness of two varying levels of virtual reality simulation. Nurs Educ Perspect 2018;39(6):E10-E15. [CrossRef] [Medline]
- Zackoff MW, Real FJ, Sahay RD, Fei L, Guiot A, Lehmann C, et al. Impact of an immersive virtual reality curriculum on medical students' clinical assessment of infants with respiratory distress. Pediatr Crit Care Med 2020 May;21(5):477-485. [CrossRef] [Medline]
- Francis ER, Bernard S, Nowak ML, Daniel S, Bernard JA. Operating room virtual reality immersion improves self-efficacy amongst preclinical physician assistant students. J Surg Educ 2020;77(4):947-952. [CrossRef] [Medline]
- Kurul R, Ögün MN, Neriman Narin A, Avci, Yazgan B. An alternative method for anatomy training: immersive virtual reality. Anat Sci Educ 2020 Sep;13(5):648-656. [CrossRef] [Medline]
- Pierce J, Gutiérrez F, Vergara VM, Alverson DC, Qualls C, Saland L, et al. Comparative usability studies of full vs. partial immersive virtual reality simulation for medical education and training. Stud Health Technol Inform 2008;132:372-377. [Medline]
- Gutiérrez F, Pierce J, Vergara VM, Coulter R, Saland L, Caudell TP, et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform 2007;125:155-160. [Medline]
- Sapkaroski D, Baird M, Mundy M, Dimmock MR. Quantification of student radiographic patient positioning using an immersive virtual reality simulation. Simul Healthc 2019 Aug;14(4):258-263. [CrossRef] [Medline]
- Stepan K, Zeiger J, Hanchuk S, Del Signore A, Shrivastava R, Govindaraj S, et al. Immersive virtual reality as a teaching tool for neuroanatomy. Int Forum Allergy Rhinol 2017 Oct;7(10):1006-1013. [CrossRef] [Medline]
- Bakhos D, Galvin J, Aoustin J, Robier M, Kerneis S, Bechet G, et al. Training outcomes for audiology students using virtual reality or traditional training methods. PLoS One 2020;15(12):e0243380 [FREE Full text] [CrossRef] [Medline]
- Chao Y, Hu SH, Chiu H, Huang P, Tsai H, Chuang Y. The effects of an immersive 3d interactive video program on improving student nurses' nursing skill competence: a randomized controlled trial study. Nurse Educ Today 2021 Aug;103:104979. [CrossRef] [Medline]
- Berg H, Steinsbekk A. Is individual practice in an immersive and interactive virtual reality application non-inferior to practicing with traditional equipment in learning systematic clinical observation? A randomized controlled trial. BMC Med Educ 2020 Apr 22;20(1):123 [FREE Full text] [CrossRef] [Medline]
- Berg H, Steinsbekk A. The effect of self-practicing systematic clinical observations in a multiplayer, immersive, interactive virtual reality application versus physical equipment: a randomized controlled trial. Adv Health Sci Educ Theory Pract 2021 May;26(2):667-682 [FREE Full text] [CrossRef] [Medline]
- Collaço E, Kira E, Sallaberry LH, Queiroz AC, Machado MA, Crivello O, et al. Immersion and haptic feedback impacts on dental anesthesia technical skills virtual reality training. J Dent Educ 2021 Apr;85(4):589-598. [CrossRef] [Medline]
- Ros M, Debien B, Cyteval C, Molinari N, Gatto F, Lonjon N. Applying an immersive tutorial in virtual reality to learning a new technique. Neurochirurgie 2020 Aug;66(4):212-218. [CrossRef] [Medline]
- Yu M, Yang M, Ku B, Mann JS. Effects of virtual reality simulation program regarding high-risk neonatal infection control on nursing students. Asian Nurs Res (Korean Soc Nurs Sci) 2021 Aug;15(3):189-196 [FREE Full text] [CrossRef] [Medline]
- Harrington CM, Kavanagh DO, Quinlan JF, Ryan D, Dicker P, O'Keeffe D, et al. Development and evaluation of a trauma decision-making simulator in Oculus virtual reality. Am J Surg 2018 Jan;215(1):42-47. [CrossRef] [Medline]
- Adhikari R, Kydonaki C, Lawrie J, O'Reilly M, Ballantyne B, Whitehorn J, et al. A mixed-methods feasibility study to assess the acceptability and applicability of immersive virtual reality sepsis game as an adjunct to nursing education. Nurse Educ Today 2021 Aug;103:104944. [CrossRef] [Medline]
- Pritchard A. Ways of Learning Learning Theories for the Classroom. Milton Park, Abingdon-on-Thames, Oxfordshire, England, UK: Routledge; 2017.
- Schunk D. Learning Theories An Educational Perspective. London, United Kingdom: Pearson; 2014.
- Jeffries PR, Rodgers B, Adamson K. NLN Jeffries simulation theory: brief narrative description. Nurs Educ Perspect 2015;36(5):292-293. [CrossRef] [Medline]
- Hamilton D, McKechnie J, Edgerton E, Wilson C. Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. J Comput Educ 2020 Jul 11;8(1):1-32. [CrossRef]
- Polanyi M. Personal Knowledge Towards a Post-critical Philosophy. Chicago: University of Chicago Press; 1974.
- Bonasio A. Immersive experiences in education new places and spaces for learning. Microsoft. URL: https://edudownloads.azureedge.net/msdownloads/MicrosoftEducation_Immersive_Experiences_Education_2019.pdf [accessed 2022-11-08]
- Dyer E, Swartzlander BJ, Gugliucci MR. Using virtual reality in medical education to teach empathy. J Med Libr Assoc 2018 Oct;106(4):498-500 [FREE Full text] [CrossRef] [Medline]
- Pulijala Y, Ma M, Pears M, Peebles D, Ayoub A. Effectiveness of Immersive Virtual Reality in Surgical Training-A Randomized Control Trial. J Oral Maxillofac Surg 2018 May;76(5):1065-1072. [CrossRef] [Medline]
- Real FJ, DeBlasio D, Beck AF, Ollberding NJ, Davis D, Cruse B, et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal. Acad Pediatr 2017;17(4):431-435. [CrossRef] [Medline]
- Fertleman C, Aubugeau-Williams P, Sher C, Lim A, Lumley S, Delacroix S, et al. A discussion of virtual reality as a new tool for training healthcare professionals. Front Public Health 2018;6:44 [FREE Full text] [CrossRef] [Medline]
- Domingo JR, Bradley EG. Education student perceptions of virtual reality as a learning tool. J Educational Technol Sys 2017 Oct 24;46(3):329-342. [CrossRef]
- Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010 Mar 23;340:c332 [FREE Full text] [CrossRef] [Medline]
|CONSORT: Consolidated Standards of Reporting Trials|
|GRADE: Grading of Recommendations Assessment, Development, and Evaluation|
|IVR: immersive virtual reality|
|JBI: Joanna Briggs Institute|
|PICOS: Population, Intervention, Comparison, Outcome, and Study Design|
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|RCT: randomized controlled trial|
|SWiM: Synthesis Without Meta-analysis|
Edited by T Leung; submitted 30.05.22; peer-reviewed by PH Liao, Y Xu, M Mbwogge; comments to author 06.09.22; revised version received 09.11.22; accepted 24.01.23; published 06.03.23Copyright
©Justina Yat Wa Liu, Yue-Heng Yin, Patrick Pui Kin Kor, Daphne Sze Ki Cheung, Ivy Yan Zhao, Shanshan Wang, Jing Jing Su, Martin Christensen, Stefanos Tyrovolas, Angela Y M Leung. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.03.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.