Review
Abstract
Background: In recent years, the use of digital technology in the education of health care professionals has surged, partly driven by the COVID-19 pandemic. However, there is still a need for focused research to establish evidence of its effectiveness.
Objective: This study aimed to define the gaps in the evidence for the efficacy of digital education and to identify priority areas where future research has the potential to contribute to our understanding and use of digital education.
Methods: We used a 2-stage approach to identify research priorities. First, an umbrella review of the recent literature (published between 2020 and 2023) was performed to identify and build on existing work. Second, expert consensus on the priority research questions was obtained using a modified Delphi method.
Results: A total of 8857 potentially relevant papers were identified. Using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology, we included 217 papers for full review. All papers were either systematic reviews or meta-analyses. A total of 151 research recommendations were extracted from the 217 papers. These were analyzed, recategorized, and consolidated to create a final list of 63 questions. From these, a modified Delphi process with 42 experts was used to produce the top-five rated research priorities: (1) How do we measure the learning transfer from digital education into the clinical setting? (2) How can we optimize the use of artificial intelligence, machine learning, and deep learning to facilitate education and training? (3) What are the methodological requirements for high-quality rigorous studies assessing the outcomes of digital health education? (4) How does the design of digital education interventions (eg, format and modality) in health professionals’ education and training curriculum affect learning outcomes? and (5) How should learning outcomes in the field of health professions’ digital education be defined and standardized?
Conclusions: This review provides a prioritized list of research gaps in digital education in health care, which will be of use to researchers, educators, education providers, and funding agencies. Additional proposals are discussed regarding the next steps needed to advance this agenda, aiming to promote meaningful and practical research on the use of digital technologies and drive excellence in health care education.
doi:10.2196/66157
Keywords
Introduction
In just over two decades, the use of digital technology for delivering education and learning has increased globally []. The growth of digital education in health care has seen a similar trajectory internationally [].
The adoption of digital technologies in health care education forms part of a global strategy to invest in the health workforce [] and support the United Nations 2023 Agenda for Sustainable Development []. This agenda is echoed within the United Kingdom’s National Health Service (NHS) Long Term Workforce Plan []. In its plan to address projected workforce shortages, the NHS has undertaken to expand health care education and training over the next 15 years. Informed by the Topol Review [], the Long Term Workforce Plan acknowledges the need to embrace a wide range of learning methods to achieve the scale of education delivery that will be required []. These methods include the use of web-based learning, blended learning, hybrid learning, and simulation and immersive technologies. In addition, it is clear that easily accessible and usable artificial intelligence (AI) will have an increasing role in education [].
The COVID-19 pandemic saw a surge in the use of digital technologies for health care education. Consequently, several bibliometric analyses illustrate how research activity across a variety of digital technologies significantly increased internationally [,-]. Since 2020, research activity has shifted from rapid development during the early stages of the COVID-19 pandemic to a focus on quality improvement and opportunities for sustained use in the postpandemic period []. Zhang et al [] state that the data project a trend of continued growth in digital education in health care globally, including increased application of advanced technologies. In the United Kingdom, the Council of Deans of Health highlighted in their report “Post-Pandemic Progress: Lessons Learnt in Healthcare Education” [] that attitudes of educators and students toward the use of digital modalities have shifted. Universities UK [] echoes this sentiment, recommending increased investment and further expansion of digital technologies in health professions education.
To successfully leverage the full potential of existing technologies for education delivery and understand the opportunities presented by advanced technologies, there is a requirement to ensure that learning needs are being effectively met [] and that there is proven return on the investment needed to deploy these technologies. The Immersive Healthcare Collaboration [] emphasized the importance of rigorous evaluation when adopting new technologies and highlighted the need for continued investment in digital education research. However, currently it is not clear where these research efforts and resources should be focused.
Recent studies on research priorities in health care digital education have mostly focused on surgery [-]. However, a recent literature review by Tudor Car et al [] explored evidence across all health care disciplines and across multiple digital education modalities. The authors sought to broaden the scope of their inquiry beyond evidence of effectiveness and to identify where gaps in the evidence lay around other factors such as context, infrastructure, education, and learners []. In doing so, they constructed an evidence map, a contextual framework, and a potential research agenda. They offered a list of research questions that, at that time, were still to be fully addressed by the scientific literature []. While this work is both comprehensive and informative, it was conducted before or early in the COVID-19 pandemic era; all the reviewed papers were published between January 2014 and July 2020. With the massive increase in the use of digital education in response to the COVID-19 pandemic, the authors of this study felt that this work warranted revisiting and updating to uncover additional evidence gaps and new areas of inquiry that may have emerged since 2020.
Therefore, this study aimed to build on work by Tudor Car et al [] and identify, within the conceptual framework, any new research priorities in health care digital education that have emerged since 2020. Identifying these research priorities would help shape future global research in digital education and guide the determination of priorities for activity and resource allocation within NHS England.
Methods
Overview
To identify research priorities in health care digital education, a 2-stage approach was used. First, an umbrella review was conducted to build on previous work, and second, experts were invited to reach consensus on the relevant research questions using a modified Delphi method. This 2-stage approach was used to ensure a balance between the published literature and test its applicability through expert opinion. Details of each stage are described in the following sections.
Umbrella Review
As a first step to identifying any additional research questions since the study by Tudor Car et al [], an umbrella review (a review of systematic reviews) of the literature was conducted based on the method used by Tudor Car et al []. The systematic review and research syntheses checklist has been used to self-appraise the process on completion () [].
Inclusion and Exclusion Criteria
Studies eligible for inclusion were peer-reviewed systematic reviews or meta-analyses with a clear focus on digital education for health professions education, which were published between 2020 and 2023 and written in English. The list of digital education technologies as described by Tudor Car et al [] was used to guide eligibility screening. For convenience, definitions have been shared in (
Textbox 1).
Included (studies involving one or more of the following digital education technologies)
- Online digital education
- Massive open web-based course
- Mobile education
- Serious gaming and gamification
- Virtual reality
- Virtual patients
- High-fidelity manikins
- Blended education
- Augmented reality
- Extended reality
Excluded
- 3D printing (after considerable discussion, it was decided that 3D printing should not be considered a digital educational intervention because this intervention leads to an analogue product and is used as such)
- Primary research
- Conference abstracts
- Non-English language papers
To further assess eligibility for inclusion, all the following questions had to be answered in the affirmative:
- Are the participants in the intervention preservice (ie, students) or in-service professionals (ie, after degree completion, including postgraduate trainees and those in independent practice)?
- Are the participants from disciplines such as medicine, dentistry, nursing and midwifery, medical diagnostic and treatment technology, physiotherapy and rehabilitation, and pharmacy? (practitioners of traditional, alternative, and complementary medicine were excluded)
- Is the educational intervention using digital technology?
Information Sources
A knowledge specialist (EW) searched the following major bibliographic databases for review papers meeting the inclusion criteria and published between July 2020 and April 2023: MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCO), and the Cochrane Library. In addition, the knowledge specialist (EW) searched ERIC, Google Scholar, and Research Gate using the following search terms: systematic review, digital education, eLearning, health professionals, and health professions education. All searches were conducted on April 21, 2023.
Search Strategy
The MEDLINE (Ovid) search strategy was identical to that used by Tudor Car et al [] (). Minor modifications were needed to convert the strategy for CINAHL (EBSCO) and the Cochrane Library.
Study Selection
The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology was used to identify studies for inclusion. The search results were captured in RefWorks [], and duplicates were removed. Pairs of independent reviewers screened the title and abstract of each paper against the eligibility criteria and predefined question set for inclusion. The full-text papers meeting the inclusion criteria were then retrieved and reviewed by pairs of independent reviewers. At both stages of screening, any discrepancy between reviewer pairs was resolved by a third independent reviewer.
Data Extraction
A data capture tool was developed in Jisc Online Surveys [] and piloted by 8 reviewers (AP, CM, EH, SK, ION, EN and 2 volunteers from the NHS England technology enhanced learning team) using a sample paper, with refinements made to the tool before data extraction from all papers included in the review.
Data extracted were study characteristics and future research recommendations as set out by the authors. Future research recommendations were mapped to the conceptual domain framework devised by Tudor Car et al [] and any previously identified research questions, where possible. If there was insufficient alignment with existing research questions, a new one was created. All papers with future research recommendations were reviewed a second time, with any discrepancies resolved by a third independent reviewer.
Analysis
AP and EW consolidated the extracted research questions by mapping them to existing questions, removing duplicates, and rephrasing those with similar meanings. New questions were then grouped and categorized by CM into overarching research questions. CM and AP discussed and reached a consensus on the final set of questions.
Delphi Consensus
Overview
A 4-round modified Delphi was conducted through January to March 2024 to arrive at 5 final research questions. The method consisted of a desktop exercise to formulate the Delphi survey from the results of the umbrella review (round 1), assessment (round 2), prioritization (round 3), and ranking (round 4). Each survey was created by AP and tested by members of the research team.
Round 1: Umbrella Review and Preparation of the Survey
The results of the umbrella review provided the material to compile a web-based survey of research questions for consideration by experts in later rounds. Research questions, previously identified in the umbrella review, meeting the inclusion criteria formed the basis of a web-based survey in round 2.
Inclusion criteria were as follows:
- The median number of papers identified and mapped to an existing research question in the study by Tudor Car et al [] was 7.5, and this was used as an inclusion threshold.
- Following consolidation, all newly identified questions were included in the survey.
Round 2: Assessment Against Criteria
A total of 42 participants were recruited, including digital education professionals and simulation faculty, clinical educators, health and care workforce development and transformation leads, academics, and health care students, using the set criteria (Textbox 2). Recruitment was advertised via the Association for Learning Technologies’ weekly news digest and a nationwide digital education mailing list for health professionals maintained by NHS England ().
A web-based survey, created in JISC Online Surveys and comprising all research questions from round 1, was distributed to the expert panel.
Using an adapted set of criteria from the Delphi process used by Schneider et al [], participants were invited to anonymously score each research question against 5 criteria (
Table 1).
Research questions that met the consensus threshold formed the basis for round 3.
The consensus thresholds for Round 2 were as follows:
- Inclusion: >75% of respondents provide a positive result (3) on a 3-point Likert scale for all criteria.
- Exclusion: >75% of respondents provide a negative result (2) on a 3-point Likert scale for all criteria.
- Nonconsensus: when the proposed priority research question has met neither the inclusion nor the exclusion consensus thresholds.
Questions that met the inclusion criteria and those with nonconsensus were carried forward into round 3.
Stakeholder group and inclusion criteria for expertise
- Technology enhanced learning (TEL) professional and simulation faculty: substantial experience working in TEL or simulation in the National Health Service or academia
- Educator: anyone in clinical practice who trains health care professionals (such as consultant or nurse educator)
- Workforce development and transformation lead: experience or knowledge of TEL in health and care education within their geography or domain
- Academic: working in academia, with at least 1 first or senior author paper in the field of TEL
- Learner: health care student, either final-year undergraduate or postgraduate student or trainee
| Criteria | Scoring options | ||
| Scientific merit | 1=unable to respond | 2=disagree | 3=agree |
| Significance for workforce development | 1=unable to respond | 2=disagree | 3=agree |
| Innovation | 1=unable to respond | 2=disagree | 3=agree |
| Relevance to the LTWPa | 1=unable to respond | 2=disagree | 3=agree |
| Feasibility for further study | 1=unable to respond | 2=disagree | 3=agree |
aLTWP: Long Term Workforce Plan.
Round 3: Prioritization
Participants’ voting in this round was informed by knowledge of the consensus outcomes for each research question from round 2. A web-based survey was shared with participants, consisting of 2 sections: one with research questions that met the inclusion criteria for round 2 (including the percentage consensus agreement for each question) and another with questions from round 2 that did not reach consensus.
Participants were invited to score each research question again, but this time based on priority, with 1 score assigned to each question. The scoring options were as follows: 1=not to be studied, 2=low priority, 3=medium priority, and 4=high priority.
The consensus thresholds for Round 3 were as follows:
- Inclusion: ≥80% of respondents scored the research question as a 3 or 4
- Exclusion: 100% of respondents scored the research question as a 1 (not to be studied)
- Nonconsensus: when the proposed priority research question met neither the inclusion nor the exclusion consensus thresholds
Those meeting the inclusion criteria were taken forward for ranking in round 4.
Round 4: Ranking
A web-based meeting was conducted with participants, facilitated by CM and AP. Participants were sent results of the previous round in advance (). The meeting offered the opportunity to discuss results from round 3 in small groups. Finally, participants were asked to select and rank their top 5 research questions independently via a web-based survey constructed in Slido [], a web-based audience interaction tool that offers in-meeting and between-meeting polling and surveying. Participants unable to attend the meeting were invited to complete this exercise within 1 week of the meeting date. After this time, for each ranked question, points were assigned in a descending order, where higher ranks received more points, for example, a question ranked first place received 5 points, and each subsequent rank received 1 point less, down to 1 point for a question ranked fifth. The 5 research questions with the highest mean formed the final set of research priorities.
Finally, a narrative synthesis approach was chosen to summarize the diverse range of selected studies in a structured manner.
Ethical Considerations
The NHS Research Ethics screening tool was completed, and approval was not required for this study. However, key ethical requirements were considered when designing and conducting the Delphi. The NHS England Data Protection Impact Assessment was completed. Delphi participant recruitment was voluntary, with opt out being possible. Compensation was not provided. A participant information sheet outlining the purpose of the study, key contacts, their potential involvement with anticipated time commitment, and how data would be handled and stored and by whom was shared. Personal data were stored in a spreadsheet hosted on an organizational SharePoint and were only accessible to relevant members of the research team. Participants were not asked for any personal details in the Delphi surveys; instead, they were assigned a code number. This code number allowed the research team to keep track of response rates and to administrate any follow up.
Results
Umbrella Review
Study Selection
A total of 8857 potentially relevant papers were identified across all information sources. Following removal of duplicates, of the 8857 papers, 6044 (68.24%) were screened by title and abstract. Of these 6044 papers, 441 (7.3%) were assessed for eligibility via full-text review, leading to 217 (3.6%) papers being included in this review (
Figure 1). Refer to for a complete list of key study characteristics for all included studies.
Study Characteristics
All included papers were systematic reviews or meta-analyses. Of the included studies across all the systematic reviews, high-income settings were predominant, with 23% (50/217) exclusively consisting of high-income countries and 33.6% (73/217) including a mix of high- and middle-income countries. In total, 14.3% (31/217) covered high-, middle-, and low-income countries, with only 0.9% (2/217) identifying middle- and low-income countries. The remaining 28.1% (61/217) of systematic reviews did not report the economic setting of their included studies.
In total, 29% (63/217) of the systematic reviews were focused on preservice alone. Of these, 11% (24/217) were nursing students, 5.1% (11/217) were medical students, and 7.8% (17/217) were mixed student populations. The least common population groups to be studied in the included reviews were pharmacy students (4/217, 1.8%), dentistry students (4/217, 1.8%), physiotherapy students (2/217, 1.8%), and health sciences students (1/217, 0.5%). This reflects the number of publications from these disciplines.
Reviews exploring the use of interventions with in-service professionals (44/217, 20.3%) are divided into fewer, less distinct categories: 13.8% (30/217) physicians, 6.5% (14/217) mixed health care professionals, and 0.5% (1/217) health care workers.
Of the systematic reviews comprising studies for both pre- and in-service professionals (110/217, 50.7%), 22.1% (48/217) focused on medical students and physicians, 17.5% (38/217) focused on mixed students and health care professionals, and 6% (14/217) focused on nursing students and nurses. Reviews comprising studies on dentistry, pharmacy, and physiotherapy were less common: 1.4% (3/217) dentistry students and dentists, 0.9% (2/217) pharmacy students and pharmacists, and 0.5% (1/217) physiotherapy students and physiotherapists.
The included reviews investigated digital education interventions for learners at a variety of stages in their career: 45.2% (98/217) were created for both students and graduates, 29.5% (64/217) for students only, 10.1% (22/217) for practicing postgraduate health care professionals, 7.8% (17/217) for a mix of trainees and postgraduate health care professionals, and 7.4% (16/217) for graduate trainees.
Figure 2 displays a chart showing the number of systematic reviews published for each digital modality, organized by publication year. Several additional digital modalities were identified in the systematic reviews over and above the modalities included in the study by Tudor Car et al []. These were consolidated into three new categories:
- AI: this category includes traditional AI and generative AI, as well as novel AI-driven education modalities such as personalized learning, adaptive learning, AI-enabled extended reality (XR), chatbots, and virtual learning coaches. It also covers AI as an enabling technology, for example, when it helps educators produce content or generates data-driven insights.
- XR and immersive technologies: this category includes augmented reality, XR, mixed reality, and haptic technology. Virtual reality was also reclassified to fall within this definition.
- Simulation: this category includes high-fidelity simulators and robotic simulation.
Refer to for a full list of definitions.
Most systematic reviews focused on XR and immersive technologies (132/217, 60.8%) and online digital education (99/217, 45.6%). The least focus was given to serious gaming and gamification (16/217, 7.4%), simulation (15/217, 6.9%), massive online open courses (7/217, 3.2%), and AI (6/217, 2.8%).
Publication numbers were highest in 2022, although the annual totals between January 2020 and April 2023 (Figure 2) exceed those published in the original study by Tudor Car et al [].
Mapped against the conceptual framework of digital health education for health care professionals [], most systematic reviews were focused on education modality (170/217, 78.3%) and instructional design (99/217, 45.6%). Very few reviews focused on human resource requirements (7/217, 3.2%) and regulatory factors (6/217, 2.8%) in relation to digital education infrastructure, with only 0.9% (2/217) of reviews concerned with quality assurance of digital health education in the setting.
The majority of studies emphasized reporting educational outcomes related to skills (161/571, 28.1%) and knowledge (133/571, 23.3%), followed by studies focusing on satisfaction (77/571, 13.5%), attitude (54/571, 9.5%), and behavior (38/571, 6.7%). Few studies (29/571, 5.1%) reported patient outcomes.
Future Research Recommendations
A total of 151 future research recommendations were extracted from the 217 systematic reviews. These were analyzed, recategorized, and consolidated by 3 authors (AP, CM, and EW) to create a final list of 63 updated research questions. Refer to Tables 2 and 3 for the complete lists.
Across the questions, online digital education (299/1257, 23.7%) and XR and immersive technologies (266/1257, 21.1%) were the most predominant digital modalities requiring further research. Most questions suggest further inquiry within the following domains of education: modality (21 questions), instructional design (16 questions), and assessment (14 questions).
The least predominant modalities are massive online open courses (20/1257, 1.59%) and AI (18/1257, 1.43%).
| New research questions identified from the included papers | Evidence source references |
| Can the positive outcomes observed in studies at levels 2B and 3 of the Kirkpatrick model be further validated through studies at level 4, and what are the potential organizational and participant-level changes that may result from implementing these techniques? | [] |
| How do immersive technologies impact learning outcomes? | [-] |
| How do we measure the learning transfer from digital education to the clinical setting? | [-] |
| How does research and evidence inform education commissioning and selection of digital technologies? | [] |
| How does the design of VPSa impact learning outcomes in health professions education and training curriculum? | [,] |
| How does the use of social media as a pedagogical tool contribute to learner efficacy? | [,] |
| How effective is blended learning in different disciplines, age groups, and learning environments for nursing students? | [] |
| How can digital health education be used to offer an inclusive learning experience for staff and students? | [-] |
| How can we optimize the use of AIb, machine learning, and deep learning to facilitate education and training? | [,] |
| What are health care students’ learning needs, and can they be met by the use of digital gamification? | [-] |
| What are the advantages and disadvantages of the recognized standard setting methods in simulation-based training? | [,-] |
| What are the challenges of setting up digital simulation education in primary care? | [] |
| What are the experiences and attitudes of teachers and trainers toward developing and implementing digital learning designs in health care education? | [-] |
| What are the outcomes and implications of high-quality randomized controlled trials evaluating the impact of virtual reality and augmented reality simulators on the entire robotic surgery learning curve? | [,] |
| What are the specific effects of video as a learning delivery method? | [] |
| What automated performance metrics might prove effective in remote proctoring and assessment? For example, eye tracking | [] |
| What is the comparative efficacy of various e-learning platforms in teaching specific subtopics when standardized guidelines for information delivery and assessment are used? | [,] |
| What is the effectiveness of digital health education on long-term retention of skills and knowledge? | [,,-] |
| What is the effectiveness of high-fidelity simulation compared with other forms of simulation training? | [-] |
| What is the effectiveness of immersive technologies compared with other forms of simulation training? | [,,] |
| What is the effectiveness of synchronous digital education compared with asynchronous digital education to deliver health professions education? | [-] |
| What is the impact of digital health education on clinical outcomes in the short and long term? | [-] |
| What is the long-term impact of the COVID-19 pandemic on the educational gaps of health care learners? | [-] |
| What is the validity of using AI, machine learning, and deep learning to generate automated feedback for procedural skills training? | [,] |
aVPS: virtual patient simulation.
bAI: artificial intelligence.
| Research questions identified from the included papers | Evidence source references |
| What are the instructional design barriers and facilitators that affect the continued adoption of digital tools in health professions education? | [,,,-] |
| How does the frequency and duration of digital simulation–based psychomotor skills training affect health professionals’ skills transfer to the clinical setting? | [-] |
| How does the interactivity of digital education programs affect the learning and clinical outcomes of health professionals? | [,,,-] |
| What are the technical resources needed to deliver digital education to health care professionals? | [,,,,-] |
| Is spacing digital simulation–based training more or as effective as traditional education in clinical psychomotor skills development? | [,,-,,,,] |
| What are learners’ acceptability of digital education with different levels of interactivity? | [,,,,,,-] |
| What is the differential impact of digital education on the clinical performance of trainee or expert surgeons? | [,,-,-] |
| Can digital education be used to overcome challenges in delivering content-specific topics for health professions education (eg, surgical training in rare pathologic states)? | [,,,,,,,,-] |
| Which components of digital health education (eg, interactivity and feedback) contribute to enhanced learning outcomes? | [,,,,,,,,,-] |
| Can digital simulation–based training be used to train nontechnical skills in health professionals? | [,,,,,,,,,-] |
| Is health professions’ digital education more time efficient than traditional education? | [,,,,-,-] |
| Which learning theories can be used to inform the development of effective digital health professions education? | [,,,,,,,-] |
| Does digital simulation–based psychomotor skills training provide any benefit to medical trainees? | [,,,,-,,,-] |
| How does health professions’ digital education affect individual and health services outcomes and organizational practice? | [,,,,,,,,,,,,-] |
| What is the effectiveness of using digital education to train and assess nontechnical skills in health care professionals? | [,,,,,,,,,,-] |
| How do cost and cost-related outcomes influence the adoption of digital technology in health professions education? | [,,,,,,,,,-] |
| Which features of digital education (eg, technical features, fidelity, safety, and adaptability) affect the learning outcomes of health professions education? | [,,,,,,-,,,,,,,,,] |
| Is mastery learning via digital education more or as effective as traditional education in terms of clinical psychomotor skills improvement? | [,,,,,,,,,,,,,-] |
| What are health professionals’ attitudes toward digital delivery of education and training programs? | [,,,,,,,,,,,,,,,,,,] |
| What are the challenges of digital education for health professionals training in different socioeconomic settings? | [,,,,,,,-] |
| Can digital education be designed to achieve learning outcomes denoted in the Kirkpatrick model? | [,,,,,,,,,,,,,,,,,,] |
| What are health care professionals’ learning needs and can they be met using digital simulation training? | [,,,,,,,,,,,,,,,,,, ,,,] |
| What type of instructional design is used in the effective digital education of health professions education? | [,,,,,,,,,,,,,,,,,-] |
| What is the optimal duration, frequency, and intensity of digital health professions education programs to improve the learning and clinical outcomes for health care professionals? | [,,,,,-,,,,,,,,,,,] |
| What is the long-term cost-effectiveness of digital education compared with traditional education for health professionals? | [-,,,,,,,,,,,,,,,,-,,] |
| How does providing access to digital education improve the learning outcomes of health professionals? | [,,,,,,,,,-,,,,,,,,,, ,,,] |
| How should learning outcomes in the field of health professions’ digital education be defined and standardized? | [,,,,,,,,,,,,,,,,,,,, ,,,,-] |
| What pedagogy should be used in the digital education of health professionals to improve their knowledge and skills? | [,,-,,,,,,,,,,,-,,,,, -] |
| Can digital education complement (ie, blended) or substitute traditional education for health professionals? | [,,,,,,,,,,,,,,,,,,,,, ,-] |
| How can digital technology be incorporated into the current health professions education and training curriculum to improve learning outcomes? | [,,,,,,,,,-,,,,,,,, ,-] |
| What is the ideal approach to assessing health professionals’ knowledge, skills, attitudes, satisfaction, and clinical outcomes from digital technology–based education and training programs? | [,,,,,-,,,,,,,,,,,,,, ,,,,,,-] |
| How should studies on digital health professions education be reported? | [,,,,,,,,,,,,,,,,,,,,,, ,,,,,,,,,,,,,-] |
| Is digital education effective in improving health professionals’ knowledge and skills performance in the clinical setting? | [,,,,,,,,,-,,-,,,,,,,, ,,,,,,,,,,-,,,] |
| Which performance metrics or measurement instrument should be used to assess health professionals’ knowledge, skills, attitudes, satisfaction, and clinical outcomes from digital technology–based training programs? | [,,,,,,,,,,,,,,,,,,,,,, ,,,,,,,,,,,-,,,,,] |
| How does the design of digital education interventions (eg, format and modality) in health professions education and training curriculum affect learning outcomes? | [,,-,,,-,,,,,-,,,,-,,,,, ,,,,,,,,,,,-] |
| How should studies of digital health professions education be designed to ensure the generalizability of their findings across different settings? | [,,,,,,,,,,,,,,,,,-,,,, ,,,,,,-,,,,,,-,,,,, -,-] |
| What is the impact of digital simulation–based training on clinical outcomes in the short and long term? | [,,,,,,,,,,,,,,,,,-,,, ,,,,,,,,,,,,,,,,,,-] |
| What is the effectiveness of digital education (mixed or single modality) compared with nondigital education to deliver health professions education? | [,,,,,,,,,,,,,-,,,,,,,,,, ,,,,,,,,,-,,,,,-,, ,,,] |
| What are the methodological requirements for high-quality, rigorous studies assessing the outcomes of digital health education? | [,-,,,,,,,,,,,,,-,,,,,,,,, -,-,,,,,,,,,-,,,,,, ,,-,,,-,,,,,-,,,,,-] |
Delphi Consensus
A total of 42 experts were enrolled to participate in the Delphi consensus activity. Of these, there were 11 academics, 18 clinical educators, 14 simulation faculty, 6 immersive technology professionals, 6 workforce and transformation professionals, 1 learner, and 13 TEL professionals.
In round 1, a total of 151 research questions identified from the literature review were consolidated into 63 questions. These 63 questions were then used to form the web-based survey for round 2.
In round 2, experts assessed each question against the set criteria (scientific merit, significance on workforce development, innovation, relevance to the Long Term Workforce Plan, and feasibility for further study). Two participants chose to withdraw from the Delphi process after reviewing the survey. There was a 78% (31/40) response rate, with nonconsensus thresholds across all questions. Therefore, all 63 questions went through to the next round.
In round 3, in total, 53% (21/40) experts responded to the survey to determine the priority with which each question ought to be pursued with further research. A total of 18 research questions met the consensus threshold for the final round.
In the fourth and final round, the response rate was 55% (22/40), and consensus was reached on the 5 research questions (Table 4) that the expert group identified as the most important and likely to impact the future delivery of digital education.
| Research question | Sum of score |
| How do we measure the learning transfer from digital education to the clinical setting? | 47 |
| How can we optimize the use of AIa, machine learning, and deep learning to facilitate education and training? | 38 |
| What are the methodological requirements for high-quality, rigorous studies assessing the outcomes of digital health education? | 32 |
| How does the design of digital education interventions (eg, format and modality) in health professions education and training curriculum affect learning outcomes? | 31 |
| How should learning outcomes in the field of health professions’ digital education be defined and standardized? | 30 |
aAI: artificial intelligence.
Discussion
Principal Findings
This study presents findings from 217 systematic reviews on digital education in health care, published between 2020 and 2023. The findings expand upon previous research conducted by Tudor Car et al [] and define current gaps in the evidence for the efficacy of digital education. After analysis and consolidation, this study identified 63 future research questions: 24 new questions derived from the umbrella review and 39 from the original study by Tudor Car et al []. Most reviews focused on XR and immersive technologies as well as web-based digital education, targeting a mixed audience of students and in-service health care professionals. This was also the case for studies with the least commonly represented population groups (dentistry, pharmacy, and physiotherapy).
The 63 research questions were then prioritized through a consensus process with 40 experts from health care and academia, resulting in 5 key research questions (Table 4). These questions span several domains outlined in the conceptual framework []: assessment, engagement, learner, level of education, modality, and research.
The 5 Top-Rated Research Questions
Each of the top-rated questions is supported by a wealth of recommendations for conducting research in digital education. A high-level summary of this evidence is provided for each question.
Question 1: How Do We Measure the Learning Transfer from Digital Education to the Clinical Setting?
The evidence behind this research question points to several related yet distinct avenues of inquiry. A common theme among them is the need for further research to demonstrate how learning outcomes translate into real-world contexts, both for technical and nontechnical skills, particularly in nursing and medical fields [-]. One study specifically recommends further research to ensure that the competence and confidence in clinical practice skills demonstrated in a learning environment are maintained when applied in the clinical setting, where additional external factors may come into play []. Ensuring that the learning experience is relevant and closely mirrors real-world situations is a key consideration when designing instruction to facilitate effective transfer of knowledge and skills [].
Furthermore, studies should track and evaluate the effectiveness of learning throughout a health care professional’s career [], using clinical outcomes and patient-reported outcome measures to assess the transfer of learning [].
Question 2: How Can We Optimize the Use of AI, Machine Learning, and Deep Learning to Facilitate Education and Training?
The use of AI to optimize health education is a rapidly developing field. Current evidence proposes further research on the potential of this technology to elevate and shift performance assessment from benchtop and simulated environments to real-world context, offering more precise and scalable feedback mechanisms for both trainees and practitioners. For instance, machine learning has already shown promise in surgical skills training, where it could provide automated, objective assessments of technical skills, provided that more reliable assessment data were available []. Besides enhancing existing approaches to learning and assessment, AI also enables new modalities, such as personalized learning, which can enable more self-directed learning and may also support learner motivation by connecting data-driven insights to learning experiences, and adaptive learning []. AI integrated into personalized or adaptive learning could help students practice clinical decision-making by simulating different patient cases customized to their knowledge gaps based on previous performance. Furthermore, AI could generate dialogue, actions, and complex situations that replicate clinical settings, further enabling students to develop critical skills with reduced risk []. However, while AI offers substantial benefits, its implementation must be carefully considered to mitigate against potential challenges such as ethical concerns, bias from training data, and over-reliance on technology []. To safely and effectively adopt AI into health care education, a careful approach could integrate AI in a way that complements existing strategies that emphasize critical thinking and adaptability as well as technical proficiency.
Question 3: What Are the Methodological Requirements for High-Quality, Rigorous Studies Assessing the Outcomes of Digital Health Education?
Across the spectrum of digital education modalities, several studies highlighted the need for research to adopt appropriate study designs to determine learning gains [, , , , ,,,,,,,]. Designs need to be larger in scale [,,,,,,,,,,,], multicentered [,,,], and carry out longer-term follow-up [,,,,,,,,,,,] to ensure retention and transfer of learning. Many of the included studies recommended more randomized controlled trials [,,,,,,,,,,,,,,], although one paper supports the suggestion provided by Tudor Car et al [] that quasi-experimental designs may be better suited to educational research [] and another study, in relation to cultural competency, suggests that observational studies and other designs may be preferred [].
Question 4: How Does the Design of Digital Education Interventions (eg, Format and Modality) Used in Health Professions Education and Training Curriculum Affect Learning Outcomes?
Key recommendations from the data sources that derive this research question are primarily concerned with the integration and sequencing of pedagogical devices (ranging from flipped classrooms, timing of simulation briefing, e-learning, and active vs didactic approaches) in health education training and curricula as well as the quality of the approach used [,,,,,,,]. Some studies suggest delving further by closely comparing and evaluating the impact of nuanced difference in the design elements of an intervention or specific educational technology [,,,,].
Stepping back from this detail within the intervention design itself, studies also reinforce the need to define educational needs, the context, and the setting to select suitable, appropriate, and effective modalities for specific learner groups [,,,,].
Looking to the future, some studies highlight several novel technologies and suggest exploring their potential for designing more advanced educational interventions, such as AI [], adaptive learning [], wearable technology [], eye tracking for assessment [,], and gamification to increase learner motivation [].
While many still recommend further comparison studies between digital and nondigital interventions [,,], others support the recommendation by Tudor Car et al [] that comparisons between digital-to-digital interventions are now needed and necessary [,].
Question 5: How Should Learning Outcomes in the Field of Health Professions Digital Education Be Defined and Standardized?
Principally, the foundations of this research question lie within a body of evidence that urges a more standardized approach to outcome measurement, using validated measurement tools [,,,,]. Beyond this, there is little guidance on how to achieve this, although 1 study (specific to surgical skills training) suggests that machine learning could help identify which metrics accurately assess skills [].
In many ways, each of these research priorities are connected and interdependent. For example, to be able to measure learning transfer, learning outcomes need to be defined from the beginning and integrated into the learning design. This, in turn, will influence the research methodology used.
Comparison With Prior Work
This study builds upon and extends research by Tudor Car et al []. By doing so, the study not only updates their findings to include data up to 2023 but also extends them by prioritizing 5 research questions through Delphi consensus, thereby addressing a limitation previously identified by Tudor Car et al [].
In comparison to the 6-year period covered by Tudor Car et al [], the 3-year timeframe (2020 to 2023) in this study shows a dramatic increase in the number of publications on digital education in health care (from 77 to 217). This surge of activity is reflective of the rise in uptake of digital education in response to the COVID-19 pandemic.
Research with medical students and physicians has seen the greatest increase (from 12% to 22% of studies) and accounts for the largest portion of studies included in this review. In contrast, most earlier studies were more heterogeneous, with a combination of students and health professionals spanning several professions and specialties []. Dentistry staff and students remain one of the least represented groups in the literature, along with pharmacy and physiotherapy. The digital education modalities studied align with the broader outcomes of this study (namely, virtual reality, augmented reality, and web-based digital education), and their recommendations for future research mainly focus on the benefits, effectiveness, and perceptions of these modalities.
This study sees the addition of 3 new modalities: AI (in the form of personalized learning, adaptive learning, AI-enabled XR avatars, chatbots, and virtual learning coaches), XR and immersive technologies, and high-fidelity simulation. There has been an explosion of research around the use of XR technologies (132/217, 60.8% of included studies) as well as the emergence of AI (6/217, 3% of included studies). Aside from virtual reality, these technologies were not present in the earlier study.
There was a slight increase (from 5% to 14%) in the number of studies from high-, medium-, and low-income countries []. This could be indicative of greater collaborative relationships during the COVID-19 pandemic, as digital education was used globally to overcome the challenges of social distancing in health education delivery. Although evidence relating to digital education research in low-income countries continues to be sparce, the data from this study suggest that future research should focus more closely on learner audiences and intercultural differences to better understand what works across various socioeconomic settings [,].
While there have been noticeable shifts in the research populations and the education technologies studies since the findings shared by Tudor Car et al [], some aspects remain unchanged. Studies comparing digital education interventions with traditional education remain popular, with a continued emphasis on the randomized control trial study design. Comparison studies between digital and nondigital approaches have made significant progress in recent years. Researchers are now encouraged to focus more on comparisons between digital methods in future studies to further advance digital education research. However, despite a move toward research focusing on the context of the learner since the study by Tudor Car et al [], this review has shown that there continues to be a lack of studies relating to quality assurance, human resources, and regulatory aspects of digital education infrastructure.
Strengths and Limitations
The findings of this extensive umbrella review replicated a robust methodology used by Tudor Car et al [] and expanded it to identify the most pressing research questions currently needing attention in digital education within UK health care. Furthermore, these findings were strengthened by consultation with 40 stakeholders across health care and academia to arrive at consensus for the 5 research questions.
By replicating the methodology used by Tudor Car et al [], this study was constrained by many of the same limitations set out in their study, particularly the exclusion of original research outside of systematic reviews. In addition, this study excluded conference abstracts and non-English papers. However, the impact of excluding non-English papers is considered minimal on the overall outcome [], and while there may be advantages in including conference abstracts in a systematic review [], they did not provide the level of information required for data extraction in this study.
There was very low representation from the learner stakeholder group on the Delphi expert panel. Initially, there was uncertainty among the research team members about whether learners would be classed as “experts” for a Delphi panel, which delayed promotion to this group. According to Nasa et al [], an “expert can be defined as someone with knowledge and experience on a particular subject matter.” In addition, there were similarities drawn with the concept of “experts by experience” []. While there are significant differences between being a patient, carer, or service user and a student or learner, the principles of “experts by experience” were considered to be similar. This approach involves gathering the views of people receiving the interventions, in this case, their views and expertise regarding their experience with education. Therefore, a heterogeneous group of stakeholders including final-year undergraduate health care students and postgraduate students or trainees, including doctors in training, was considered applicable to the aims of this study.
Next Steps
Having defined 5 high-level, key research priorities, it remains to determine what the next steps should be. Stefanidis et al [] observed that, in the field of surgical simulation, little actual research evidence has been generated despite numerous papers attempting to define priorities. To avoid repeating this shortcoming, the next stage should be to identify specific research questions that have the greatest potential to have significant and practical impact across the system. As proposed by Stefanidis et al [], establishing a number of expert working groups to devise specific, high-impact and feasible research proposals based on these 5 priorities is suggested. These ideas will subsequently be developed into formal research projects including research questions, collaborators, project design, and required resources. It is anticipated that successful completion of these projects will answer important questions about the practice of digital education in health care.
Conclusions
This review provides a list of current research gaps in digital education in health care, presenting them as a set of research questions along with the sources from which they are derived.
While the ongoing focus of digital education research on medical trainees and professionals is acknowledged, there is a need for more research involving nursing and allied health professions. It also highlights that there continues to be a need to address the quality assurance, human resources, and regulatory aspects of digital education infrastructure through research.
The prioritized set of 5 research questions, developed with input from expert representatives across health care and academia, is expected to serve as a valuable guide for researchers, funding agencies, and educators. For researchers, these questions offer a helpful starting point for addressing some of the existing gaps in knowledge. For funding agencies, they provide a framework for allocating resources to the areas identified as top priorities for the health care system. Finally, for educators and education providers, these questions can inform decision-making regarding implementation, curriculum design, and quality.
Acknowledgments
The authors would like to thank their volunteers (Angela Holmes, Claire Beattie, Harbie Rathore-Khatkar, Chris Jacobs, Ian Blacklock, Eleni Keogh, Sarah Gray, and Matt O’Brien) for their time and commitment throughout the process. The authors would also like to thank Nick Peres, Patricia Howe, and Faisal Mushtaq for their valuable advice and for helping to refine the methodological approach. The authors also thank the experts who participated in the Delphi process.
Conflicts of Interest
NR is the Deputy Director of Technology Enhanced Learning at NHS England. This team delivers digital learning services and products directly to the National Health Service (NHS) as well as providing system leadership and transformation programs.
References
- Vaicondam Y, Sikandar H, Irum S, Khan N, Qureshi MI. Research landscape of digital learning over the past 20 years: a bibliometric and visualisation analysis. Int J Online Biomed Eng. Jun 28, 2022;18(08):4-22. [FREE Full text] [CrossRef]
- Sweileh WM. Global research activity on e-learning in health sciences education: a bibliometric analysis. Med Sci Educ. Mar 02, 2021;31(2):765-775. [FREE Full text] [CrossRef] [Medline]
- Working for health and growth: investing in the health workforce - high-level commission on health employment and economic growth. World Health Organization. Dec 03, 2016. URL: https://www.who.int/publications/i/item/9789241511308 [accessed 2024-06-04]
- Transforming our world: the 2030 agenda for sustainable development. United Nations Sustainable Development Goals Knowledge Platform. 2015. URL: https://sustainabledevelopment.un.org/post2015/transformingourworld/publication [accessed 2024-07-01]
- NHS long term workforce plan. NHS England. URL: https://www.england.nhs.uk/publication/nhs-long-term-workforce-plan/ [accessed 2024-05-16]
- The Topol review: preparing the healthcare workforce to deliver the digital future. The Topol Review, Health Education England. URL: https://topol.hee.nhs.uk/ [accessed 2024-07-01]
- Jia K, Wang P, Li Y, Chen Z, Jiang X, Lin CL, et al. Research landscape of artificial intelligence and e-learning: a bibliometric research. Front Psychol. Feb 16, 2022;13:795039. [FREE Full text] [CrossRef] [Medline]
- Zhang P, Li X, Pan Y, Zhai H, Li T. Global trends and future directions in online learning for medical students during and after the COVID-19 pandemic: a bibliometric and visualization analysis. Medicine (Baltimore). Dec 15, 2023;102(50):e35377. [FREE Full text] [CrossRef] [Medline]
- Martins J, Gonçalves R, Branco F. A bibliometric analysis and visualization of e-learning adoption using VOSviewer. Univers Access Inf Soc. Dec 05, 2022:1-15. [FREE Full text] [CrossRef] [Medline]
- Fauzi MA. E-learning in higher education institutions during COVID-19 pandemic: current and future trends through bibliometric analysis. Heliyon. May 2022;8(5):e09433. [FREE Full text] [CrossRef] [Medline]
- Yao S, Tang Y, Yi C, Xiao Y. Research hotspots and trend exploration on the clinical translational outcome of simulation-based medical education: a 10-year scientific bibliometric analysis from 2011 to 2021. Front Med (Lausanne). Feb 07, 2022;8:801277. [FREE Full text] [CrossRef] [Medline]
- Li S. Immersive technologies in health professions education: a bibliometric analysis. Comput Educ X Real. 2024;4:100051. [CrossRef]
- Rojas-Sánchez MA, Palos-Sánchez PR, Folgado-Fernández JA. Systematic literature review and bibliometric analysis on virtual reality and education. Educ Inf Technol (Dordr). 2023;28(1):155-192. [FREE Full text] [CrossRef] [Medline]
- Report: post-pandemic progress – lessons learnt in healthcare education. Council of Deans of Health. May 02, 2024. URL: https://tinyurl.com/mu6p73xm [accessed 2024-06-17]
- Universities powering the NHS. Universities UK. URL: https://tinyurl.com/5y8x6we8 [accessed 2024-06-17]
- Logeswaran A, Munsch C, Chong YJ, Ralph N, McCrossnan J. The role of extended reality technology in healthcare education: towards a learner-centred approach. Future Healthc J. Mar 2021;8(1):e79-e84. [FREE Full text] [CrossRef] [Medline]
- Mathew RK, Immersive Healthcare Collaboration, Mushtaq F. Three principles for the progress of immersive technologies in healthcare training and education. BMJ Simul Technol Enhanc Learn. May 25, 2021;7(5):459-460. [FREE Full text] [CrossRef] [Medline]
- Anton N, Calhoun AC, Stefanidis D. Current research priorities in healthcare simulation: results of a Delphi survey. Simul Healthc. Feb 01, 2022;17(1):e1-e7. [CrossRef] [Medline]
- Stefanidis D, Lee G, Blair PG, Johnson KA, Sachdeva AK. What are the top research priorities in surgical simulation and how can they be best addressed? Results from a multidisciplinary consensus conference. Ann Surg. Dec 01, 2022;276(6):e1052-e1056. [CrossRef] [Medline]
- Anton NE, Gardner AK, Stefanidis D. Priorities in surgical simulation research: what do the experts say? Am J Surg. Jul 2020;220(1):95-99. [CrossRef] [Medline]
- Garfinkle R, Petersen RP, DuCoin C, Altieri MS, Aggarwal R, Pryor A, et al. Consensus priority research questions in gastrointestinal and endoscopic surgery in the year 2020: results of a SAGES Delphi study. Surg Endosc. Sep 2022;36(9):6688-6695. [CrossRef] [Medline]
- Cook DA, Andersen DK, Combes JR, Feldman DL, Sachdeva AK. The value proposition of simulation-based education. Surgery. Apr 2018;163(4):944-949. [CrossRef] [Medline]
- Johnston MJ, Paige JT, Aggarwal R, Stefanidis D, Tsuda S, Khajuria A, et al. An overview of research priorities in surgical simulation: what the literature shows has been achieved during the 21st century and what remains. Am J Surg. Jan 2016;211(1):214-225. [CrossRef] [Medline]
- Hunt EA, Duval-Arnould J, Chime NO, Auerbach M, Kessler D, Duff JP, et al. Building consensus for the future of paediatric simulation: a novel 'KJ Reverse-Merlin' methodology. BMJ Simul Technol Enhanc Learn. Apr 12, 2016;2(2):35-41. [FREE Full text] [CrossRef] [Medline]
- McLaughlin C, Barry W, Barin E, Kysh L, Auerbach MA, Upperman JS, et al. Multidisciplinary simulation-based team training for trauma resuscitation: a scoping review. J Surg Educ. 2019;76(6):1669-1680. [CrossRef] [Medline]
- Tudor Car L, Poon S, Kyaw BM, Cook DA, Ward V, Atun R, et al. Digital education for health professionals: an evidence map, conceptual framework, and research agenda. J Med Internet Res. Mar 17, 2022;24(3):e31977. [FREE Full text] [CrossRef] [Medline]
- Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. Sep 2015;13(3):132-140. [CrossRef] [Medline]
- RefWorks login. ProQuest LLC. URL: https://refworks.proquest.com [accessed 2024-05-21]
- Online survey. Jisc. URL: https://www.onlinesurveys.ac.uk/ [accessed 2024-08-13]
- Schneider P, Evaniew N, Rendon JS, McKay P, Randall RL, Turcotte R, et al. Moving forward through consensus: protocol for a modified Delphi approach to determine the top research priorities in the field of orthopaedic oncology. BMJ Open. May 24, 2016;6(5):e011780. [FREE Full text] [CrossRef] [Medline]
- Slido homepage. Slido. URL: https://www.slido.com/ [accessed 2024-08-13]
- Duarte ML, Santos LR, Iared W, Peccin MS. Comparison of ultrasonography learning between distance teaching and traditional methodology. An educational systematic review. Sao Paulo Med J. 2022;140(6):806-817. [FREE Full text] [CrossRef] [Medline]
- Ryan GV, Callaghan S, Rafferty A, Higgins MF, Mangina E, McAuliffe F. Learning outcomes of immersive technologies in health care student education: systematic review of the literature. J Med Internet Res. Feb 01, 2022;24(2):e30082. [FREE Full text] [CrossRef] [Medline]
- Zhao G, Fan M, Yuan Y, Zhao F, Huang H. The comparison of teaching efficiency between virtual reality and traditional education in medical education: a systematic review and meta-analysis. Ann Transl Med. Feb 2021;9(3):252. [FREE Full text] [CrossRef] [Medline]
- Suresh D, Aydin A, James S, Ahmed K, Dasgupta P. The role of augmented reality in surgical training: a systematic review. Surg Innov. Jun 22, 2023;30(3):366. [FREE Full text] [CrossRef] [Medline]
- Shorey S, Ng ED. The use of virtual reality simulation among nursing students and registered nurses: a systematic review. Nurse Educ Today. Mar 2021;98:104662. [CrossRef] [Medline]
- Chahal B, Aydın A, Amin MS, Ong K, Khan A, Khan MS, et al. Transfer of open and laparoscopic skills to robotic surgery: a systematic review. J Robot Surg. Aug 2023;17(4):1207-1225. [FREE Full text] [CrossRef] [Medline]
- Dzyuba N, Jandu J, Yates J, Kushnerev E. Virtual and augmented reality in dental education: the good, the bad and the better. Eur J Dent Educ. Nov 06, 2022. (forthcoming). [CrossRef] [Medline]
- Sim JJ, Rusli KD, Seah B, Levett-Jones T, Lau Y, Liaw SY. Virtual simulation to enhance clinical reasoning in nursing: a systematic review and meta-analysis. Clin Simul Nurs. Aug 2022;69:26-39. [CrossRef] [Medline]
- Tolarba JE. Virtual simulation in nursing education: a systematic review. Int J Nurs Educ. Jun 14, 2021;13(3):48-54. [CrossRef]
- Ashokka B, Dong C, Law LS, Liaw SY, Chen FG, Samarasekera DD. A BEME systematic review of teaching interventions to equip medical students and residents in early recognition and prompt escalation of acute clinical deteriorations: BEME Guide No. 62. Med Teach. Jul 2020;42(7):724-737. [CrossRef] [Medline]
- Patel EA, Aydin A, Cearns M, Dasgupta P, Ahmed K. A systematic review of simulation-based training in neurosurgery, part 2: spinal and pediatric surgery, neurointerventional radiology, and nontechnical skills. World Neurosurg. Jan 2020;133:e874-e892. [CrossRef] [Medline]
- Lei YY, Zhu L, Sa YT, Cui XS. Effects of high-fidelity simulation teaching on nursing students' knowledge, professional skills and clinical ability: a meta-analysis and systematic review. Nurse Educ Pract. Mar 2022;60:103306. [CrossRef] [Medline]
- Stone R, Cooke M, Mitchell M. Undergraduate nursing students' use of video technology in developing confidence in clinical skills for practice: a systematic integrative literature review. Nurse Educ Today. Jan 2020;84:104230. [CrossRef] [Medline]
- Lohre R, Wang JC, Lewandrowski KU, Goel DP. Virtual reality in spinal endoscopy: a paradigm shift in education to support spine surgeons. J Spine Surg. Jan 2020;6(Suppl 1):S208-S223. [FREE Full text] [CrossRef] [Medline]
- Tori AA, Tori R, Nunes F. Serious game design in health education: a systematic review. IEEE Trans Learning Technol. Dec 1, 2022;15(6):827-846. [CrossRef]
- Plackett R, Kassianos AP, Mylan S, Kambouri M, Raine R, Sheringham J. The effectiveness of using virtual patient educational tools to improve medical students' clinical reasoning skills: a systematic review. BMC Med Educ. May 13, 2022;22(1):365. [FREE Full text] [CrossRef] [Medline]
- Beshir SA, Mohamed AP, Soorya A, Sir Loon Goh S, Moussa El-Labadd E, Hussain N, et al. Virtual patient simulation in pharmacy education: a systematic review. Pharm Educ. Dec 17, 2022;22(1):954-970. [CrossRef]
- Fouasson-Chailloux A, Daley P, Menu P, Gross R, Dauty M. Social media in health studies: a systematic review of comparative learning methods. Int J Environ Res Public Health. Feb 15, 2022;19(4):2205. [FREE Full text] [CrossRef] [Medline]
- Almutairi M, Simpson A, Khan E, Dickinson T. The value of social media use in improving nursing students' engagement: a systematic review. Nurse Educ Pract. Oct 2022;64:103455. [FREE Full text] [CrossRef] [Medline]
- Du L, Zhao L, Xu T, Wang Y, Zu W, Huang X, et al. Blended learning vs traditional teaching: the potential of a novel teaching strategy in nursing education - a systematic review and meta-analysis. Nurse Educ Pract. Aug 2022;63:103354. [CrossRef] [Medline]
- Walshe N, Condon C, Gonzales RA, Burke E, Chianáin LN, Thamanam N, et al. Cultural simulations, authenticity, focus, and outcomes: a systematic review of the healthcare literature. Clin Simul Nurs. Oct 2022;71:65-81. [CrossRef]
- Yogeswaran V, El Morr C. Effectiveness of online mindfulness interventions on medical students' mental health: a systematic review. BMC Public Health. Dec 18, 2021;21(1):2293. [FREE Full text] [CrossRef] [Medline]
- Chen B, Wang Y, Xiao L, Xu C, Shen Y, Qin Q, et al. Effects of mobile learning for nursing students in clinical education: a meta-analysis. Nurse Educ Today. Feb 2021;97:104706. [CrossRef] [Medline]
- Baetzner AS, Wespi R, Hill Y, Gyllencreutz L, Sauter TC, Saveman BI, et al. Preparing medical first responders for crises: a systematic literature review of disaster training programs and their effectiveness. Scand J Trauma Resusc Emerg Med. Dec 24, 2022;30(1):76. [FREE Full text] [CrossRef] [Medline]
- Chiang FK, Shang X, Qiao L. Augmented reality in vocational training: a systematic review of research and applications. Comput Hum Behav. Apr 2022;129:107125. [CrossRef]
- Lam K, Chen J, Wang Z, Iqbal FM, Darzi A, Lo B, et al. Machine learning for technical skill assessment in surgery: a systematic review. NPJ Digit Med. Mar 03, 2022;5(1):24. [FREE Full text] [CrossRef] [Medline]
- Noyes JA, Welch PM, Johnson JW, Carbonneau KJ. A systematic review of digital badges in health care education. Med Educ. Jul 2020;54(7):600-615. [CrossRef] [Medline]
- Thangavelu DP, Tan AJ, Cant R, Chua WL, Liaw SY. Digital serious games in developing nursing clinical competence: a systematic review and meta-analysis. Nurse Educ Today. Jun 2022;113:105357. [CrossRef] [Medline]
- Min A, Min H, Kim S. Effectiveness of serious games in nurse education: a systematic review. Nurse Educ Today. Jan 2022;108:105178. [CrossRef] [Medline]
- Chen IH, Ghazi A, Sridhar A, Stoyanov D, Slack M, Kelly JD, et al. Evolving robotic surgery training and improving patient safety, with the integration of novel technologies. World J Urol. Aug 2021;39(8):2883-2893. [FREE Full text] [CrossRef] [Medline]
- van Gaalen AE, Brouwer J, Schönrock-Adema J, Bouwkamp-Timmer T, Jaarsma AD, Georgiadis JR. Gamification of health professions education: a systematic review. Adv Health Sci Educ Theory Pract. May 2021;26(2):683-711. [FREE Full text] [CrossRef] [Medline]
- Xu Y, Lau Y, Cheng LJ, Lau ST. Learning experiences of game-based educational intervention in nursing students: a systematic mixed-studies review. Nurse Educ Today. Dec 2021;107:105139. [CrossRef] [Medline]
- Lucena-Anton D, Fernandez-Lopez JC, Pacheco-Serrano AI, Garcia-Munoz C, Moral-Munoz JA. Virtual and augmented reality versus traditional methods for teaching physiotherapy: a systematic review. Eur J Investig Health Psychol Educ. Dec 02, 2022;12(12):1780-1792. [FREE Full text] [CrossRef] [Medline]
- Kim EJ, Lim JY, Kim GM. A systematic review and meta-analysis of studies on extended reality-based pediatric nursing simulation program development. Child Health Nurs Res. Jan 31, 2023;29(1):24-36. [FREE Full text] [CrossRef] [Medline]
- Pietersen P, Bjerrum F, Tolsgaard MG, Konge LS, Andersen SA. Standard setting in simulation-based training of surgical procedures: a systematic review. Ann Surg. May 01, 2022;275(5):872-882. [CrossRef] [Medline]
- Moro C, Birt J, Stromberga Z, Phelps C, Clark J, Glasziou P, et al. Virtual and augmented reality enhancements to medical and science student physiology and anatomy test performance: a systematic review and meta-analysis. Anat Sci Educ. May 2021;14(3):368-376. [CrossRef] [Medline]
- Bray L, Krogh TB, Østergaard D. Simulation-based training for continuing professional development within a primary care context: a systematic review. Educ Prim Care. Mar 02, 2023;34(2):64-73. [CrossRef] [Medline]
- Nowell L, Dhingra S, Carless-Kane S, McGuinness C, Paolucci A, Jacobsen M, et al. A systematic review of online education initiatives to develop students remote caring skills and practices. Med Educ Online. Dec 12, 2022;27(1):2088049. [FREE Full text] [CrossRef] [Medline]
- Ødegaard NB, Myrhaug HT, Dahl-Michelsen T, Røe Y. Digital learning designs in physiotherapy education: a systematic review and meta-analysis. BMC Med Educ. Jan 13, 2021;21(1):48. [FREE Full text] [CrossRef] [Medline]
- Shorey S, Pereira TL, Teo WZ, Ang E, Lau TC, Samarasekera DD. Navigating nursing curriculum change during COVID-19 pandemic: a systematic review and meta-synthesis. Nurse Educ Pract. Nov 2022;65:103483. [FREE Full text] [CrossRef] [Medline]
- Srinivasa K, Chen Y, Henning MA. The role of online videos in teaching procedural skills to post-graduate medical learners: a systematic narrative review. Med Teach. Jun 2020;42(6):689-697. [CrossRef] [Medline]
- Anugrahsari S, Chaeruman UA, Abbas H, Suryadi S. Patient safety education for clinical students: a systematic literature review. Open Access Maced J Med Sci. Mar 11, 2022;10(F):208-214. [CrossRef]
- Grafton-Clarke C, Uraiby H, Gordon M, Clarke N, Rees E, Park S, et al. Pivot to online learning for adapting or continuing workplace-based clinical learning in medical education following the COVID-19 pandemic: a BEME systematic review: BEME Guide No. 70. Med Teach. Mar 2022;44(3):227-243. [CrossRef] [Medline]
- Savage AJ, McNamara PW, Moncrieff TW, O'Reilly GM. Review article: e-learning in emergency medicine: a systematic review. Emerg Med Australas. Jun 27, 2022;34(3):322-332. [FREE Full text] [CrossRef] [Medline]
- Aydın A, Baig U, Al-Jabir A, Sarıca K, Dasgupta P, Ahmed K. Simulation-based training models for urolithiasis: a systematic review. J Endourol. Jul 01, 2021;35(7):1098-1117. [CrossRef] [Medline]
- Guckian J, Utukuri M, Asif A, Burton O, Adeyoju J, Oumeziane A, et al. Social media in undergraduate medical education: a systematic review. Med Educ. Nov 04, 2021;55(11):1227-1241. [CrossRef] [Medline]
- Tursø-Finnich T, Jensen RO, Jensen LX, Konge L, Thinggaard E. Virtual reality head-mounted displays in medical education: a systematic review. Simul Healthc. Feb 01, 2023;18(1):42-50. [CrossRef] [Medline]
- Clerkin R, Patton D, Moore Z, Nugent L, Avsar P, O'Connor T. What is the impact of video as a teaching method on achieving psychomotor skills in nursing? A systematic review and meta-analysis. Nurse Educ Today. Apr 2022;111:105280. [FREE Full text] [CrossRef] [Medline]
- Foronda CL, Fernandez-Burgos M, Nadeau C, Kelley CN, Henry MN. Virtual simulation in nursing education: a systematic review spanning 1996 to 2018. Simul Healthc. 2020;15(1):46-54. [CrossRef]
- Piot MA, Dechartres A, Attoe C, Romeo M, Jollant F, Billon G, et al. Effectiveness of simulation in psychiatry for nursing students, nurses and nurse practitioners: a systematic review and meta-analysis. J Adv Nurs. Feb 11, 2022;78(2):332-347. [CrossRef] [Medline]
- Lapierre A, Bouferguene S, Gauvin-Lepage J, Lavoie P, Arbour C. Effectiveness of interprofessional manikin-based simulation training on teamwork among real teams during trauma resuscitation in adult emergency departments: a systematic review. Simul Healthc. Dec 2020;15(6):409-421. [CrossRef] [Medline]
- Li YY, Au ML, Tong LK, Ng WI, Wang SC. High-fidelity simulation in undergraduate nursing education: a meta-analysis. Nurse Educ Today. Apr 2022;111:105291. [FREE Full text] [CrossRef] [Medline]
- Alshehri FD, Jones S, Harrison D. The effectiveness of high-fidelity simulation on undergraduate nursing students' clinical reasoning-related skills: a systematic review. Nurse Educ Today. Feb 2023;121:105679. [CrossRef] [Medline]
- Hildreth AF, Maggio LA, Iteen A, Wojahn AL, Cook DA, Battista A. Technology-enhanced simulation in emergency medicine: updated systematic review and meta-analysis 1991-2021. AEM Educ Train. Apr 14, 2023;7(2):e10848. [FREE Full text] [CrossRef] [Medline]
- Kim JY, Kim ME. Can online learning be a reliable alternative to nursing students' learning during a pandemic? - A systematic review and meta-analysis. Nurse Educ Today. Mar 2023;122:105710. [CrossRef] [Medline]
- Muirhead K, Macaden L, Smyth K, Chandler C, Clarke C, Polson R, et al. Establishing the effectiveness of technology-enabled dementia education for health and social care practitioners: a systematic review. Syst Rev. Sep 21, 2021;10(1):252. [FREE Full text] [CrossRef] [Medline]
- Muirhead K, Macaden L, Smyth K, Chandler C, Clarke C, Polson R, et al. The characteristics of effective technology-enabled dementia education: a systematic review and mixed research synthesis. Syst Rev. Feb 23, 2022;11(1):34. [FREE Full text] [CrossRef] [Medline]
- Abualadas HM, Xu L. Achievement of learning outcomes in non-traditional (online) versus traditional (face-to-face) anatomy teaching in medical schools: a mixed method systematic review. Clin Anat. Jan 25, 2023;36(1):50-76. [FREE Full text] [CrossRef] [Medline]
- Arqub SA, Waleed M, Al-Abedalla K, Iverson MG, Uribe F. Insight on the influence of technology-enhanced learning in orthodontics' education: a systematic review. Eur J Dent Educ. Aug 31, 2023;27(3):729-745. [CrossRef] [Medline]
- Martinengo L, Yeo NJ, Markandran KD, Olsson M, Kyaw BM, Car LT. Digital health professions education on chronic wound management: a systematic review. Int J Nurs Stud. Apr 2020;104:103512. [CrossRef] [Medline]
- Wirth T, Peters C, Nienhaus A, Schablon A. Interventions for workplace violence prevention in emergency departments: a systematic review. Int J Environ Res Public Health. Aug 10, 2021;18(16):8459. [FREE Full text] [CrossRef] [Medline]
- Lindhard MS, Thim S, Laursen HS, Schram AW, Paltved CT, Henriksen TB. Simulation-based neonatal resuscitation team training: a systematic review. Pediatrics. Apr 2021;147(4):e2020042010. [CrossRef] [Medline]
- Lee IR, Kim HW, Lee Y, Koyanagi A, Jacob L, An S, et al. Changes in undergraduate medical education due to COVID-19: a systematic review. Eur Rev Med Pharmacol Sci. Jun 2021;25(12):4426-4434. [FREE Full text] [CrossRef] [Medline]
- Patano A, Cirulli N, Beretta M, Plantamura P, Inchingolo AD, Inchingolo AM, et al. Education technology in orthodontics and paediatric dentistry during the COVID-19 pandemic: a systematic review. Int J Environ Res Public Health. Jun 04, 2021;18(11):6056. [FREE Full text] [CrossRef] [Medline]
- Dedeilia A, Sotiropoulos MG, Hanrahan JG, Janga D, Dedeilias P, Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo. Jun 05, 2020;34(3 Suppl):1603-1611. [FREE Full text] [CrossRef] [Medline]
- Brown N, Margus C, Hart A, Sarin R, Hertelendy A, Ciottone G. Virtual reality training in disaster medicine: a systematic review of the literature. Simul Healthc. Aug 01, 2023;18(4):255-261. [CrossRef] [Medline]
- Castillo-Segura P, Fernández-Panadero C, Alario-Hoyos C, Muñoz-Merino PJ, Delgado Kloos C. Objective and automated assessment of surgical technical skills with IoT systems: a systematic literature review. Artif Intell Med. Feb 2021;112:102007. [CrossRef] [Medline]
- Davids J, Manivannan S, Darzi A, Giannarou S, Ashrafian H, Marcus HJ. Simulation for skills training in neurosurgery: a systematic review, meta-analysis, and analysis of progressive scholarly acceptance. Neurosurg Rev. Aug 18, 2021;44(4):1853-1867. [FREE Full text] [CrossRef] [Medline]
- Regmi K, Jones L. A systematic review of the factors - enablers and barriers - affecting e-learning in health sciences education. BMC Med Educ. Mar 30, 2020;20(1):91. [FREE Full text] [CrossRef] [Medline]
- Lockey A, Bland A, Stephenson J, Bray J, Astin F. Blended learning in health care education: an overview and overarching meta-analysis of systematic reviews. J Contin Educ Health Prof. Oct 01, 2022;42(4):256-264. [CrossRef] [Medline]
- Banks L, Kay R. Exploring flipped classrooms in undergraduate nursing and health science: a systematic review. Nurse Educ Pract. Oct 2022;64:103417. [CrossRef] [Medline]
- Tabatabaeichehr M, Babaei S, Dartomi M, Alesheikh P, Tabatabaee A, Mortazavi H, et al. Medical students’ satisfaction level with e-learning during the COVID-19 pandemic and its related factors: a systematic review. J Educ Eval Health Prof. Dec 20, 2022;19:37. [FREE Full text] [CrossRef] [Medline]
- Oh SY, Cook DA, van Gerven PW, Nicholson J, Fairbrother H, Smeenk FW, et al. Physician training for electrocardiogram interpretation: a systematic review and meta-analysis. Acad Med. Apr 01, 2022;97(4):593-602. [CrossRef] [Medline]
- Haiser A, Aydin A, Kunduzi B, Ahmed K, Dasgupta P. A systematic review of simulation-based training in vascular surgery. J Surg Res. Nov 2022;279:409-419. [FREE Full text] [CrossRef] [Medline]
- Chae D, Yoo JY, Kim J, Ryu J. Effectiveness of virtual simulation to enhance cultural competence in pre-licensure and licensed health professionals: a systematic review. Clin Simul Nurs. Jul 2021;56:137-154. [CrossRef]
- Mao RQ, Lan L, Kay J, Lohre R, Ayeni OR, Goel DP, et al. Immersive virtual reality for surgical training: a systematic review. J Surg Res. Dec 2021;268:40-58. [CrossRef] [Medline]
- Gomaa AR, Grafton-Clarke C, Saratzis A, Davies RS. The role of high-fidelity simulation in the acquisition of endovascular surgical skills: a systematic review. Ann Vasc Surg. Jul 2023;93:405-427. [FREE Full text] [CrossRef] [Medline]
- Howlader D, Daga D, Mehrotra D. The scope of computerized simulation in competency-based maxillofacial training: a systematic review. Int J Oral Maxillofac Surg. Aug 2022;51(8):1101-1110. [CrossRef] [Medline]
- Fahl JT, Duvivier R, Reinke L, Pierie JP, Schönrock-Adema J. Towards best practice in developing motor skills: a systematic review on spacing in VR simulator-based psychomotor training for surgical novices. BMC Med Educ. Mar 13, 2023;23(1):154. [FREE Full text] [CrossRef] [Medline]
- Schmidt MW, Köppinger KF, Fan C, Kowalewski KF, Schmidt LP, Vey J, et al. Virtual reality simulation in robot-assisted surgery: meta-analysis of skill transfer and predictability of skill. BJS Open. Mar 05, 2021;5(2):zraa066. [FREE Full text] [CrossRef] [Medline]
- Ortega MA, Marchese VG, Zarro MJ, Film RJ, Shipper AG, Felter C. Digital and blended curriculum delivery in health professions education: an umbrella review with implications for Doctor of Physical Therapy education programs. Phys Ther Rev. Nov 09, 2021;27(1):4-24. [CrossRef]
- Aweid B, Haider Z, Wehbe M, Hunter A. Educational benefits of the online journal club: a systematic review. Med Teach. Jan 2022;44(1):57-62. [CrossRef] [Medline]
- Özbay Ö, Çınar S. Effectiveness of flipped classroom teaching models in nursing education: a systematic review. Nurse Educ Today. Jul 2021;102:104922. [CrossRef] [Medline]
- Mao BP, Teichroeb ML, Lee T, Wong G, Pang T, Pleass H. Is online video-based education an effective method to teach basic surgical skills to students and surgical trainees? A systematic review and meta-analysis. J Surg Educ. 2022;79(6):1536-1545. [FREE Full text] [CrossRef] [Medline]
- Yucel C, Hawley G, Terzioglu F, Bogossian F. The effectiveness of simulation-based team training in obstetrics emergencies for improving technical skills: a systematic review. Simul Healthc. Apr 2020;15(2):98-105. [CrossRef] [Medline]
- Jujo S, Nakahira A, Kataoka Y, Banno M, Tsujimoto Y, Tsujimoto H, et al. Transesophageal echocardiography simulator training: a systematic review and meta-analysis of randomized controlled trials. Simul Healthc. Oct 01, 2021;16(5):341-352. [CrossRef] [Medline]
- Gharib AM, Bindoff IK, Peterson GM, Salahudeen MS. Computer-based simulators in pharmacy practice education: a systematic narrative review. Pharmacy (Basel). Jan 02, 2023;11(1):8. [FREE Full text] [CrossRef] [Medline]
- Lo CK, Hew KF. Design principles for fully online flipped learning in health professions education: a systematic review of research during the COVID-19 pandemic. BMC Med Educ. Oct 13, 2022;22(1):720. [FREE Full text] [CrossRef] [Medline]
- Polce EM, Kunze KN, Williams BT, Krivicich LM, Maheshwer B, Beletsky A, et al. Efficacy and validity of orthopaedic simulators in surgical training: a systematic review and meta-analysis of randomized controlled trials. J Am Acad Orthop Surg. Dec 15, 2020;28(24):1027-1040. [CrossRef] [Medline]
- Neubauer R, Recker F, Bauer CJ, Brossart P, Schäfer VS. The current situation of musculoskeletal ultrasound education: a systematic literature review. Ultrasound Med Biol. Jun 2023;49(6):1363-1374. [CrossRef] [Medline]
- Santos GN, da Silva HE, Leite AF, Mesquita CR, Figueiredo PT, Stefani CM, et al. The scope of dental education during COVID-19 pandemic: a systematic review. J Dent Educ. Jul 23, 2021;85(7):1287-1300. [FREE Full text] [CrossRef] [Medline]
- Rothschild P, Richardson A, Beltz J, Chakrabarti R. Effect of virtual reality simulation training on real-life cataract surgery complications: systematic literature review. J Cataract Refract Surg. Mar 01, 2021;47(3):400-406. [CrossRef] [Medline]
- Capitani P, Zampogna B, Monaco E, Frizziero A, Moretti L, Losco M, et al. The role of virtual reality in knee arthroscopic simulation: a systematic review. Musculoskelet Surg. Mar 15, 2023;107(1):85-95. [CrossRef] [Medline]
- Dickinson KJ, Bass BL. A systematic review of educational mobile-applications (apps) for surgery residents: simulation and beyond. J Surg Educ. Sep 2020;77(5):1244-1256. [CrossRef] [Medline]
- Lin IC, Lee A, Mauch JT. Does e-learning improve plastic surgery education?: a systematic review of asynchronous resources. Ann Plast Surg. Jul 01, 2021;87(1s Suppl 1):S40-S51. [CrossRef] [Medline]
- Naciri A, Radid M, Kharbach A, Chemsi G. E-learning in health professions education during the COVID-19 pandemic: a systematic review. J Educ Eval Health Prof. Oct 29, 2021;18:27. [FREE Full text] [CrossRef] [Medline]
- O'Brien B, Bevan K, Brockington C, Murphy J, Gilbert R. Effects of simulation-based cardiopulmonary and respiratory case training experiences on interprofessional teamwork: a systematic review. Can J Respir Ther. Mar 28, 2023;59:85-94. [FREE Full text] [CrossRef] [Medline]
- Plotzky C, Lindwedel U, Sorber M, Loessl B, König P, Kunze C, et al. Virtual reality simulations in nurse education: a systematic mapping review. Nurse Educ Today. Jun 2021;101:104868. [CrossRef] [Medline]
- Gelmini AY, Duarte ML, Silva MO, Guimarães Junior JB, Santos LR. Augmented reality in interventional radiology education: a systematic review of randomized controlled trials. Sao Paulo Med J. 2022;140(4):604-614. [FREE Full text] [CrossRef] [Medline]
- Aditya I, Kwong JC, Canil T, Lee JY, Goldenberg MG. Current educational interventions for improving technical skills of urology trainees in endourological procedures: a systematic review. J Endourol. Jul 01, 2020;34(7):723-731. [CrossRef] [Medline]
- Orejuela FJ, Aschkenazi SO, Howard DL, Jeppson PC, Balgobin S, Walter AJ, et al. Gynecologic surgical skill acquisition through simulation with outcomes at the time of surgery: a systematic review and meta-analysis. Am J Obstet Gynecol. Jul 2022;227(1):29.e1-2.e24. [CrossRef] [Medline]
- Clarke E. Virtual reality simulation-the future of orthopaedic training? A systematic review and narrative analysis. Adv Simul (Lond). Jan 13, 2021;6(1):2. [FREE Full text] [CrossRef] [Medline]
- Zaed I, Chibbaro S, Ganau M, Tinterri B, Bossi B, Peschillo S, et al. Simulation and virtual reality in intracranial aneurysms neurosurgical training: a systematic review. J Neurosurg Sci. Dec 2022;66(6):494-500. [CrossRef] [Medline]
- Cant R, Ryan CM, Kelly MA. Use and effectiveness of virtual simulations in nursing student education: an umbrella review. Comput Inform Nurs. Jan 01, 2023;41(1):31-38. [CrossRef] [Medline]
- Garnier A, Vanherp R, Bonnabry P, Bouchoud L. Use of simulation for education in hospital pharmaceutical technologies: a systematic review. Eur J Hosp Pharm. Mar 23, 2023;30(2):70-76. [FREE Full text] [CrossRef] [Medline]
- Gelmini AY, Duarte ML, de Assis AM, Guimarães Junior JB, Carnevale FC. Virtual reality in interventional radiology education: a systematic review. Radiol Bras. Aug 2021;54(4):254-260. [FREE Full text] [CrossRef] [Medline]
- Fontaine G, Zagury-Orly I, Maheu-Cadotte MA, Lapierre A, Thibodeau-Jarry N, Denus SD, et al. A meta-analysis of the effect of paper versus digital reading on reading comprehension in health professional education. Am J Pharm Educ. Nov 2021;85(10):8525. [FREE Full text] [CrossRef] [Medline]
- Margulies IG, Xu H, Henderson PW. Microsurgery training in the digital era: a systematic review of accessible digital resources. Ann Plast Surg. Oct 9, 2020;85(4):337-343. [CrossRef] [Medline]
- Ozdemir NG, Kaya H. The effect of high-fidelity simulation on experiences of urinary catheterization in nursing students: a mixed-method systematic review. Int J Caring Sci. 2022;15(2):1218-1232. [FREE Full text]
- Berg MN, Ngune I, Schofield P, Grech L, Juraskova I, Strasser M, et al. Effectiveness of online communication skills training for cancer and palliative care health professionals: a systematic review. Psychooncology. Sep 28, 2021;30(9):1405-1419. [CrossRef] [Medline]
- Oliveira Silva G, Oliveira FS, Coelho AS, Fonseca LM, Vieira FV, Campbell SH, et al. Influence of simulation design on stress, anxiety and self-confidence of nursing students: systematic review with meta-analysis. J Clin Nurs. Sep 09, 2023;32(17-18):5668-5692. [CrossRef] [Medline]
- Richardson CL, White S, Chapman S. Virtual patient technology to educate pharmacists and pharmacy students on patient communication: a systematic review. BMJ Simul Technol Enhanc Learn. Dec 11, 2020;6(6):332-338. [FREE Full text] [CrossRef] [Medline]
- Hao X, Peng X, Ding X, Qin Y, Lv M, Li J, et al. Application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic: a systematic review. Nurse Educ Today. Jan 2022;108:105183. [FREE Full text] [CrossRef] [Medline]
- Ong CW, Tan MC, Lam M, Koh VT. Applications of extended reality in ophthalmology: systematic review. J Med Internet Res. Aug 19, 2021;23(8):e24152. [FREE Full text] [CrossRef] [Medline]
- Tonapa SI, Mulyadi M, Ho KF, Efendi F. Effectiveness of using high-fidelity simulation on learning outcomes in undergraduate nursing education: systematic review and meta-analysis. Eur Rev Med Pharmacol Sci. Jan 2023;27(2):444-458. [FREE Full text] [CrossRef] [Medline]
- Woodhead NJ, Mahmud A, Clark J. Effects of simulation for gynaecological ultrasound scan training: a systematic review. BMJ Simul Technol Enhanc Learn. Oct 30, 2020;6(6):320-331. [FREE Full text] [CrossRef] [Medline]
- Zhao J, Xu X, Jiang H, Ding Y. The effectiveness of virtual reality-based technology on anatomy teaching: a meta-analysis of randomized controlled studies. BMC Med Educ. Apr 25, 2020;20(1):127. [FREE Full text] [CrossRef] [Medline]
- Smith SC, Siau K, Cannatelli R, Antonelli G, Shivaji UN, Ghosh S, et al. Training methods in optical diagnosis and characterization of colorectal polyps: a systematic review and meta-analysis. Endosc Int Open. May 22, 2021;9(5):E716-E726. [FREE Full text] [CrossRef] [Medline]
- de Gagne JC, Koppel PD, Wang EJ, Rushton S, Ledbetter L, Yamane SS, et al. A systematic review of videoconferencing in health professions education: the digital divide revisited in the COVID-19 era. Int J Nurs Educ Scholarsh. Jan 01, 2023;20(1). [FREE Full text] [CrossRef] [Medline]
- Zhang X, Al-Mekhled D, Choate J. Are virtual physiology laboratories effective for student learning? A systematic review. Adv Physiol Educ. Sep 01, 2021;45(3):467-480. [FREE Full text] [CrossRef] [Medline]
- O'Connor S, Wang Y, Cooke S, Ali A, Kennedy S, Lee JJ, et al. Designing and delivering digital learning (e-Learning) interventions in nursing and midwifery education: a systematic review of theories. Nurse Educ Pract. May 2023;69:103635. [FREE Full text] [CrossRef] [Medline]
- Lee J, Kim H, Kim KH, Jung D, Jowsey T, Webster CS. Effective virtual patient simulators for medical communication training: a systematic review. Med Educ. Sep 22, 2020;54(9):786-795. [CrossRef] [Medline]
- Qiao J, Huang CR, Liu Q, Li S, Xu J, Li L, et al. Effectiveness of non-immersive virtual reality simulation in learning knowledge and skills for nursing students: meta-analysis. Clin Simul Nurs. Mar 2023;76:26-38. [CrossRef]
- Lowe S, Mares K, Khadjesari Z. Immersive technology in ophthalmology education: a systematic review. BMJ Simul Technol Enhanc Learn. Jun 29, 2021;7(6):600-604. [FREE Full text] [CrossRef] [Medline]
- Stojan J, Haas M, Thammasitboon S, Lander L, Evans S, Pawlik C, et al. Online learning developments in undergraduate medical education in response to the COVID-19 pandemic: a BEME systematic review: BEME Guide No. 69. Med Teach. Feb 2022;44(2):109-129. [CrossRef] [Medline]
- O'Connor S, Kennedy S, Wang Y, Ali A, Cooke S, Booth RG. Theories informing technology enhanced learning in nursing and midwifery education: a systematic review and typological classification. Nurse Educ Today. Nov 2022;118:105518. [FREE Full text] [CrossRef] [Medline]
- Patel EA, Aydin A, Cearns M, Dasgupta P, Ahmed K. A systematic review of simulation-based training in neurosurgery, part 1: cranial neurosurgery. World Neurosurg. Jan 2020;133:e850-e873. [CrossRef] [Medline]
- Sun P, Zhao Y, Men J, Ma ZR, Jiang HZ, Liu CY, et al. Application of virtual and augmented reality technology in hip surgery: systematic review. J Med Internet Res. Mar 10, 2023;25:e37599. [FREE Full text] [CrossRef] [Medline]
- Davies H, Sundin D, Robinson S, Jacob E. Does participation in extended immersive ward-based simulation improve the preparedness of undergraduate bachelor's degree nursing students to be ready for clinical practice as a registered nurse? An integrative literature review. J Clin Nurs. Oct 19, 2021;30(19-20):2897-2911. [CrossRef] [Medline]
- Al Asmri MA, Ennis J, Stone RJ, Bello F, Haque MS, Parle J. Effectiveness of technology-enhanced simulation in teaching digital rectal examination: a systematic review narrative synthesis. BMJ Simul Technol Enhanc Learn. Sep 01, 2021;7(5):414-421. [FREE Full text] [CrossRef] [Medline]
- Susilawati E, Johari A, Marzal J, Anggereini E. Effects of multimedia e-books and augmented reality on knowledge and skills of health sciences students: a systematic review. J Client Centered Nurs Care. 2022;8(2):99-110. [CrossRef]
- Taba JV, Cortez VS, Moraes WA, Iuamoto LR, Hsing WT, Suzuki MO, et al. The development of laparoscopic skills using virtual reality simulations: a systematic review. PLoS One. Jun 17, 2021;16(6):e0252609. [FREE Full text] [CrossRef] [Medline]
- Maheu-Cadotte MA, Cossette S, Dubé V, Fontaine G, Lavallée A, Lavoie P, et al. Efficacy of serious games in healthcare professions education: a systematic review and meta-analysis. Simul Healthc. Jun 01, 2021;16(3):199-212. [CrossRef] [Medline]
- Tudor Car L, Kyaw BM, Teo A, Fox TE, Vimalesvaran S, Apfelbacher C, et al. Outcomes, measurement instruments, and their validity evidence in randomized controlled trials on virtual, augmented, and mixed reality in undergraduate medical education: systematic mapping review. JMIR Serious Games. Apr 13, 2022;10(2):e29594. [FREE Full text] [CrossRef] [Medline]
- Brown L, Ilhan E, Pacey V, Hau W, van der Kooi V, Dale M. The effect of high-fidelity simulation–based learning in acute cardiorespiratory physical therapy—a mixed-methods systematic review. J Phys Ther. Educ. Apr 14, 2021;35(2):146-158. [CrossRef]
- Portelli M, Bianco S, Bezzina T, Abela JE. Virtual reality training compared with apprenticeship training in laparoscopic surgery: a meta-analysis. Ann R Coll Surg Engl. Nov 2020;102(9):672-684. [FREE Full text] [CrossRef] [Medline]
- Lee R, Raison N, Lau WY, Aydin A, Dasgupta P, Ahmed K, et al. A systematic review of simulation-based training tools for technical and non-technical skills in ophthalmology. Eye (Lond). Oct 13, 2020;34(10):1737-1759. [FREE Full text] [CrossRef] [Medline]
- Jin C, Dai L, Wang T. The application of virtual reality in the training of laparoscopic surgery: a systematic review and meta-analysis. Int J Surg. Mar 2021;87:105859. [FREE Full text] [CrossRef] [Medline]
- Engberg M, Taudorf M, Rasmussen NK, Russell L, Lönn L, Konge L. Training and assessment of competence in resuscitative endovascular balloon occlusion of the aorta (REBOA) - a systematic review. Injury. Feb 2020;51(2):147-156. [CrossRef] [Medline]
- Hippe DS, Umoren RA, McGee A, Bucher SL, Bresnahan BW. A targeted systematic review of cost analyses for implementation of simulation-based education in healthcare. SAGE Open Med. 2020;8:2050312120913451. [FREE Full text] [CrossRef] [Medline]
- Youhasan P, Chen Y, Lyndon M, Henning MA. Exploring the pedagogical design features of the flipped classroom in undergraduate nursing education: a systematic review. BMC Nurs. Mar 22, 2021;20(1):50. [FREE Full text] [CrossRef] [Medline]
- Rangarajan K, Davis H, Pucher PH. Systematic review of virtual haptics in surgical simulation: a valid educational tool? J Surg Educ. Mar 2020;77(2):337-347. [CrossRef] [Medline]
- Singh T, Reyes-Portillo JA. Using technology to train clinicians in evidence-based treatment: a systematic review. Psychiatr Serv. Apr 01, 2020;71(4):364-377. [CrossRef] [Medline]
- Umphrey L, Lenhard N, Lam SK, Hayward NE, Hecht S, Agrawal P, et al. Virtual global health in graduate medical education: a systematic review. Int J Med Educ. Aug 31, 2022;13:230-248. [FREE Full text] [CrossRef] [Medline]
- Asegid A, Assefa N. Effect of simulation-based teaching on nursing skill performance: a systematic review and meta-analysis. Front Nurs. 2021;8(3). [CrossRef]
- Jones A, Wilson I, McClean S, Kerr D, Breen C. Supporting the learning experience of health‐related profession students during clinical placements with technology: a systematic review. Rev Educ. Aug 16, 2022;10(2):e3364. [CrossRef]
- Khalaf K, El-Kishawi M, Mustafa S, Al Kawas S. Effectiveness of technology-enhanced teaching and assessment methods of undergraduate preclinical dental skills: a systematic review of randomized controlled clinical trials. BMC Med Educ. Aug 28, 2020;20(1):286. [FREE Full text] [CrossRef] [Medline]
- Olexa J, Cohen J, Alexander T, Brown C, Schwartzbauer GG, Woodworth GF. Expanding educational frontiers in neurosurgery: current and future uses of augmented reality. Neurosurgery. Feb 01, 2023;92(2):241-250. [CrossRef] [Medline]
- Berthold DP, Muench LN, Rupp MC, Siebenlist S, Cote MP, Mazzocca AD, et al. Head-mounted display virtual reality is effective in orthopaedic training: a systematic review. Arthrosc Sports Med Rehabil. Oct 2022;4(5):e1843-e1849. [FREE Full text] [CrossRef] [Medline]
- Kovoor JG, Gupta AK, Gladman MA. Validity and effectiveness of augmented reality in surgical education: a systematic review. Surgery. Jul 2021;170(1):88-98. [CrossRef] [Medline]
- Pires C. Perceptions of pharmacy students on the e-learning strategies adopted during the COVID-19 pandemic: a systematic review. Pharmacy (Basel). Feb 15, 2022;10(1):31. [FREE Full text] [CrossRef] [Medline]
- Helming AG, Adler DS, Keltner C, Igelman AG, Woodworth GE. The content quality of YouTube videos for professional medical education: a systematic review. Acad Med. Oct 01, 2021;96(10):1484-1493. [CrossRef] [Medline]
- Gordon M, Patricio M, Horne L, Muston A, Alston SR, Pammi M, et al. Developments in medical education in response to the COVID-19 pandemic: a rapid BEME systematic review: BEME Guide No. 63. Med Teach. Nov 2020;42(11):1202-1215. [CrossRef] [Medline]
- Chou WK, Ullah N, Arjomandi Rad A, Vardanyan R, Shah V, Zubarevich A, et al. Simulation training for obstetric emergencies in low- and lower-middle income countries: a systematic review. Eur J Obstet Gynecol Reprod Biol. Sep 2022;276:74-81. [CrossRef] [Medline]
- Sultana N, Betran AP, Khan KS, Sobhy S. Simulation-based teaching and models for caesarean sections: a systematic review to evaluate the tools for the 'See One, Practice Many, Do One' slogan. Curr Opin Obstet Gynecol. Oct 2020;32(5):305-315. [CrossRef] [Medline]
- Berry MC, de M Neto JM, de Souza MI, Figueredo CM, Reher V, Evans JL. Effectiveness of technology-enhanced learning to improve periodontics educational outcomes: a systematic review. J Dent Educ. Jul 18, 2020;84(7):830-839. [CrossRef] [Medline]
- Nielsen CA, Lönn L, Konge L, Taudorf M. Simulation-based virtual-reality patient-specific rehearsal prior to endovascular procedures: a systematic review. Diagnostics (Basel). Jul 20, 2020;10(7):500. [FREE Full text] [CrossRef] [Medline]
- Higgins M, Madan C, Patel R. Development and decay of procedural skills in surgery: a systematic review of the effectiveness of simulation-based medical education interventions. Surgeon. Aug 2021;19(4):e67-e77. [CrossRef] [Medline]
- Asadzandi S, Mojtahedzadeh R, Mohammadi A. What are the factors that enhance metacognitive skills in nursing students? A systematic review. Iran J Nurs Midwifery Res. 2022;27(6):475-484. [FREE Full text] [CrossRef] [Medline]
- Delungahawatta T, Dunne SS, Hyde S, Halpenny L, McGrath D, O'Regan A, et al. Advances in e-learning in undergraduate clinical medicine: a systematic review. BMC Med Educ. Oct 07, 2022;22(1):711. [FREE Full text] [CrossRef] [Medline]
- Lee J, Campbell S, Choi M, Bae J. Authentic learning in healthcare education: a systematic review. Nurse Educ Today. Dec 2022;119:105596. [CrossRef] [Medline]
- Keinänen AL, Lähdesmäki R, Juntunen J, Tuomikoski AM, Kääriäinen M, Mikkonen K. Effectiveness of mentoring education on health care professionals´ mentoring competence: a systematic review. Nurse Educ Today. Feb 2023;121:105709. [FREE Full text] [CrossRef] [Medline]
- Sezgin MG, Bektas H. Effectiveness of interprofessional simulation-based education programs to improve teamwork and communication for students in the healthcare profession: a systematic review and meta-analysis of randomized controlled trials. Nurse Educ Today. Jan 2023;120:105619. [CrossRef] [Medline]
- Svellingen AH, Søvik MB, Røykenes K, Brattebø G. The effect of multiple exposures in scenario-based simulation-a mixed study systematic review. Nurs Open. Jan 29, 2021;8(1):380-394. [FREE Full text] [CrossRef] [Medline]
- Woon AP, Mok WQ, Chieng YJ, Zhang HM, Ramos P, Mustadi HB, et al. Effectiveness of virtual reality training in improving knowledge among nursing students: a systematic review, meta-analysis and meta-regression. Nurse Educ Today. Mar 2021;98:104655. [CrossRef] [Medline]
- Sleiman Z, Bitar R, Christoforou C, Török P, Fichera M, Panella MM, et al. Is there proof of transferability of laparoscopic psychomotor skills from the simulator laboratory to the operating room? Results from a systematic review. J Obstet Gynaecol. Feb 24, 2022;42(2):181-187. [FREE Full text] [CrossRef] [Medline]
- Lakhani S, Selim OA, Saeed MZ. Arthroscopic simulation: the future of surgical training: a systematic review. JBJS Rev. Mar 2021;9(3):Ae20.00076. [CrossRef] [Medline]
- Conte DB, Zancanaro M, Guollo A, Schneider LR, Lund RG, Rodrigues-Junior SA. Educational interventions to improve dental anatomy carving ability of dental students: a systematic review. Anat Sci Educ. Jan 08, 2021;14(1):99-109. [CrossRef] [Medline]
- Chawla S, Devi S, Calvachi P, Gormley WB, Rueda-Esteban R. Evaluation of simulation models in neurosurgical training according to face, content, and construct validity: a systematic review. Acta Neurochir (Wien). Apr 04, 2022;164(4):947-966. [FREE Full text] [CrossRef] [Medline]
- Iop A, El-Hajj VG, Gharios M, de Giorgio A, Monetti FM, Edström E, et al. Extended reality in neurosurgical education: a systematic review. Sensors (Basel). Aug 14, 2022;22(16):6067. [FREE Full text] [CrossRef] [Medline]
- Au ML, Tong LK, Li YY, Ng WI, Wang SC. Impact of scenario validity and group size on learning outcomes in high-fidelity simulation: a systematics review and meta-analysis. Nurse Educ Today. Feb 2023;121:105705. [CrossRef] [Medline]
- Hovgaard LH, Al-Shahrestani F, Andersen SA. Current evidence for simulation-based training and assessment of myringotomy and ventilation tube insertion: a systematic review. Otol Neurotol. Oct 01, 2021;42(9):e1188-e1196. [CrossRef] [Medline]
- Tokas T, Ortner G, Peteinaris A, Somani BK, Herrmann T, Nagele U, et al. Simulation training in transurethral resection/laser vaporization of the prostate; evidence from a systematic review by the European Section of Uro-Technology. World J Urol. May 2022;40(5):1091-1110. [CrossRef] [Medline]
- Vallée A, Blacher J, Cariou A, Sorbets E. Blended learning compared to traditional learning in medical education: systematic review and meta-analysis. J Med Internet Res. Aug 10, 2020;22(8):e16504. [FREE Full text] [CrossRef] [Medline]
- Gao J, Yang L, Zou J, Fan X. Comparison of the influence of massive open online courses and traditional teaching methods in medical education in China: a meta-analysis. Biochem Mol Biol Educ. Jul 24, 2021;49(4):639-651. [FREE Full text] [CrossRef] [Medline]
- Baashar Y, Alkawsi G, Ahmad WN, Alhussian H, Alwadain A, Capretz LF, et al. Effectiveness of using augmented reality for training in the medical professions: meta-analysis. JMIR Serious Games. Jul 05, 2022;10(3):e32715. [FREE Full text] [CrossRef] [Medline]
- Sinacori B, Williams-Gregory M. The effect of distance learning on knowledge acquisition in undergraduate second-degree nursing students: a systematic review. Nurs Educ Perspect. Dec 16, 2021;42(3):136-141. [CrossRef] [Medline]
- ShahAli S, Shahabi S, Kohan N, Ebrahimi Takamjani I, Ebrahimi R. Using e-learning methods for physiotherapy students learning – a systematic review and meta-analysis of the impact on knowledge, skills, satisfaction and attitudes. Eur J Physiother. Jun 17, 2022;25(5):252-264. [CrossRef]
- Chan J, Pangal DJ, Cardinal T, Kugener G, Zhu Y, Roshannai A, et al. A systematic review of virtual reality for the assessment of technical skills in neurosurgery. Neurosurg Focus. Aug 2021;51(2):E15. [CrossRef] [Medline]
- Tudor Car L, Kyaw BM, Dunleavy G, Smart NA, Semwal M, Rotgans JI, et al. Digital problem-based learning in health professions: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. Feb 28, 2019;21(2):e12945. [FREE Full text] [CrossRef] [Medline]
- Astbury J, Ferguson J, Silverthorne J, Willis S, Schafheutle E. High-fidelity simulation-based education in pre-registration healthcare programmes: a systematic review of reviews to inform collaborative and interprofessional best practice. J Interprof Care. Jun 12, 2021;35(4):622-632. [CrossRef] [Medline]
- Daniel R, McKechnie T, Kruse CC, Levin M, Lee Y, Doumouras AG, et al. Video-based coaching for surgical residents: a systematic review and meta-analysis. Surg Endosc. Feb 23, 2023;37(2):1429-1439. [FREE Full text] [CrossRef] [Medline]
- Lee W, Kim GU, Yoon HJ, Kim S. A systematic review of the effect of web-based trauma-education programs for mental-health professionals. J Korean Acad Psychiatr Ment Health Nurs. Dec 2020;29(4):325-338. [CrossRef]
- Gosak L, Štiglic G, Budler LC, Félix IB, Braam K, Fijačko N, et al. Digital tools in behavior change support education in health and other students: a systematic review. Healthcare (Basel). Dec 21, 2021;10(1):1. [FREE Full text] [CrossRef] [Medline]
- Kanika, Kaur H. Effect of blended learning on academic achievements and attitude of nursing students: a systematic review. Indian J Forensic Med Toxicol. Jul 30, 2020;14(3):383-387. [CrossRef]
- Chan SC, Choa G, Kelly J, Maru D, Rashid MA. Implementation of virtual OSCE in health professions education: a systematic review. Med Educ. Sep 20, 2023;57(9):833-843. [CrossRef] [Medline]
- Legoux C, Gerein R, Boutis K, Barrowman N, Plint A. Retention of critical procedural skills after simulation training: a systematic review. AEM Educ Train. Jul 16, 2021;5(3):e10536. [FREE Full text] [CrossRef] [Medline]
- Nayar SK, Musto L, Baruah G, Fernandes R, Bharathan R. Self-assessment of surgical skills: a systematic review. J Surg Educ. Mar 2020;77(2):348-361. [CrossRef] [Medline]
- Rashidian N, Vierstraete M, Alseidi A, Troisi RI, Willaert W. Surgical education interventions in liver surgery: a systematic review. Updates Surg. Sep 27, 2020;72(3):583-594. [CrossRef] [Medline]
- Santos VA, Barreira MP, Saad KR. Technological resources for teaching and learning about human anatomy in the medical course: systematic review of literature. Anat Sci Educ. Mar 2022;15(2):403-419. [CrossRef] [Medline]
- Mydin FH, Yuen CW, Othman S. The effectiveness of educational intervention in improving primary health-care service providers' knowledge, identification, and management of elder abuse and neglect: a systematic review. Trauma Violence Abuse. Oct 11, 2021;22(4):944-960. [CrossRef] [Medline]
- Jallad ST, Işık B. The effectiveness of virtual reality simulation as learning strategy in the acquisition of medical skills in nursing education: a systematic review. Ir J Med Sci. Jun 05, 2022;191(3):1407-1426. [CrossRef] [Medline]
- Ayivi-Vinz G, Bakwa Kanyinga F, Bergeron L, Décary S, Adisso ÉL, Zomahoun HT, et al. Use of the CPD-REACTION questionnaire to evaluate continuing professional development activities for health professionals: systematic review. JMIR Med Educ. May 02, 2022;8(2):e36948. [FREE Full text] [CrossRef] [Medline]
- Jhou HJ, Ou-Yang LJ, Lin MH, Chen PH, Ho CL, Lee CH. Different pedagogies for acquisition of knowledge and skill: a systematic review and network meta-analysis. Postgrad Med J. Aug 30, 2022;98(1162):604-609. [CrossRef] [Medline]
- Adewuyi M, Morales K, Lindsey A. Impact of experiential dementia care learning on knowledge, skills and attitudes of nursing students: a systematic literature review. Nurse Educ Pract. Jul 2022;62:103351. [CrossRef] [Medline]
- Tropea J, Bicknell R, Nestel D, Brand CA, Johnson CE, Paul SK, et al. Simulation training in non-cancer palliative care for healthcare workers: a systematic review of controlled studies. BMJ Simul Technol Enhanc Learn. Aug 14, 2021;7(4):262-269. [FREE Full text] [CrossRef] [Medline]
- Dromey BP, Peebles DM, Stoyanov DV. A systematic review and meta-analysis of the use of high-fidelity simulation in obstetric ultrasound. Simul Healthc. Feb 01, 2021;16(1):52-59. [FREE Full text] [CrossRef] [Medline]
- Gawronski O, Thekkan KR, Genna C, Egman S, Sansone V, Erba I, et al. Instruments to evaluate non-technical skills during high fidelity simulation: a systematic review. Front Med (Lausanne). Nov 3, 2022;9:986296. [FREE Full text] [CrossRef] [Medline]
- Ahmed TM, Hussain B, Siddiqui MA. Can simulators be applied to improve cataract surgery training: a systematic review. BMJ Open Ophthalmol. Sep 08, 2020;5(1):e000488. [FREE Full text] [CrossRef] [Medline]
- Corvetto MA, Altermatt FR, Belmar F, Escudero E. Health care simulation as a training tool for epidemic management: a systematic review. Simul Healthc. Dec 01, 2023;18(6):382-391. [CrossRef] [Medline]
- Chan RJ, Agbejule OA, Yates PM, Emery J, Jefford M, Koczwara B, et al. Outcomes of cancer survivorship education and training for primary care providers: a systematic review. J Cancer Surviv. Apr 24, 2022;16(2):279-302. [FREE Full text] [CrossRef] [Medline]
- Lin JC, Yu Z, Scott IU, Greenberg PB. Virtual reality training for cataract surgery operating performance in ophthalmology trainees. Cochrane Database Syst Rev. Dec 21, 2021;12(12):CD014953. [FREE Full text] [CrossRef] [Medline]
- Balakrishnan A, Puthean S, Satheesh G, M K U, Rashid M, Nair S, et al. Effectiveness of blended learning in pharmacy education: a systematic review and meta-analysis. PLoS One. Jun 17, 2021;16(6):e0252461. [FREE Full text] [CrossRef] [Medline]
- Chandran VP, Balakrishnan A, Rashid M, Pai Kulyadi G, Khan S, Devi ES, et al. Mobile applications in medical education: a systematic review and meta-analysis. PLoS One. Mar 24, 2022;17(3):e0265927. [FREE Full text] [CrossRef] [Medline]
- Heuer A, Bienstock J, Zhang Y. Simulation-based training within selected allied health professions: an evidence-based systematic review. J Allied Health. 2022;51(1):59-71. [Medline]
- Jacob C, Sanchez-Vazquez A, Ivory C. Social, organizational, and technological factors impacting clinicians' adoption of mobile health tools: systematic literature review. JMIR Mhealth Uhealth. Feb 20, 2020;8(2):e15935. [FREE Full text] [CrossRef] [Medline]
- Verville L, Dc PC, Grondin D, Mior S, Moodley K, Kay R, et al. Using technology-based educational interventions to improve knowledge about clinical practice guidelines. J Chiropr Educ. Mar 01, 2021;35(1):149-157. [FREE Full text] [CrossRef] [Medline]
- Moussa R, Alghazaly A, Althagafi N, Eshky R, Borzangy S. Effectiveness of virtual reality and interactive simulators on dental education outcomes: systematic review. Eur J Dent. Feb 24, 2022;16(1):14-31. [FREE Full text] [CrossRef] [Medline]
- Okano H, Mayumi T, Kataoka Y, Banno M, Tsujimoto Y, Shiroshita A, et al. Outcomes of simulation-based education for vascular access: a systematic review and meta-analysis. Cureus. Aug 2021;13(8):e17188. [FREE Full text] [CrossRef] [Medline]
- Richard O, Jollant F, Billon G, Attoe C, Vodovar D, Piot MA. Simulation training in suicide risk assessment and intervention: a systematic review and meta-analysis. Med Educ Online. Dec 18, 2023;28(1):2199469. [FREE Full text] [CrossRef] [Medline]
- Ge L, Chen Y, Yan C, Chen Z, Liu J. Effectiveness of flipped classroom vs traditional lectures in radiology education: a meta-analysis. Medicine (Baltimore). Oct 02, 2020;99(40):e22430. [FREE Full text] [CrossRef] [Medline]
- Le Lous M, Simon O, Lassel L, Lavoue V, Jannin P. Hybrid simulation for obstetrics training: a systematic review. Eur J Obstet Gynecol Reprod Biol. Mar 2020;246:23-28. [CrossRef] [Medline]
- Rueda Esteban RJ, López-McCormick JS, Rodríguez-Bermeo AS, Andrade M, Hernández Restrepo JD, Targarona Soler EM. Face, content, and construct validity evaluation of simulation models in general surgery laparoscopic training and education: a systematic review. Surg Innov. Apr 04, 2023;30(2):251-260. [CrossRef] [Medline]
- Li BZ, Cao NW, Ren CX, Chu XJ, Zhou HY, Guo B. Flipped classroom improves nursing students' theoretical learning in China: a meta-analysis. PLoS One. Aug 27, 2020;15(8):e0237926. [FREE Full text] [CrossRef] [Medline]
- Chen J, Xun H, Abousy M, Long C, Sacks JM. No microscope? No problem: a systematic review of microscope-free microsurgery training models. J Reconstr Microsurg. Feb 23, 2022;38(2):106-114. [CrossRef] [Medline]
- Schafer KM, Kremer MJ. Outcomes of simulation-based experiences related to family presence during resuscitation: a systematic review. Clin Simul Nurs. Apr 2022;65:62-81. [CrossRef]
- Shank PO. Practice and Feedback for Deeper Learning: 26 Evidence-Based and Easy-to-Apply Tactics That Promote Deeper Learning and Application. Scotts Valley, CA. Createspace Independent Publishing; 2017.
- Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel). Mar 19, 2023;11(6):887. [FREE Full text] [CrossRef] [Medline]
- Nussbaumer-Streit B, Klerings I, Dobrescu AI, Persad E, Affengruber L, Gartlehner G. Excluding non-English studies from systematic reviews does not change conclusions: a meta-epidemiological study. In: Proceedings of the Cochrane Colloquium 2019. 2019. Presented at: Cochrane Colloquium 2019; October 22-25, 2019; Santiago, Chile. [CrossRef]
- Scherer RW, Saldanha IJ. How should systematic reviewers handle conference abstracts? A view from the trenches. Syst Rev. Nov 07, 2019;8(1):264. [FREE Full text] [CrossRef] [Medline]
- Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: how to decide its appropriateness. World J Methodol. Jul 20, 2021;11(4):116-129. [FREE Full text] [CrossRef] [Medline]
- Experts by experience. Care Quality Commission. URL: https://www.cqc.org.uk/about-us/jobs/experts-experience [accessed 2024-07-08]
Abbreviations
| AI: artificial intelligence |
| NHS: National Health Service |
| PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
| XR: extended reality |
Edited by A Mavragani; submitted 05.09.24; peer-reviewed by P Dezyani, ST Arasteh; comments to author 04.10.24; revised version received 10.10.24; accepted 29.10.24; published 19.02.25.
Copyright©Alison Potter, Chris Munsch, Elaine Watson, Emily Hopkins, Sofia Kitromili, Iain Cameron O'Neill, Judy Larbie, Essi Niittymaki, Catriona Ramsay, Joshua Burke, Neil Ralph. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.02.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.