Review
Abstract
Background: The integration of artificial intelligence (AI) systems for automating medical history taking and triage can significantly enhance patient flow in health care systems. Despite the promising performance of numerous AI studies, only a limited number of these systems have been successfully integrated into routine health care practice. To elucidate how AI systems can create value in this context, it is crucial to identify the current state of knowledge, including the readiness of these systems, the facilitators of and barriers to their implementation, and the perspectives of various stakeholders involved in their development and deployment.
Objective: This study aims to map and summarize empirical research on AI systems designed for automating medical history taking and triage in health care settings.
Methods: The study was conducted following the framework proposed by Arksey and O’Malley and adhered to the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) guidelines. A comprehensive search of 5 databases—PubMed, CINAHL, PsycINFO, Scopus, and Web of Science—was performed. A detailed protocol was established before the review to ensure methodological rigor.
Results: A total of 1248 research publications were identified and screened. Of these, 86 (6.89%) met the eligibility criteria. Notably, most (n=63, 73%) studies were published between 2020 and 2022, with a significant concentration on emergency care (n=32, 37%). Other clinical contexts included radiology (n=12, 14%) and primary care (n=6, 7%). Many (n=15, 17%) studies did not specify a clinical context. Most (n=31, 36%) studies used retrospective designs, while others (n=34, 40%) did not specify their methodologies. The predominant type of AI system identified was the hybrid model (n=68, 79%), with forecasting (n=40, 47%) and recognition (n=36, 42%) being the most common tasks performed. While most (n=70, 81%) studies included patient populations, only 1 (1%) study investigated patients’ views on AI-based medical history taking and triage, and 2 (2%) studies considered health care professionals’ perspectives. Furthermore, only 6 (7%) studies validated or demonstrated AI systems in relevant clinical settings through real-time model testing, workflow implementation, clinical outcome evaluation, or integration into practice. Most (n=76, 88%) studies were concerned with the prototyping, development, or validation of AI systems. In total, 4 (5%) studies were reviews of several empirical studies conducted in different clinical settings. The facilitators and barriers to AI system implementation were categorized into 4 themes: technical aspects, contextual and cultural considerations, end-user engagement, and evaluation processes.
Conclusions: This review highlights current trends, stakeholder perspectives, stages of innovation development, and key influencing factors related to implementing AI systems in health care. The identified literature gaps regarding stakeholder perspectives and the limited research on AI systems for automating medical history taking and triage indicate significant opportunities for further investigation and development in this evolving field.
doi:10.2196/53741
Keywords
Introduction
Background
In health care, efficient management of the flow of patients through the health care system is important to ensure that care is accessible and appropriate, given the needs of patients []. With the introduction of artificial intelligence (AI) systems, there has been growing interest in using these technologies to improve patient flow []. Research in this area remains in its early stages, constituting a disjointed field []. In addition, AI systems hold great promise to make patient flows more efficient at the admission of patients by improving triage [,].
Traditionally, triage involves a health care professional taking the patient’s medical history to systematically decide the optimal prioritization and assess the appropriate treatment for the patient. There are different types of triage, each with its distinctive elements, such as inpatient triage, emergency department triage, incident triage, military triage, and mass casualty triage, all of which possess the aforementioned characteristics []. AI systems could potentially increase the efficiency of triage processes by replacing health care professionals in taking the patient’s medical history and directing patients to the most appropriate treatment. This could effectively reduce health care professionals’ workload and allow for a more optimal allocation of their time [,]. Despite the high hopes and AI systems showing good performance in studies, only a small number have been implemented in operational health care systems [,].
To fully understand how AI systems for automating medical history taking and triage could be used to create value in the health care context, it is important to understand the entire development and implementation process, including data collection, algorithm development, model validation, and real-world implementation []. Each of these steps presents unique challenges and considerations that must be carefully addressed to ensure successful integration into health care. It is essential to determine at which stage of the innovation development process the AI systems are halted or hindered before they are integrated into health care practices [,]. Many technical and nontechnical factors affect the implementation process. Technical challenges, such as data quality and interoperability, algorithm accuracy, and system reliability, must be carefully evaluated and addressed [-]. In addition, ethical considerations, regulatory frameworks, organizational barriers, and patient acceptance play critical roles in the implementation and use of these technologies []. By studying these factors, we can identify the barriers that hinder the effective implementation of AI systems and facilitate strategies to overcome them.
However, it is equally important to recognize that the successful implementation of AI systems in health care extends beyond the technical and regulatory aspects. The perspectives of a wide range of stakeholders must be considered as they can offer valuable insights and contribute to the overall effectiveness and acceptance of these systems [,]. Stakeholders include health care providers, patients, decision makers, regulatory bodies, administrators, and other context-based relevant actors. Each stakeholder group brings unique perspectives, priorities, and concerns, which must be integrated into the development and deployment of AI systems for medical history taking and triage [,]. By incorporating such diverse perspectives, we can ensure that these technologies align with the needs, values, and expectations of the stakeholders they seek to serve.
In light of these considerations, it is crucial to conduct a scoping review to map and summarize empirical research on AI systems for automating medical history taking and triage. By systematically reviewing the literature, we can identify the current state of knowledge across various stages of the innovation development process, gain insights into the hindering and facilitating factors affecting the implementation and use of AI systems, and consider the perspectives of different stakeholders. Such a comprehensive review will not only help identify gaps in knowledge but also guide future research efforts and inform evidence-based practice for integrating AI systems for automating medical history taking and triage in health care settings.
Aim and Research Questions
The aim of the scoping review was to map and summarize empirical research on AI systems for automating medical history taking and triage. The following research questions (RQs) guided this review:
- What are the characteristics of research publications on AI systems for automated medical history taking and triage in health care?
- Whose perspective (researcher, health care professional, or patient) of the AI systems is described?
- At which stage of the innovation development process are the AI systems studied?
- What facilitating factors and barriers are considered in relation to the introduction of the AI systems?
Methods
Study Design
The scoping review was designed according to the framework of Arksey and O’Malley [], which has been widely used to explore, summarize, and draw conclusions about the overall state of research on new technology and health care (refer to the studies by Sharma et al [], Gama et al [], and Wikström et al []). The characteristics of included research publications (RQ1) and whose perspective is described in the publications (RQ2) are questions commonly addressed in scoping reviews. To investigate at which stages of the innovation development process the AI systems in the research publications were studied (RQ3), we adopted the Technology Readiness Level (TRL) scale for clinical readiness by Fleuren et al []. The scale is based on the original TRL scale developed by the National Aeronautics and Space Administration to evaluate technological maturity. It measures the clinical applicability of AI (ie, machine learning) systems on a scale of 1 to 9 (1 being the least mature and 9 being the most mature) [,]. To identify and report patterns in facilitating factors and barriers to the introduction of AI systems (RQ4), we used techniques from thematic analysis (ie, we coded the data and generated themes) []. The scoping review was reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) guidelines [] (), and a scoping review protocol was established before conducting the review.
Search Strategy
To cover any literature relevant to the scope of the review, we used 4 key terms: health care, AI systems, medical history-taking, and triage (
Textbox 1). To systematically capture literature related to the key terms and synonyms, they were searched using both index terms (ie, subject headings) and free text, as outlined in . To generate an extensive coverage of relevant studies, as recommended in scoping reviews [], 5 major databases for health care–related research were searched: PubMed, CINAHL, PsycINFO, Scopus, and Web of Science Core Collection. The search was conducted in October 2022. All the databases were searched for studies published between January 2000 and September 2022. The search strings were first piloted in one of the databases (PubMed) and then further developed with the support of a research librarian before systematically searching all 5 databases. Textbox 2 provides an illustration of one of the search strings.
- Healthcare: “...all the organizations, institutions and resources that are devoted to producing health actions” []
- Artificial intelligence (AI) systems: “...a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy.” []
- Medical history-taking: The process of acquiring information from a patient on medical conditions and past treatment [,]
- Triage: The process upon which the optimal prioritization and assessment of the appropriate next step for the patient can be decided []
((“neural networks, computer”[MeSH Terms:noexp] OR “artificial intelligence”[MeSH Terms:noexp] OR “deep learning”[MeSH Terms] OR “supervised machine learning”[MeSH Terms] OR “artificial intelligence”[Title/Abstract] OR “deep learning”[Title/Abstract] OR “supervised machine learning”[Title/Abstract]) AND (“delivery of health care”[MeSH Terms] OR “health care”[Title/Abstract] OR “healthcare”[Title/Abstract] OR “medical”[Title/Abstract] OR “clinical”[Title/Abstract])) AND (“Medical History Taking”[MeSH Terms] OR “anamnes*”[All Fields] OR “Triage”[MeSH Terms])
Eligibility Criteria
The eligibility criteria were set to identify research publications that presented empirical studies or systematically conducted reviews of empirical studies that focused on AI systems for automating medical history taking and triage in health care. A retrieved study was eligible for inclusion if it (1) contained a title and abstract; (2) was written in English; (3) was published between January 1, 2000, and September 30, 2022; (4) was peer reviewed; (5) analyzed empirical data or systematically reviewed empirical data; and focused on (6) health care, (7) AI, and (8) medical history taking or triage (Textbox 3). Before using the eligibility criteria for study selection in all databases, we piloted the criteria for the studies retrieved from one of the databases (PubMed). The criteria were not altered after piloting.
Inclusion criteria
Contained title and abstract: Yes
English language: Yes
Publication date: January 1, 2000, to September 30, 2022
Peer reviewed: Yes
Empirical study: Analyzed empirical data or systematically reviewed articles with empirical data
Focused on health care: Yes
Focused on artificial intelligence: Yes
Focused on medical history taking or triage: Yes
Exclusion criteria
Contained title and abstract: No
English language: No
Publication date: Before December 31, 1999
Peer reviewed: No
Empirical study: Analyzed based on theoretical or hypothetical reasoning
Focused on health care: No
Focused on artificial intelligence: No
Focused on medical history taking or triage: No
Study Selection
First, studies identified through the search strategy were imported into Endnote software (version 20; Clarivate) and exported to the Rayyan web application (Rayyan Systems, Inc) [] for controlled removal of duplicates. After the removal of duplicates, 2 of the authors (ES and HJ) independently screened the retrieved titles and abstracts for checking their eligibility for inclusion. When screening studies for inclusion, the authors checked the retrieved studies for inclusion according to the inclusion criteria in chronological order, that is, if criterion 1 was met, the authors checked if the retrieved study met inclusion criterion 2, and so on. Conference proceedings were excluded because they are often preliminary and based on limited analyses []. Editorials, commentaries, viewpoints, discussion and opinion papers, and book chapters were excluded because they often do not undergo a rigorous and impartial peer-review process. Narrative and integrative reviews were excluded because they did not systematically analyze or review empirical data. Any disagreements during the screening of titles and abstracts were discussed with a third author (JN) until a consensus was reached. Full texts of the studies that met the inclusion criteria after screening titles and abstracts were assessed for eligibility. ES and HJ independently screened the full texts of the studies. Disagreements concerning the full-text screening were discussed among all the authors until a consensus was reached.
Data Extraction
A data extraction template was developed by one of the authors (ES) based on the RQs. The template was piloted with 5 studies before extracting the data from all included research publications. The template was not altered after piloting. To extract data relevant to the RQs, the template contained columns for information on the characteristics of the research publications (RQ1; authors, publication year, country, clinical setting, sample population, study aim, study design, type of AI system, and main task performed by AI system), the perspective described in the research publications (RQ2; ie, the researchers’, health care professionals’, or patients’ perspectives), the TRLs (1-9) of the AI systems in the research (RQ3), and facilitators and barriers to the introduction of the AI systems (RQ4). To identify the type of AI system and the main task performed by the AI system (RQ1), we adopted the operational terms presented in the Organisation for Economic Co-operation and Development Framework for the Classification of AI systems [].
We extracted data in relation to RQs 1 to 3 from all included studies, whereas data in relation to RQ4 were extracted only from the studies deemed to have TRLs equal to 6 or higher (ie, they reported a demonstration of an AI system in a real health care environment) and for literature reviews. In total, 2 (3%) of the 77 studies with TRLs ≥6 [,] and 2 (33%) of the 6 literature reviews did not report the introduction of AI systems in real health care environments [,]. Consequently, data from these studies did not address RQ4.
Data on the characteristics of the research publications (RQ1) were extracted from the entire article; data on the perspective described in the research publications (RQ2) and the TRLs (RQ3) were extracted from the methods, results, and discussion sections of the articles. Finally, data on facilitators and barriers to the introduction of the AI systems (RQ4) were extracted from the results, discussion, and conclusions sections of the research publications.
Data Analysis
Following the framework proposed by Arksey and O’Malley [], we began the analysis with a basic numerical analysis of the extracted data and produced tables to report the distribution of the characteristics of the studies (RQ1), the perspective described in the studies (RQ2), and their TRLs (RQ3). Thereafter, to provide a narrative account of the facilitating factors and barriers to the introduction of the AI systems (RQ4), we coded and organized the extracted data thematically [] This involved an inductive iterative process in which we coded the extracted text on the facilitating factors and barriers and collated the codes into potential themes, which we then revised and refined. ES and HJ independently coded the data, compiled all the codes, and developed the final themes. The thematic analysis was finalized through a group discussion among all 3 authors.
Results
Overview
A total of 2008 records were retrieved from the electronic databases. After removing duplicates (n=760, 37.85%), 1248 records remained and were included in the screening. Of these, 133 (10.66%) records were assessed for eligibility in full text, and 86 (64.7%) were deemed eligible for inclusion. The PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flow diagram gives the details on the identification and screening of studies (
Figure 1) [].
RQ1: Characteristics of the Research Publications
The included studies were published between 2000 and 2022, but most (63/86, 73%) of the studies were published between 2020 and 2022. The remaining studies (16/86, 19%) were published predominantly between 2018 and 2019. Only a small number of studies (7/86, 8%) were published between 2000 and 2017. Most of the studies were from Asia (26/86, 30%), particularly China (8/86, 9%); North America (25/86, 29%), predominantly the United States (23/86, 27%); and Europe (22/86, 26%), primarily the United Kingdom (9/86, 10%). A small number of studies (13/86, 15%) were from Africa (1/86, 1%), Oceania (5/86, 6%), and South America (4/86, 5%) or did not specify the country of origin (3/86, 3%). The most common clinical contexts in focus in the included studies were emergency care (32/86, 37%), followed by radiology (12/86, 14%) and primary care (6/86, 7%). In addition, a large number of studies (15/86, 17%) did not specify the clinical context in focus. The sample population in most studies consisted of patients (70/86, 81%), which included datasets of different types of patient records. In the remaining studies, the sample population was health care professionals (9/86, 10%), citizens (2/86, 2%), or unspecified (5/86, 6%). In these studies, the sample size was small, and there was a higher degree of involvement of the participants in the study. Regarding study design, a large number of studies were retrospective studies (31/86, 36%) or did not specify any specific design (29/86, 34%). Other study designs were, for example, prospective studies (7/86, 8%), different types of pilot studies (4/86, 5%), or reviews (4/86, 5%). The most common type of AI system was hybrid models (68/86, 79%), that is, they combined statistical and symbolic approaches. Most (40/86, 47%) of the AI systems studied were used for forecasting, that is, they predicted future outcomes based on past and present behavior or recognition (36/86, 42%), which involved identifying and categorizing data into specific classifications or performing image segmentation and object recognition. Definitions of the types of AI systems and their main tasks used in this review can be found in the Organisation for Economic Co-operation and Development Framework for the Classification of AI systems []. provides an overview of the characteristics of all the 86 included studies [-,-].
RQ2: Perspectives Described in the Publications (Among Researchers, Health Care Professionals, or Patients)
Although the sample population in a large number of the included studies consisted of patients, these studies did not portray their perspectives. In most (83/86, 97%) of the included studies, researchers’ perspectives were portrayed (ie, researchers’ description and interpretation of the AI system and the results of the particular study. Only 1 (1%) study reflected the patients’ perspectives []. It described patient ratings of the usability and interface of an AI system for medical history taking in an emergency department. The same study also portrayed physicians’ and 10 nurses’ ratings for the same AI system. Overall, patients, physicians, and nurses were strongly positive about the AI system as a support for patient-clinician communication []. Another study described health care professionals’ perspectives []. More specifically, it explored how emergency triage nurses understood, contextualized, and incorporated an AI-based decision support system into their work on triaging patients. Initially, nurses expressed apprehension that the AI system lacked the cultural and contextual understanding necessary for patient triage. However, they later discovered that the system helped them provide safe care []. In total, 2% (2/86) of the studies presented health care professionals’ perspectives [,]. Additional studies were conducted using health care professionals as the sample population. However, these studies focused solely on health are professionals’ decision-making performance in relation to triage with AI systems, without providing insights into their perspectives [,,,,,,]. In addition, 1% (1/86) of the studies portrayed citizens’ (ie, men and women aged 18-80 y) perspectives on an AI system with a conversational user interface for self-guided medical history taking []. The citizens completed a usability test and questionnaire concerning the usability, design, and functionality of the AI system. Overall, the citizens stated that they felt comfortable with the system; however, the authors noted that the level of technical competencies and educational levels were high among the participants, limiting the generalization to patient groups [].
RQ3: Stages of the Innovation Development Process (TRLs)
Most (76/86, 88%) of the included studies did not demonstrate or validate AI systems for medical history taking and triage in clinical environments (ie, they were concerned with clinical problem identification; proposal of a model or solution; or model prototyping, development, or validation. These studies scored ≤5 on the TRL scale for clinical readiness []. A few (6/86, 7%) studies scored ≥6 on the TRL scale (ie, they described real-time testing of AI models, workflow implementation, evaluation of clinical outcomes, or model integration). In total, 5% (4/86) of the studies, which were reviews of several different empirical studies, were not considered applicable to the TRL scale [,,,]. An overview of the included studies and their TRL scores is presented in 
Table 1.
| TRLs | Clinical definitions | Studies, n (%) |
| 1 | Clinical problem identification | 0 (0) |
| 2 | Proposal of model or solution | 0 (0) |
| 3 | Model prototyping | 8 (9) |
| 4 | Model development | 50 (58) |
| 5 | Model validation | 18 (21) |
| 6 | Real-time model testing | 1 (1) |
| 7 | Workflow implementation | 2 (2) |
| 8 | Clinical outcome evaluation | 1 (1) |
| 9 | Model integration | 2 (2) |
No studies were classified as TRL 1 or 2. A small number (8/86, 9%) of studies achieved TRL 3, indicating prototyping or model development of an AI system [,,,,,,,]. Examples of studies in this category include an AI-based field triage tool [], triage of medical referrals [], and a self-diagnosis system for emergency dental care []. A substantial number (50/86, 58%) of studies achieved TRL 4, that is, they demonstrated the potential of AI systems or optimized and validated these using clinical data. Examples of these studies include triage of adult chest radiographs [], triage using digitalized patient histories in primary care [], and triage of patients with COVID-19 under limited health care resources []. Several (18/86, 21%) studies were categorized as TRL 5. This means that they validated AI models using realistic datasets other than the population used to train or test the AI model. This particular type of validation data was either retrospective [] or prospective []. One study attained a TRL of 6; it investigated the testing of an AI model in real time, specifically, an algorithm for automated diabetic retinopathy screening []. In addition, 2% (2/86) of the studies examined AI models integrated into clinical workflows and assessed their outcomes, resulting in an assessment of TRL 7 [,]. The studies had a common theme of emergency care and focused on a symptom-taking tool [] and a rapid and laboratory-free COVID-19 triage [], respectively. One study achieved a score of 8 on the TRL scale, that is, it evaluated the clinical outcomes of an implemented AI model. This study examined a medical history–taking system that generated differential diagnoses in an outpatient department []. In total, 2% (2/86) of the studies focused on the postimplementation phase of AI systems and thus achieved the TRL 9 [,]. One study focused on the impact of cultural embeddedness when implementing an AI system for emergency care triage []. Another study investigated the user demographics and levels of triage acuity of a symptom checker chatbot used within a large integrated health system [].
RQ4: Facilitating Factors and Barriers to the Introduction of the AI Systems
From the 6 retrieved studies that reported the introduction of AI systems for automating medical history taking and triage in health care, we identified 4 themes relating to facilitating factors and barriers to their introduction [,,,,,]. The themes were related to technical aspects, contextual and cultural factors, end users, and evaluation (Table 2). The theme denoted as technical aspects pertained to the AI systems’ design, integration, requirements, and performance. The theme of contextual and cultural factors was associated with the organization and patient populations in which the AI systems were introduced. The theme end users encompassed the viewpoints and experiences of patients and health care professionals with regard to AI systems. The theme evaluation and regulation covered various aspects of evaluating, regulating, and setting guidelines for AI systems.
| Facilitating factors | Barriers | |
| Technical aspects |
|
|
| Context and culture |
|
|
| End users |
|
|
| Evaluation |
|
|
Of the 6 studies, 5 (80%) reported factors related to technical aspects of the introduction of AI systems for automating medical history taking and triage that either facilitated or hindered its introduction into clinical settings [,,,,]. Technical facilitators included a flexible design that allows the AI system to be implemented in different geographical areas []. Furthermore, doctors and nurses perceived that integrating AI systems with electronic health records would enhance their usefulness []. Technical barriers involved the amount of information required by AI systems, which might not be possible to collect in the short time required for the triage process in emergency departments []. An additional issue to consider was the inadequate performance of AI systems. This was due to their lack of cultural understanding of the context and the patient population they served [], limited accuracy, particularly when dealing with patients with complex presentations, such as older adults [], and the absence of optimization for certain patient groups, such as those with neurological symptoms []. The lack of integration with established health care systems constituted an additional barrier associated with the technical attributes of the AI systems [].
A total of 3 (50%) of the 6 studies reported facilitating barriers related to context and culture [,,]. Facilitating factors included greater integration of the AI system into the clinical workflow, as nurses and physicians believed this would enhance its usefulness []. In addition, the knowledge of the organizational context, clinical workload and culture, skills, and recognized behavior change techniques should be considered when implementing AI systems. Contextual and cultural barriers included delays in the start-up of AI systems due to strategic decision-making processes [] and the use of these systems in settings with highly mixed patient populations, such as emergency departments. This posed a challenge for AI systems, as reported by physicians, ultimately impacting their usefulness [].
Of the 6 studies, 5 (80%) reported facilitating factors and barriers in relation to end users [,,,,]. The consideration of patients’ perspectives in adapting the AI system for accessibility and usefulness as well as innovative approaches to promote use among health care professionals, such as having superusers [], were identified as facilitating factors. Other factors that facilitated the introduction of AI systems for automating medical history taking and triage included providing adequate training for health care professionals, as this is crucial given the systems’ dependence on their users, and enabling them to effectively use the system [,,] while also supporting patients through familiar health delivery mechanisms []. In addition, effective communication regarding the potential benefits of AI systems in enhancing patient care, resource use [], and patient outcomes [] was found to facilitate their adoption. Notably, the significance of the latter aspect was emphasized for its widespread uptake []. End-user barriers to the uptake of the AI system included physicians’ nonacceptance of AI-generated diagnosis [], negative receptivity toward rapid implementation [], and health care professionals constrained by limited time to use the AI system []. In addition, a lack of comprehension among certain patient groups (such as those with neurological symptoms), inadequate eHealth literacy [,], linguistic deficiencies in patients (specifically, in expressing complaints in a way that the AI system can understand) [], and limited opportunities for patients to acquire the necessary skills to use AI tools, due to limited occasions for using it [,], were additional barriers linked to end users.
Facilitating factors and barriers related to evaluation were reported in 50% (3/6) of the studies [,,]. Factors suggested to facilitate the introduction of AI systems were the formulation and implementation of evaluation guidelines and regulations to ease the development and use of AI systems []. Initial and ongoing assessments of AI systems and evaluations focused on users were facilitating factors for assessing the safety and effectiveness of introducing AI systems in complex environments []. Barriers related to evaluation included the current lack of evaluation regulations and guidelines [] as well as the validation of AI systems [].
Discussion
Principal Findings
The goal of this study was to analyze and compile existing empirical studies on AI systems used for automating medical history taking and triage in health care. This review shows that research on AI systems to automate medical history taking and triage in clinical settings is still at an early stage, articulating a large gap to be bridged between model development and the bedside, similar to what has been highlighted in previous research on AI in health care [,,,,]. There is a paucity of studies demonstrating AI systems to automate medical history taking and triage in clinical settings, representing the perspectives of patients or health care professionals, and providing information on facilitators and barriers to the introduction of these AI systems. The findings on the state of research in this area should be understood in light of some of the characteristics of the included research studies, indicating that research on AI systems to automate medical history taking and triage in health care remains relatively new and underdeveloped, and is mainly conducted in a nonclinical research environment of prototyping, development, and validation. While it is crucial to establish a solid research foundation that tests and validates AI systems before their implementation in clinical settings, which could potentially impact care quality, safety, clinical outcomes, and working conditions for health care professionals, it is also essential to conduct research that puts these systems into practical use to showcase their potential []. However, the current level of research on AI systems for automating medical history taking and triage remains inadequate.
Characteristics of the Research Publications
The study revealed a notable surge in research on AI systems for automating medical history taking and triage over the past 3 years, indicating a growing interest in leveraging AI technologies in relation to managing patient flows in health care settings. Geographically, research was predominantly concentrated in Asia, North America, and Europe, aligning with regions known for both advanced health care infrastructure and research capabilities. The prevalence of studies focusing on emergency and primary care underscores the potential of AI systems to automate medical history taking and triage in first-line care, which globally faces challenges in meeting the needs of the people it is intended to serve [,]. Many (12/86, 14%) studies also focused on radiology, illustrating the potential of AI systems to assist radiologists in image triage, a finding that is not surprising, given the progress of AI in image recognition tasks []. However, a significant (15/86, 17%) proportion of studies did not specify the clinical context, suggesting the need for more context-specific investigations to better understand the applicability of AI systems for medical history taking and triage. A large number (31/86, 36%) of studies also had a retrospective design, which, together with the large group of non-context–specific studies, may indicate a trend toward horizontal research rather than toward the implementation of AI systems for automating medical history taking and triage in clinical settings. As previous studies have suggested, this type of research and development may be partly due to the challenges of conducting studies that actually implement AI in real clinical settings []. The most common type of AI system was hybrid models that performed tasks related to forecasting or recognition. These findings resonate with previous research on the ways in which machine learning has been used to predict the flow of patients through health care systems []. Most (70/86, 81%) studies used patient datasets as their sample population, comprising patient records that were used to test, validate, and prototype AI models. This also meant that they focused on the technical capabilities of AI systems to automate medical history taking and triage based on a specific set of patient records, rather than their usability for health care professionals or patients.
Perspectives Described in the Publications (Among Researchers, Health Care Professionals, or Patients)
While the promise of AI systems for automating medical history taking and triage to transform health care is well recognized, this study highlights an imbalance in stakeholder perspectives in the current research. Researchers’ viewpoints overwhelmingly dominated the literature. Despite providing insights into the description and interpretation of AI systems designed for medical history taking and triage, there has been limited exploration of the critical viewpoints stemming from patients and health care professionals regarding their practical application in clinical settings. Previous research has argued that the imbalance between different stakeholder groups’ perspectives in research on AI in health care should be addressed to meet the distinct needs of different groups []. Furthermore, the importance of the acceptability of interventions in routine health care practice is widely recognized in the fields of intervention, innovation, implementation, and improvement science []. Therefore, it is essential to consider the views of patients and health care professionals regarding the use of AI systems to automate medical history taking and triage. This will help to ensure the successful implementation of these systems. The positive feedback from patients on usability and support in patient-clinician communication from AI systems for automating medical history taking and triage emphasizes the potential benefits of involving patients in the design and evaluation of such systems. Previous research investigating a rule-based, automated medical history–taking app [] even deemed patient involvement in the development process of these types of tools necessary. One additional study delved into health care professionals’ perspectives, focusing on concerns and adjustments when integrating AI systems for automating medical history taking and triage into the triaging processes. The study found that nurses were uneasy with the AI system’s acontextual nature []. To increase trust in AI systems, researchers suggest that focusing solely on technical aspects, such as objectivity, efficiency, and accuracy, is insufficient. AI systems must also be aligned with the values and principles of the clinical environment in which they are used. This is crucial because people are more inclined to trust and embrace AI systems when they perceive them as aligning with the inherent value, significance, and utility of the context in which they are deployed [].
Stages of the Innovation Development Process: TRLs
The application of the TRL framework [] provided a structured understanding of AI systems for automating medical history taking and triage development stages. Most (76/86, 88%) studies focused on lower TRLs, which aligns with the prevalence of studies that emphasize developmental work and potential demonstrations in the broader field of applying AI in health care []. This disparity underscores a critical gap in our understanding. This highlights the pressing need for more comprehensive investigations into the practical implementation and validation of AI systems in real health care settings. These higher TRL evaluations are pivotal in ensuring that AI technologies not only show promise in controlled environments but also prove their effectiveness and reliability when deployed in complex and dynamic safety-critical settings in health care practice []. Such research can contribute significantly to building trust and confidence in AI systems within the medical domain and promoting their implementation and use in practice.
Facilitating Factors and Barriers to the Introduction of AI Systems
The introduction of AI systems for automating medical history taking and triage is a complex process influenced by several factors, both hindering and facilitating its implementation. One set of barriers described lies in technical aspects, encompassing system performance and integration. The effectiveness and seamless integration of these AI systems into existing health care workflows will significantly impact their adoption and use. In addition to the technical issues, contextual and cultural factors are also important. The organizational context within health care settings as well as the diversity of patient populations shaped adoption. Different health care institutions, stakeholder roles, and patient perspectives may have unique requirements and expectations regarding AI integration in health care [,]. Recognizing the importance of stakeholder and patient perspectives highlights the need for tailoring AI systems to cater to context- and situation-specific needs and concerns [,,]. Regulatory and evaluative considerations add another layer of complexity to the integration process. The development of guidelines and user-focused evaluations is essential to ensure the safety, efficacy, and ethical use of AI systems in health care [-]. These considerations underscore the need for a well-structured and carefully monitored approach to the implementation of AI in medical history taking and triage.
Limitations
The study’s strengths include a thorough review of records with rigorous screening. The search strategy covered 5 electronic databases, ensuring comprehensiveness. However, it missed the opportunity to explore gray literature, which could have revealed additional cases that may present ongoing or completed work not yet in the research literature. We decided not to include conference proceedings in this review due to their often preliminary nature and limited analysis []. Upon examining the conference papers identified in the search, it became evident that they represented a diverse array of formats and scopes and varied significantly in how they were quality assessed and made accessible. This heterogeneity posed a challenge for their integration into a pooled analysis alongside more uniform and rigorously peer-reviewed papers. Nevertheless, in instances where the evidence is limited or contradictory, which is often the case in research on AI, the incorporation of conference proceedings can prove advantageous for systematic reviews []. Therefore, it is recommended that future reviews in this domain of research consider the incorporation of high-quality conference proceedings. We did not manually search the reference lists of the included studies to add more content to the search because an as stringent as possible review of the current literature was sought. Identifying studies focusing on AI systems can be a complex task due to the lack of consensus on their definition. To ensure clarity for the reader, we have aimed to be transparent about how we define AI systems in this review. To maintain clarity, we based our inclusion on how the original authors described their systems as AI, relying on their accuracy. Because AI definitions vary, this introduces subjectivity, potentially affecting the consistency and comparability of the included articles. We suggest that future research consider this challenge in greater depth. We used a search strategy similar to that used in previous scoping reviews to identify AI systems [,]. The studies reviewed in this study concentrate on medical history taking and triage, both crucial processes for managing patient flow by assessing a patient’s condition and referring them to the appropriate care. However, it is important to note that this approach may have some limitations. Therefore, we recommend that future research conduct separate literature reviews for each of these patient flow management processes. The eligibility criteria for study publication dates were set to identify any early studies on the topic of this review, despite AI applications in health care being a more recent phenomenon []. To ensure consistency, all full-text screening decisions were confirmed in pairs and discussed at biweekly meetings. This approach encouraged open discussions to address doubts, exclusion criteria, and differing interpretations. Conflicting interpretations and uncertainties were resolved through discussion.
Conclusions
This scoping review contributes to the understanding of AI system implementation in health care by providing insights into current trends, stakeholder perspectives, innovation development stages, and influencing factors. It highlights gaps in patient perspectives and the underrepresentation of higher TRLs, indicating opportunities for further research. To realize the full potential of AI systems for automating medical history taking and triage in health care settings, it is crucial to integrate patients’ voices into AI development, enhance real-world validation, and address the multifaceted challenges highlighted by end users and regulations. As health care systems evolve, embracing AI technologies while ensuring alignment with clinical and patient needs remains a key challenge for future research and implementation efforts. These multifaceted factors collectively shape the landscape of AI system adoption in the context of automating medical history taking and triage. Therefore, a holistic and inclusive approach is necessary to ensure the effectiveness and ethical use of these types of AI systems.
Acknowledgments
This project has received funding from the Swedish Innovation Agency and the Knowledge Foundation.
Data Availability
All data generated or analyzed during this study are included in this published article and its supplementary information files.
Authors' Contributions
Study conception and design were formulated by ES and JN. ES and HJ were responsible for data collection. ES, HJ, and JN contributed to the analysis and interpretation of results. All authors collaborated in the preparation of the draft and approved the final version of the paper.
Conflicts of Interest
None declared.
Multimedia Appendix 1
PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.
PDF File (Adobe PDF File), 530 KBReferences
- Kollberg B, Dahlgaard JJ, Brehmer PO. Measuring lean initiatives in health care services: issues and findings. Int J Product Perform Manag. Dec 12, 2006;56(1):7-24. [CrossRef]
- Ellahham S, Ellahham N. Use of artificial intelligence for improving patient flow and healthcare delivery. J Comput Sci Syst Biol. 2019;12(3):1-6. [FREE Full text]
- El-Bouri R, Taylor T, Youssef A, Zhu T, Clifton DA. Machine learning in patient flow: a review. Prog Biomed Eng (Bristol). Apr 22, 2021;3(2):022002. [FREE Full text] [CrossRef] [Medline]
- Cecula P, Yu J, Dawoodbhoy FM, Delaney J, Tan J, Peacock I, et al. Applications of artificial intelligence to improve patient flow on mental health inpatient units - narrative literature review. Heliyon. Apr 2021;7(4):e06626. [FREE Full text] [CrossRef] [Medline]
- Dawoodbhoy FM, Delaney J, Cecula P, Yu J, Peacock I, Tan J, et al. AI in patient flow: applications of artificial intelligence to improve patient flow in NHS acute mental health inpatient units. Heliyon. May 2021;7(5):e06993. [FREE Full text] [CrossRef] [Medline]
- Iserson KV, Moskop JC. Triage in medicine, part I: concept, history, and types. Ann Emerg Med. Mar 2007;49(3):275-281. [CrossRef] [Medline]
- Fleuren LM, Thoral P, Shillan D, Ercole A, Elbers PW, Right Data Right Now Collaborators. Machine learning in intensive care medicine: ready for take-off? Intensive Care Med. Jul 2020;46(7):1486-1488. [CrossRef] [Medline]
- Naseri Jahfari A, Tax D, Reinders M, van der Bilt I. Machine learning for cardiovascular outcomes from wearable data: systematic review from a technology readiness level point of view. JMIR Med Inform. Jan 19, 2022;10(1):e29434. [FREE Full text] [CrossRef] [Medline]
- Petersson L, Larsson I, Nygren JM, Nilsen P, Neher M, Reed JE, et al. Challenges to implementing artificial intelligence in healthcare: a qualitative interview study with healthcare leaders in Sweden. BMC Health Serv Res. Jul 01, 2022;22(1):850. [FREE Full text] [CrossRef] [Medline]
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. Jan 2019;25(1):44-56. [CrossRef] [Medline]
- Ilicki J. Challenges in evaluating the accuracy of AI-containing digital triage systems: a systematic review. PLoS One. Dec 27, 2022;17(12):e0279636. [FREE Full text] [CrossRef] [Medline]
- Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. Nov 01, 2017;19(11):e367. [FREE Full text] [CrossRef] [Medline]
- Kueper JK, Terry A, Bahniwal R, Meredith L, Beleno R, Brown JB, et al. Connecting artificial intelligence and primary care challenges: findings from a multi stakeholder collaborative consultation. BMJ Health Care Inform. Jan 2022;29(1):e100493. [FREE Full text] [CrossRef] [Medline]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. Feb 2005;8(1):19-32. [CrossRef]
- Sharma M, Savage C, Nair M, Larsson I, Svedberg P, Nygren JM. Artificial intelligence applications in health care practice: scoping review. J Med Internet Res. Oct 05, 2022;24(10):e40238. [FREE Full text] [CrossRef] [Medline]
- Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation frameworks for artificial intelligence translation into health care practice: scoping review. J Med Internet Res. Jan 27, 2022;24(1):e32215. [FREE Full text] [CrossRef] [Medline]
- Wikström L, Schildmeijer K, Nylander EM, Eriksson K. Patients' and providers' perspectives on e-health applications designed for self-care in association with surgery - a scoping review. BMC Health Serv Res. Mar 23, 2022;22(1):386. [FREE Full text] [CrossRef] [Medline]
- Héder M. From NASA to EU: the evolution of the TRL scale in public sector innovation. Innov J. Aug 2017;22(2):1. [FREE Full text]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [FREE Full text] [CrossRef] [Medline]
- The world health report 2000: health systems: improving performance. World Health Organization. Jun 14, 2000. URL: https://www.who.int/publications/i/item/924156198X [accessed 2022-10-01]
- Recommendation of the council on artificial intelligence. Organisation for Economic Co-Operation and Development. 2019. URL: https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449#mainText [accessed 2023-01-01]
- Nichol JR, Sundjaja JH, Nelson G. Medical history. In: StatPearls. Treasure Island, FL. StatPearls Publishing; 2024.
- Svensk MeSH. Karolinska Institutet. URL: https://mesh.kib.ki.se/term/D008487/medical-history-taking [accessed 2022-10-01]
- Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. Dec 05, 2016;5(1):210. [FREE Full text] [CrossRef] [Medline]
- Scherer RW, Saldanha IJ. How should systematic reviewers handle conference abstracts? A view from the trenches. Syst Rev. Nov 07, 2019;8(1):264. [FREE Full text] [CrossRef] [Medline]
- OECD framework for the classification of AI systems. Organisation for Economic Co-Operation and Development. URL: https://www.oecd.org/en/publications/oecd-framework-for-the-classification-of-ai-systems_cb6d9eca-en.html [accessed 2023-03-03]
- Heydon P, Egan C, Bolter L, Chambers R, Anderson J, Aldington S, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol. May 2021;105(5):723-728. [FREE Full text] [CrossRef] [Medline]
- Soltan AA, Yang J, Pattanshetty R, Novak A, Yang Y, Rohanian O, et al. Real-world evaluation of rapid and laboratory-free COVID-19 triage for emergency care: external validation and pilot deployment of artificial intelligence driven screening. Lancet Digit Health. Apr 2022;4(4):e266-e278. [FREE Full text] [CrossRef] [Medline]
- Joudar SS, Albahri AS, Hamid RA. Triage and priority-based healthcare diagnosis using artificial intelligence for autism spectrum disorder and gene contribution: a systematic review. Comput Biol Med. Jul 2022;146:105553. [CrossRef] [Medline]
- Sánchez-Salmerón R, Gómez-Urquiza JL, Albendín-García L, Correa-Rodríguez M, Martos-Cabrera MB, Velando-Soriano A, et al. Machine learning methods applied to triage in emergency services: a systematic review. Int Emerg Nurs. Jan 2022;60:101109. [CrossRef] [Medline]
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [FREE Full text] [CrossRef] [Medline]
- Abe D, Inaji M, Hase T, Takahashi S, Sakai R, Ayabe F, et al. A prehospital triage system to detect traumatic intracranial hemorrhage using machine learning algorithms. JAMA Netw Open. Jun 01, 2022;5(6):e2216393. [FREE Full text] [CrossRef] [Medline]
- Ahmad R, Xie L, Pyle M, Suarez MF, Broger T, Steinberg D, et al. A rapid triage test for active pulmonary tuberculosis in adult patients with persistent cough. Sci Transl Med. Oct 23, 2019;11(515):eaaw8287. [CrossRef] [Medline]
- Ahmed MM, Sayed AM, Khafagy GM, El Sayed IT, Elkholy YS, Fares AH, et al. Accuracy of the traditional COVID-19 phone triaging system and phone triage-driven deep learning model. J Prim Care Community Health. 2022;13:21501319221113544. [FREE Full text] [CrossRef] [Medline]
- Annarumma M, Withey SJ, Bakewell RJ, Pesce E, Goh V, Montana G. Automated triaging of adult chest radiographs with deep artificial neural networks. Radiology. Apr 2019;291(1):196-202. [FREE Full text] [CrossRef] [Medline]
- Ayling RM, Wong A, Cotter F. Use of ColonFlag score for prioritisation of endoscopy in colorectal cancer. BMJ Open Gastroenterol. Jun 03, 2021;8(1):e000639. [FREE Full text] [CrossRef] [Medline]
- Azeez D, Ali MA, Gan KB, Saiboon I. Comparison of adaptive neuro-fuzzy inference system and artificial neutral networks model to categorize patients in the emergency department. Springerplus. 2013;2:416. [FREE Full text] [CrossRef] [Medline]
- Azeez D, Gan KB, Mohd Ali MA, Ismail MS. Secondary triage classification using an ensemble random forest technique. Technol Health Care. 2015;23(4):419-428. [CrossRef] [Medline]
- Baker A, Perov Y, Middleton K, Baxter J, Mullarkey D, Sangar D, et al. A comparison of artificial intelligence and human doctors for the purpose of triage and diagnosis. Front Artif Intell. 2020;3:543405. [FREE Full text] [CrossRef] [Medline]
- Chang YH, Shih HM, Wu JE, Huang FW, Chen WK, Chen DM, et al. Machine learning-based triage to identify low-severity patients with a short discharge length of stay in emergency department. BMC Emerg Med. May 20, 2022;22(1):88. [FREE Full text] [CrossRef] [Medline]
- Choi SW, Ko T, Hong KJ, Kim KH. Machine learning-based prediction of Korean triage and acuity scale level in emergency department patients. Healthc Inform Res. Oct 2019;25(4):305-312. [FREE Full text] [CrossRef] [Medline]
- Dehghani Soufi M, Samad-Soltani T, Shams Vahdati S, Rezaei-Hachesu P. Decision support system for triage management: a hybrid approach using rule-based reasoning and fuzzy logic. Int J Med Inform. Jun 2018;114:35-44. [CrossRef] [Medline]
- Delshad S, Dontaraju VS, Chengat V. Artificial intelligence-based application provides accurate medical triage advice when compared to consensus decisions of healthcare providers. Cureus. Aug 2021;13(8):e16956. [FREE Full text] [CrossRef] [Medline]
- Dembrower K, Wåhlin E, Liu Y, Salim M, Smith K, Lindholm P, et al. Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digit Health. Sep 2020;2(9):e468-e474. [FREE Full text] [CrossRef] [Medline]
- Denecke K, Hochreutener SL, Pöpel A, May R. Self-anamnesis with a conversational user interface: concept and usability study. Methods Inf Med. Nov 2018;57(5-06):243-252. [CrossRef] [Medline]
- Duceau B, Alsac JM, Bellenfant F, Mailloux A, Champigneulle B, Favé G, et al. Prehospital triage of acute aortic syndrome using a machine learning algorithm. Br J Surg. Jul 2020;107(8):995-1003. [CrossRef] [Medline]
- Dyer T, Chawda S, Alkilani R, Morgan TN, Hughes M, Rasalingham S. Validation of an artificial intelligence solution for acute triage and rule-out normal of non-contrast CT head scans. Neuroradiology. Apr 2022;64(4):735-743. [CrossRef] [Medline]
- Entezarjou A, Bonamy AK, Benjaminsson S, Herman P, Midlöv P. Human- versus machine learning-based triage using digitalized patient histories in primary care: comparative study. JMIR Med Inform. Sep 03, 2020;8(9):e18930. [FREE Full text] [CrossRef] [Medline]
- Fernandes M, Mendes R, Vieira SM, Leite F, Palos C, Johnson A, et al. Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing. PLoS One. Apr 2, 2020;15(4):e0230876. [FREE Full text] [CrossRef] [Medline]
- Fernandes M, Mendes R, Vieira SM, Leite F, Palos C, Johnson A, et al. Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing. PLoS One. Mar 3, 2020;15(3):e0229331. [FREE Full text] [CrossRef] [Medline]
- Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JM. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med. Jan 2020;102:101762. [CrossRef] [Medline]
- Gao Z, Qi X, Zhang X, Gao X, He X, Guo S, et al. Developing and validating an emergency triage model using machine learning algorithms with medical big data. Risk Manag Healthc Policy. 2022;15:1545-1551. [FREE Full text] [CrossRef] [Medline]
- Giavina-Bianchi M, Cordioli E, Dos Santos AP. Accuracy of deep neural network in triaging common skin diseases of primary care attention. Front Med (Lausanne). 2021;8:670300. [FREE Full text] [CrossRef] [Medline]
- Goncharov M, Pisov M, Shevtsov A, Shirokikh B, Kurmukov A, Blokhin I, et al. CT-based COVID-19 triage: deep multitask learning improves joint identification and severity quantification. Med Image Anal. Jul 2021;71:102054. [FREE Full text] [CrossRef] [Medline]
- Goto T, Camargo CAJ, Faridi MK, Freishtat RJ, Hasegawa K. Machine learning-based prediction of clinical outcomes for children during emergency department triage. JAMA Netw Open. Jan 04, 2019;2(1):e186937. [FREE Full text] [CrossRef] [Medline]
- Gottliebsen K, Petersson G. Limited evidence of benefits of patient operated intelligent primary care triage tools: findings of a literature review. BMJ Health Care Inform. May 2020;27(1):e100114. [FREE Full text] [CrossRef] [Medline]
- Gross DP, Steenstra IA, Shaw W, Yousefi P, Bellinger C, Zaïane O. Validity of the work assessment triage tool for selecting rehabilitation interventions for workers' compensation claimants with musculoskeletal conditions. J Occup Rehabil. Sep 2020;30(3):318-330. [CrossRef] [Medline]
- Harada Y, Katsukura S, Kawamura R, Shimizu T. Efficacy of artificial-intelligence-driven differential-diagnosis list on the diagnostic accuracy of physicians: an open-label randomized controlled study. Int J Environ Res Public Health. Feb 21, 2021;18(4):2086. [FREE Full text] [CrossRef] [Medline]
- Hwang S, Lee B. Machine learning-based prediction of critical illness in children visiting the emergency department. PLoS One. Feb 17, 2022;17(2):e0264184. [FREE Full text] [CrossRef] [Medline]
- Ivanov O, Wolf L, Brecher D, Lewis E, Masek K, Montgomery K, et al. Improving ED emergency severity index acuity assignment using machine learning and clinical natural language processing. J Emerg Nurs. Mar 2021;47(2):265-78.e7. [FREE Full text] [CrossRef] [Medline]
- Jiang H, Mao H, Lu H, Lin P, Garry W, Lu H, et al. Machine learning-based models to support decision-making in emergency department triage for patients with suspected cardiovascular disease. Int J Med Inform. Jan 2021;145:104326. [CrossRef] [Medline]
- Jordan M, Hauser J, Cota S, Li H, Wolf L. The impact of cultural embeddedness on the implementation of an artificial intelligence program at triage: a qualitative study. J Transcult Nurs. Jan 2023;34(1):32-39. [CrossRef] [Medline]
- Joseph JW, Leventhal EL, Grossestreuer AV, Wong ML, Joseph LJ, Nathanson LA, et al. Deep-learning approaches to identify critically ill patients at emergency department triage using limited information. J Am Coll Emerg Physicians Open. Oct 2020;1(5):773-781. [FREE Full text] [CrossRef] [Medline]
- Kawamura R, Harada Y, Sugimoto S, Nagase Y, Katsukura S, Shimizu T. Incidence of diagnostic errors among unexpectedly hospitalized patients using an automated medical history-taking system with a differential diagnosis generator: retrospective observational study. JMIR Med Inform. Jan 27, 2022;10(1):e35225. [FREE Full text] [CrossRef] [Medline]
- Kerr E, McGinnity TM, Coleman S, Shepherd A. Human vital sign determination using tactile sensing and fuzzy triage system. Expert Syst Appl. Aug 2021;175:114781. [CrossRef]
- Kim CK, Choi JW, Jiao Z, Wang D, Wu J, Yi TY, et al. An automated COVID-19 triage pipeline using artificial intelligence based on chest radiographs and clinical data. NPJ Digit Med. Jan 14, 2022;5(1):5. [FREE Full text] [CrossRef] [Medline]
- Kim D, You S, So S, Lee J, Yook S, Jang DP, et al. A data-driven artificial intelligence model for remote triage in the prehospital environment. PLoS One. 2018;13(10):e0206006. [FREE Full text] [CrossRef] [Medline]
- Kim J, Lim H, Ahn JH, Lee KH, Lee KS, Koo KC. Optimal triage for COVID-19 patients under limited health care resources with a parsimonious machine learning prediction model and threshold optimization using discrete-event simulation: development study. JMIR Med Inform. Nov 02, 2021;9(11):e32726. [FREE Full text] [CrossRef] [Medline]
- Knitza J, Janousek L, Kluge F, von der Decken CB, Kleinert S, Vorbrüggen W, et al. Machine learning-based improvement of an online rheumatology referral and triage system. Front Med (Lausanne). 2022;9:954056. [FREE Full text] [CrossRef] [Medline]
- Kwon JM, Lee Y, Lee Y, Lee S, Park H, Park J. Validation of deep-learning-based triage and acuity score using a large national dataset. PLoS One. 2018;13(10):e0205836. [FREE Full text] [CrossRef] [Medline]
- Kyono T, Gilbert FJ, Schaar MV. Triage of 2D mammographic images using multi-view multi-task convolutional neural networks. ACM Trans Comput Healthcare. Jul 15, 2021;2(3):1-24. [FREE Full text] [CrossRef]
- Larsson A, Berg J, Gellerfors M, Gerdin Wärnberg M. The advanced machine learner XGBoost did not reduce prehospital trauma mistriage compared with logistic regression: a simulation study. BMC Med Inform Decis Mak. Jun 21, 2021;21(1):192. [FREE Full text] [CrossRef] [Medline]
- Levin S, Toerper M, Hamrock E, Hinson JS, Barnes S, Gardner H, et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med. May 2018;71(5):565-74.e2. [CrossRef] [Medline]
- Libório MP, Kritski A, Almeida IN, Miranda PF, Mesquita JR, Mota RM, et al. Impact of a computer system as a triage tool in the management of pulmonary tuberculosis in a HIV reference center in Brazil. Rev Soc Bras Med Trop. 2022;55:e0451. [FREE Full text] [CrossRef] [Medline]
- Liu Y, Gao J, Liu J, Walline JH, Liu X, Zhang T, et al. Development and validation of a practical machine-learning triage algorithm for the detection of patients in need of critical care in the emergency department. Sci Rep. Dec 15, 2021;11(1):24044. [FREE Full text] [CrossRef] [Medline]
- Love SM, Berg WA, Podilchuk C, López Aldrete AL, Gaxiola Mascareño AP, Pathicherikollamparambil K, et al. Palpable breast lump triage by minimally trained operators in Mexico using computer-assisted diagnosis and low-cost ultrasound. J Glob Oncol. Dec 2018;4:1-9. [CrossRef]
- Lowres N, Duckworth A, Redfern J, Thiagalingam A, Chow CK. Use of a machine learning program to correctly triage incoming text messaging replies from a cardiovascular text-based secondary prevention program: feasibility study. JMIR Mhealth Uhealth. Jun 16, 2020;8(6):e19200. [FREE Full text] [CrossRef] [Medline]
- Majidian M, Tejani I, Jarmain T, Kellett L, Moy R. Artificial intelligence in the evaluation of telemedicine dermatology patients. J Drugs Dermatol. Feb 01, 2022;21(2):191-194. [CrossRef] [Medline]
- Morrill J, Qirko K, Kelly J, Ambrosy A, Toro B, Smith T, et al. A machine learning methodology for identification and triage of heart failure exacerbations. J Cardiovasc Transl Res. Feb 2022;15(1):103-115. [FREE Full text] [CrossRef] [Medline]
- Morse KE, Ostberg NP, Jones VG, Chan AS. Use characteristics and triage acuity of a digital symptom checker in a large integrated health system: population-based descriptive study. J Med Internet Res. Nov 30, 2020;22(11):e20549. [FREE Full text] [CrossRef] [Medline]
- Nederpelt CJ, Mokhtari AK, Alser O, Tsiligkaridis T, Roberts J, Cha M, et al. Development of a field artificial intelligence triage tool: confidence in the prediction of shock, transfusion, and definitive surgical therapy in patients with truncal gunshot wounds. J Trauma Acute Care Surg. Jun 01, 2021;90(6):1054-1060. [CrossRef] [Medline]
- Nsengiyumva NP, Hussain H, Oxlade O, Majidulla A, Nazish A, Khan AJ, et al. Triage of persons with tuberculosis symptoms using artificial intelligence-based chest radiograph interpretation: a cost-effectiveness analysis. Open Forum Infect Dis. Dec 2021;8(12):ofab567. [FREE Full text] [CrossRef] [Medline]
- Qin ZZ, Ahmed S, Sarker MS, Paul K, Adel AS, Naheyan T, et al. Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms. Lancet Digit Health. Sep 2021;3(9):e543-e554. [FREE Full text] [CrossRef] [Medline]
- Raita Y, Goto T, Faridi MK, Brown DF, Camargo CAJ, Hasegawa K. Emergency department triage prediction of clinical outcomes using machine learning models. Crit Care. Feb 22, 2019;23(1):64. [FREE Full text] [CrossRef] [Medline]
- Sanders NW, Mann NH3. Automated scoring of patient pain drawings using artificial neural networks: efforts toward a low back pain triage application. Comput Biol Med. Sep 2000;30(5):287-298. [CrossRef] [Medline]
- Scheder-Bieschin J, Blümke B, de Buijzer E, Cotte F, Echterdiek F, Nacsa J, et al. Improving emergency department patient-physician conversation through an artificial intelligence symptom-taking tool: mixed methods pilot observational study. JMIR Form Res. Feb 07, 2022;6(2):e28199. [FREE Full text] [CrossRef] [Medline]
- Scheetz LJ, Zhang J, Kolassa JE. Using crash scene variables to predict the need for trauma center care in older persons. Res Nurs Health. Aug 2007;30(4):399-412. [CrossRef] [Medline]
- Senda A, Endo A, Kinoshita T, Otomo Y. Development of practical triage methods for critical trauma patients: machine-learning algorithm for evaluating hybrid operation theatre entry of trauma patients (THETA). Eur J Trauma Emerg Surg. Dec 2022;48(6):4755-4760. [CrossRef] [Medline]
- Shazzadur Rahman AA, Langley I, Galliez R, Kritski A, Tomeny E, Squire SB. Modelling the impact of chest X-ray and alternative triage approaches prior to seeking a tuberculosis diagnosis. BMC Infect Dis. Jan 28, 2019;19(1):93. [FREE Full text] [CrossRef] [Medline]
- Shiraz A, Egawa N, Pelt DM, Crawford R, Nicholas AK, Romashova V, et al. Cervical cell lift: a novel triage method for the spatial mapping and grading of precancerous cervical lesions. EBioMedicine. Aug 2022;82:104157. [FREE Full text] [CrossRef] [Medline]
- Singh VK, Shrivastava U, Bouayad L, Padmanabhan B, Ialynytchev A, Schultz SK. Machine learning for psychiatric patient triaging: an investigation of cascading classifiers. J Am Med Inform Assoc. Nov 01, 2018;25(11):1481-1487. [FREE Full text] [CrossRef] [Medline]
- Smith K, Clark A, Dyson K, Kruger E, Lejmanoski L, Russell A, et al. Guided self diagnosis: an innovative approach to triage for emergency dental care. Aust Dent J. Mar 2006;51(1):11-15. [FREE Full text] [CrossRef] [Medline]
- Spasic I, Button K. Patient triage by topic modeling of referral letters: feasibility study. JMIR Med Inform. Nov 06, 2020;8(11):e21252. [FREE Full text] [CrossRef] [Medline]
- Swaminathan S, Qirko K, Smith T, Corcoran E, Wysham NG, Bazaz G, et al. A machine learning approach to triaging patients with chronic obstructive pulmonary disease. PLoS One. 2017;12(11):e0188532. [FREE Full text] [CrossRef] [Medline]
- Tadesse GA, Javed H, Thanh NL, Thi HD, Tan LV, Thwaites L, et al. Multi-modal diagnosis of infectious diseases in the developing world. IEEE J Biomed Health Inform. Jul 2020;24(7):2131-2141. [CrossRef] [Medline]
- Tan Y, Bacchi S, Casson RJ, Selva D, Chan W. Triaging ophthalmology outpatient referrals with machine learning: a pilot study. Clin Exp Ophthalmol. Mar 2020;48(2):169-173. [CrossRef] [Medline]
- Tao X, Chu X, Guo B, Pan Q, Ji S, Lou W, et al. Scrutinizing high-risk patients from ASC-US cytology via a deep learning model. Cancer Cytopathol. Jun 2022;130(6):407-414. [FREE Full text] [CrossRef] [Medline]
- Tsai DJ, Tsai SH, Chiang HH, Lee CC, Chen SJ. Development and validation of an artificial intelligence electrocardiogram recommendation system in the emergency department. J Pers Med. Apr 27, 2022;12(5):700. [FREE Full text] [CrossRef] [Medline]
- Vaghefi E, Yang S, Xie L, Hill S, Schmiedel O, Murphy R, et al. THEIA™ development, and testing of artificial intelligence-based primary triage of diabetic retinopathy screening images in New Zealand. Diabet Med. Apr 2021;38(4):e14386. [FREE Full text] [CrossRef] [Medline]
- van de Leur RR, Blom LJ, Gavves E, Hof IE, van der Heijden JF, Clappers NC, et al. Automatic triage of 12-lead ECGs using deep convolutional neural networks. J Am Heart Assoc. May 18, 2020;9(10):e015138. [FREE Full text] [CrossRef] [Medline]
- Verburg E, van Gils CH, van der Velden BH, Bakker MF, Pijnappel RM, Veldhuis WB, et al. Deep learning for automated triaging of 4581 breast MRI examinations from the DENSE trial. Radiology. Jan 2022;302(1):29-36. [CrossRef] [Medline]
- Wang C, Feng F. ERNIE based intelligent triage system. J Intell Fuzzy Syst. Aug 10, 2022;43(4):5013-5022. [CrossRef]
- Wang M, Xia C, Huang L, Xu S, Qin C, Liu J, et al. Deep learning-based triage and analysis of lesion burden for COVID-19: a retrospective study with external validation. Lancet Digit Health. Oct 2020;2(10):e506-e515. [CrossRef]
- Wee CK, Zhou X, Sun R, Gururajan R, Tao X, Li Y, et al. Triaging medical referrals based on clinical prioritisation criteria using machine learning techniques. Int J Environ Res Public Health. Jun 16, 2022;19(12):7384. [FREE Full text] [CrossRef] [Medline]
- Wolff P, Ríos SA, Graña M. Setting up standards: a methodological proposal for pediatric Triage machine learning model construction based on clinical outcomes. Expert Syst Appl. Dec 2019;138:112788. [CrossRef]
- Xie F, Ong ME, Liew JN, Tan KB, Ho AF, Nadarajan GD, et al. Development and assessment of an interpretable machine learning triage tool for estimating mortality after emergency admissions. JAMA Netw Open. Aug 02, 2021;4(8):e2118467. [FREE Full text] [CrossRef] [Medline]
- Xiong C, Yang M, Kozar R, Zhang L. Integrating transportation data with emergency medical service records to improve triage decision of high-risk trauma patients. J Transp Health. Sep 2021;22:101106. [FREE Full text] [CrossRef]
- Yala A, Schuster T, Miles R, Barzilay R, Lehman C. A deep learning model to triage screening mammograms: a simulation study. Radiology. Oct 2019;293(1):38-46. [CrossRef] [Medline]
- Yang H, Gonçalves T, Quaresma P, Vieira R, Veladas R, Pinto CS, et al. Clinical trial classification of SNS24 calls with neural networks. Future Internet. Apr 26, 2022;14(5):130. [CrossRef]
- Yao LH, Leung KC, Tsai CL, Huang CH, Fu LC. A novel deep learning-based system for triage in the emergency department using electronic medical records: retrospective cohort study. J Med Internet Res. Dec 27, 2021;23(12):e27008. [FREE Full text] [CrossRef] [Medline]
- Yu JY, Jeong GY, Jeong OS, Chang DK, Cha WC. Machine learning and initial nursing assessment-based triage system for emergency department. Healthc Inform Res. Jan 2020;26(1):13-19. [FREE Full text] [CrossRef] [Medline]
- Zhong Q, Li Z, Wang W, Zhang L, He K. Integrated medical resource consumption stratification in hospitalized patients: an Auto Triage Management model based on accurate risk, cost and length of stay prediction. Sci China Life Sci. May 2022;65(5):988-999. [FREE Full text] [CrossRef] [Medline]
- Zmiri D, Shahar Y, Taieb-Maimon M. Classification of patients by severity grades during triage in the emergency department using data mining methods. J Eval Clin Pract. Apr 2012;18(2):378-388. [CrossRef] [Medline]
- Svedberg P, Reed J, Nilsen P, Barlow J, Macrae C, Nygren J. Toward successful implementation of artificial intelligence in health care practice: protocol for a research program. JMIR Res Protoc. Mar 09, 2022;11(3):e34920. [FREE Full text] [CrossRef] [Medline]
- van de Sande D, van Genderen ME, Huiskens J, Gommers D, van Bommel J. Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unit. Intensive Care Med. Jul 2021;47(7):750-760. [FREE Full text] [CrossRef] [Medline]
- Nilsen P, Reed J, Nair M, Savage C, Macrae C, Barlow J, et al. Realizing the potential of artificial intelligence in healthcare: learning from intervention, innovation, implementation and improvement sciences. Front Health Serv. Sep 15, 2022;2:961475. [FREE Full text] [CrossRef] [Medline]
- Hanson K, Brikci N, Erlangga D, Alebachew A, De Allegri M, Balabanova D, et al. The Lancet Global Health Commission on financing primary health care: putting people at the centre. Lancet Glob Health. May 2022;10(5):e715-e772. [FREE Full text] [CrossRef] [Medline]
- Lindner G, Woitok BK. Emergency department overcrowding : analysis and strategies to manage an international phenomenon. Wien Klin Wochenschr. Mar 2021;133(5-6):229-233. [CrossRef] [Medline]
- Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJ. Artificial intelligence in radiology. Nat Rev Cancer. Aug 2018;18(8):500-510. [FREE Full text] [CrossRef] [Medline]
- Camaradou JC, Hogg HD. Commentary: patient perspectives on artificial intelligence; what have we learned and how should we move forward? Adv Ther. Jun 2023;40(6):2563-2572. [FREE Full text] [CrossRef] [Medline]
- Nilsson E, Sverker A, Bendtsen P, Eldh AC. A human, organization, and technology perspective on patients' experiences of a chat-based and automated medical history-taking service in primary health care: interview study among primary care patients. J Med Internet Res. Oct 18, 2021;23(10):e29868. [FREE Full text] [CrossRef] [Medline]
- Steerling E, Siira E, Nilsen P, Svedberg P, Nygren J. Implementing AI in healthcare-the relevance of trust: a scoping review. Front Health Serv. 2023;3:1211150. [FREE Full text] [CrossRef] [Medline]
- Cummings ML. Rethinking the maturity of artificial intelligence in safety‐critical settings. AI Mag. Mar 2021;42(1):6-15. [FREE Full text] [CrossRef]
- Landers C, Vayena E, Amann J, Blasimme A. Stuck in translation: stakeholder perspectives on impediments to responsible digital health. Front Digit Health. Feb 6, 2023;5:1069410. [FREE Full text] [CrossRef] [Medline]
- Bergquist M, Rolandsson B, Gryska E, Laesser M, Hoefling N, Heckemann R, et al. Trust and stakeholder perspectives on the implementation of AI tools in clinical radiology. Eur Radiol. Jan 2024;34(1):338-347. [FREE Full text] [CrossRef] [Medline]
- Neher M, Petersson L, Nygren JM, Svedberg P, Larsson I, Nilsen P. Innovation in healthcare: leadership perceptions about the innovation characteristics of artificial intelligence-a qualitative interview study with healthcare leaders in Sweden. Implement Sci Commun. Jul 18, 2023;4(1):81. [FREE Full text] [CrossRef] [Medline]
- Katirai A. The ethics of advancing artificial intelligence in healthcare: analyzing ethical considerations for Japan's innovative AI hospital system. Front Public Health. 2023;11:1142062. [FREE Full text] [CrossRef] [Medline]
- Petersson L, Vincent K, Svedberg P, Nygren JM, Larsson I. Ethical perspectives on implementing AI to predict mortality risk in emergency department patients: a qualitative study. Stud Health Technol Inform. May 18, 2023;302:676-677. [CrossRef] [Medline]
- Crossnohere NL, Elsaid M, Paskett J, Bose-Brill S, Bridges JF. Guidelines for artificial intelligence in medicine: literature review and content analysis of frameworks. J Med Internet Res. Aug 25, 2022;24(8):e36823. [FREE Full text] [CrossRef] [Medline]
- Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. Dec 2017;2(4):230-243. [FREE Full text] [CrossRef] [Medline]
Abbreviations
| AI: artificial intelligence |
| PRISMA: Preferred Reporting Items for Systematic reviews and Meta-Analyses |
| PRISMA-ScR: Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews |
| RQ: research question |
| TRL: Technology Readiness Level |
Edited by A Mavragani; submitted 17.10.23; peer-reviewed by L-H Yao, Y Harada, E Nilsson; comments to author 10.02.24; revised version received 15.04.24; accepted 27.12.24; published 06.02.25.
Copyright©Elin Siira, Hanna Johansson, Jens Nygren. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.02.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.