Implementing AI in Hospitals to Achieve a Learning Health System: Systematic Review of Current Enablers and Barriers

Introduction

Background

The growing adoption of electronic medical records (EMRs) in many high-income countries has resulted in improvements in health care delivery through the implementation of clinical decision support systems at the point of care [1]. To meet the ever-accelerating demands for clinical care, various innovative models have been developed to harness the potential of EMR data [2-4]. These new care models aim to enable health care organizations to achieve the quadruple aim of care, which includes enhancing patient experience, advancing providers’ experience, improving the health of the population, and reducing health care costs [5].

Artificial intelligence (AI) holds the potential to improve health system outcomes by enhancing clinical decision support systems [6,7]. AI aims to augment human intelligence through complicated and iterative pattern recognition, generally on large data sets that exceed human abilities [8]. While a large body of academic literature has demonstrated the efficacy of AI models in various health domains, most of these models remain as proof of concept and have never been implemented in real-world workflows [9]. This demonstrates the relatively inconsequential endeavors of many AI studies that fail to produce any meaningful impact in the real world. Even with the substantial investments made by the health industry, the implementation of AI analytics in complex clinical practice is still at an early stage [10]. In a limited number of instances, AI has been successfully implemented, largely for nonclinical uses such as service planning or trained on limited static data sets such as chest x-rays or retinal photography [11]. The factors influencing the success or failure of AI implementations in health are poorly investigated [12]. Understanding these barriers and enablers increases the likelihood of successful implementation of AI for the digital transformation of the health system [13,14], ultimately aiding in achieving the quadruple aim of health care [5].

Toward the Digital Transformation of Health Care

A 3-horizon framework has been previously published to help health systems create an iterative pathway for successful digital health transformation (Figure 1 [15]). Horizon 1 aims to optimize the routine collection of patient data during every interaction with the health system. In horizon 2, the data collected during routine care are leveraged in real or near real time to create analytics. Finally, in horizon 3, the insights from data and digital innovations are collated to develop new models of care. A health care system focused on continuous improvement is referred to as a learning health system (LHS) that uses routinely collected data to monitor and enhance health care outcomes consistently [16]. When health care organizations reach the third horizon, they can leverage data in near real time to create ongoing learning iterations and enhance patient care, leading to the establishment of an LHS [17].

‎

Regarding the 3-horizon model, EMRs are the foundation of horizon 1 (Figure 1). While many health organizations have successfully adopted EMRs into their existing workflows, the transition to horizons 2 and 3 has been challenging for many of these health care facilities [18]. A critical phase in this transition involves moving beyond the capture of EMR data for delivering analytics, including AI, aiming to improve clinical outcomes. There is little published evidence to assist health systems in making this transition [19,20].

Analysis of Prior Work

Before conducting our review, we performed a manual search on Google Scholar using our Medical Subject Heading (MeSH) terms along with the “review” keyword to identify previous review papers that aimed at reviewing studies on the implementation of clinical AI in health care settings. We also included review papers known to our research team. Between 2020 and 2022, we identified 4 reviews that were relevant to the implementation of AI in health care systems [21-24]. Overall, these papers reviewed 189 studies between 2010 and 2022. The characteristics of these reviews, outlined in Table 1, were the year of publication, the targeted care settings, the source of data, the predictive algorithm, and whether the predictive algorithm was implemented.

The prior works identified 20 enablers and 13 barriers to AI implementation in health care across 4 categories: people, process, information, and technology (Multimedia Appendix 1 [21-24]). Overall, the findings derived from these review papers hold significant potential in providing valuable insights for health systems to navigate the path toward digital health transformation. One prevailing shortcoming of these studies is the absence of alignment with evidence-based digital health transformation principles to provide health care organizations with actionable recommendations to enable an LHS [17], therefore limiting their applicability for strategic planning within hospital organizations.

Research Significance and Objectives

Hospitals are intricate hubs within the health care ecosystem, playing a central role in providing comprehensive medical care and acting as crucial pillars supporting the foundations of health care systems worldwide. Understanding the factors influencing the success or failure of AI in hospitals provides valuable insights to optimize the integration of these emerging technologies into hospital facilities. While the previous reviews included all health care settings [21-24], our study only focused on hospital settings. Given the limited instances regarding the implementation of AI in hospital facilities, this study explored the real-world case studies that have practically reported their AI implementation solutions in hospital facilities, aiming to synthesize the evidence of enablers and barriers within their implementation process. In addition to the inclusion of these implementation case studies, we incorporated implementation guidelines as they can potentially assist in the overall understanding of AI implementation in hospitals. This study also focused on aligning the evidence of enablers and barriers within the 3-horizon framework [15], offering a way to establish an empirical infrastructure. As a result, this can enable health care organizations to learn, adapt, and accelerate progress toward an LHS [25].

This review investigated the following research questions (RQs): (1) What enablers and barriers are identified for the successful implementation of AI with EMR data in hospitals? (RQ 1) and (2) How can the identified enablers of and barriers to AI implementation lead to actions that drive the digital transformation of hospitals? (RQ 2).

In addressing these questions, our objectives were to (1) conduct a systematic review of the literature to identify the evidence of enablers of and barriers to the real-world implementation of AI in hospital settings and (2) map the identified enablers and barriers to a 3-horizon framework to enable the successful digital health transformation of hospitals to achieve an LHS.

Methods

Search Strategy

This study followed an extended version of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to outline the review methodology with comprehensive details [26]. PubMed, Scopus, Web of Science, and IEEE Xplore were searched on April 13, 2022. We reviewed prior work to determine potential MeSH keywords relevant to our study [21-24]. A research librarian helped with the definition of the MeSH keywords in PubMed and the translation of that search strategy to all platforms searched. The search strategies were applied across the 4 databases (Multimedia Appendix 2). The MeSH keywords used to search PubMed were as follows: product lifecycle management, artificial intelligence, machine learning, deep learning, natural language processing, neural networks, computer, deep learning, big data, hospital, inpatient, medical, clinic, deploy, integrate, monitor, post prediction, data drift, and regulatory. Using the Boolean operator OR, their synonyms were joined to form search phrases. Combining search phrases using the AND operator produced the final search string. We incorporated the term “data drift” to the title and abstract, and full-text search as it is a prominent concept for the continuous integration of AI. The term “regulatory” was also added to our search criteria because it is a relevant term for the implementation of AI in health care within the domain of software as a medical device. The reference lists of the included studies were examined to ensure that all relevant papers were included.

Eligibility Criteria

The inclusion criteria were articles published from January 1, 2010, to April 13, 2022, that included case studies and guidelines on the implementation of AI analytic tools in hospital settings using EMR data. Given the scarcity of real-world AI tools in hospital settings, especially the scarcity of published case studies of unsuccessful implementations of clinical AI tools, we specifically included case studies that successfully implemented AI within hospitals to understand lessons learned and provide use cases that other jurisdictions may learn from. On the basis of a review of frameworks for AI implementation in health care practice, we defined the term implementation as “an intentional effort designed to change or adapt or uptake interventions into routines” [19]. The term “barrier” was defined as “experiences that impeded, slowed, or made implementations difficult in some way” [20]. In contrast, the term enablers was defined as factors, experiences, or processes that facilitated the implementation process. Studies conducted in community or primary care settings were excluded as our main focus was hospital facilities. Studies that did not use AI models were also excluded. We also eliminated non–English-language and conference articles. Studies that focused on regulatory domains and challenges, opportunities, requirements, and recommendations were also excluded as they did not demonstrate real-world AI implementation. The selection of studies was based on the criteria specified in Textbox 1.

Screening

For the screening and data extraction procedures, the Covidence (Veritas Health Innovation) systematic review software was used [27]. A 2-stage screening process was performed with the involvement of 2 reviewers (AKR and OP). In the initial stage, the reviewers assessed the relevance of titles and abstracts based on the inclusion criteria. Subsequently, in the second stage, the full texts of the included articles were reviewed by AKR and OP independently. Consensus was reached through discussion between the reviewers whenever necessary.

Data Extraction and Synthesis

AKR and OP conducted the procedure of data extraction. The following study characteristics were extracted from all final included studies: country, clinical setting, study type (case study or guideline), and aim of study. With the adoption of EMR as a prerequisite for AI development, our focus was on extracting evidence of enablers and barriers solely within horizons 2 (implementation) and 3 (creating new models of care). In total, 2 reviewers (AKR and OP) independently extracted evidence regarding enablers and barriers (RQ 1), subsequently reaching consensus through weekly discussions and analysis. The extracted data were disseminated among our research team for review and to gather additional feedback.

To address the second RQ (RQ 2), we mapped the findings from previous reviews along with the found factors in this study across horizons 2 and 3 of the digital transformation framework [15]. Following the data extraction phase, 2 reviewers independently mapped the identified enablers and barriers to 4 categories (people, process, information, and technology). During the mapping of a given enabler or barrier, if it was related to the development of AI analytics, it was mapped to horizon 2 considering its relevance across the 4 domains (people, technology, information, and processes). When an enabler or barrier was associated with the postdevelopment phase focusing on establishing new care models, it was mapped to horizon 3. Consensus was reached between AKR and OP through a meeting to finalize the mapping phase.

Quality Assessment

For the included use case studies, we used the Mixed Methods Appraisal Tool (MMAT) [28] to conduct a quality assessment. The choice of the MMAT was suitable as the included use case studies exhibited a range of qualitative, quantitative, and mixed methods designs. For evaluating the methodology of guideline studies, we followed the ADAPTE framework [29]. With 9 modules for guideline development, this framework was designed to streamline and enhance the process of creating guidelines within the health domain. The quality assessment was conducted independently by 2 authors (AKR and OP), and any discrepancies were resolved through a meeting.

Results

Study Selection

The search strategy retrieved 1247 papers from PubMed, Scopus, IEEE Xplore, and Web of Science for analysis, and 67 (5.37%) duplicates were identified and eliminated using the EndNote (Clarivate Analytics) citation manager. After screening titles and abstracts, 92.03% (1086/1180) of the studies were removed as the inclusion criteria were not satisfied. A total of 7.97% (94/1180) of the papers remained for full-text review following title and abstract screening. In total, 48% (45/94) of papers were excluded because AI models were not implemented in clinical care. A total of 19% (18/94) of the studies were excluded because they focused on regulatory domains. In total, 9% (8/94) of the studies were excluded due to being the wrong intervention (eg, studies that did not develop AI models). A total of 3% (3/94) of the studies were found to have a clinical population that did not align with our inclusion criteria (eg, hospitalized patients). One study was not in English and was excluded. In addition, 7 studies were discovered by scanning the reference lists of the included articles. In total, 26 studies were included in this review, comprising 9 (35%) guideline studies and 17 (65%) papers with successful implementation examples (Table 2). Figure 2 presents the PRISMA flow diagram outlining the outcomes of this review.

‎

Study Characteristics

Table 2 outlines the characteristics of the included studies in this review. The publication dates of the included studies ranged from 2019 to 2022 [20,30-54]. In total, 65% (17/26) of the studies were case studies on the implementation of AI in hospitals [20,39-54], whereas the remaining 35% (9/26) were implementation guidelines [30-38].

Of the 26 identified studies, 15 (58%) originated from the United States [20,32,33,36,37,39,43-45,47-49,52-54]; 2 (8%) originated from the United Kingdom [30,42]; 2 (8%) originated from the Netherlands [34,41]; and 1 (4%) originated from China [55], Australia [46], Japan [50], Canada [51], Austria [46], Germany [35], and Sweden [31] each.

Radiology was the clinical setting in 46% (12/26) of the studies [33,34,36,37,39-41,43-45,48,49]. A total of 38% (10/26) of the studies were conducted in general inpatient wards [20,30-32,35,38,46,47,51,52], and 15% (4/26) were conducted in emergency departments [42,50,53,54].

Quality Assessment

Regarding the 35% (9/26) of guideline studies, none fully adhered to the ADAPTE framework [29]. Although these included guideline studies had clear scopes and purposes aligned with this review, they all lacked details concerning the assessment of quality, external validation, and aftercare planning procedures. The details of this assessment for all the guideline studies can be found in Multimedia Appendix 3 [20,30-54].

With respect to the 65% (17/26) of case studies, they were classified into 3 groups: quantitative descriptive (12/17, 71%) [39,40,42-49,51,54], qualitative (4/17, 24%) [20,41,52,53], and mixed methods (1/17, 6%) [50]. Overall, 5 of the case studies met the MMAT criteria: all 4 (80%) qualitative studies and the one mixed methods study. The remaining 71% (12/17) of quantitative descriptive studies failed to fully adhere to the MMAT criteria. In all but 17% (2/12) of these quantitative descriptive studies, an appropriate data sampling strategy was not used to represent their target population [40,49]. The statistical analysis of the findings was assessed as appropriate in 58% (7/12) of the quantitative descriptive studies [42,43,46,47,49,51,54]. Overall, our assessment revealed that the quality of 81% (21/26) of the included studies was poor due to insufficient reporting of their methodologies (Multimedia Appendix 3).

RQ Findings

RQ 1A Findings: Enablers of AI Implementation in Hospitals

A total of 28 enablers extracted from both prior work and this study (n=8, 29% were new enablers identified in our study) are presented in Table 3. Most of these newly identified enablers (7/8, 88%) related to the information and technology categories, highlighting the potential opportunities for hospitals regarding data readiness and required technologies for the successful implementation of AI. A total of 54% (15/28) of the enablers were shared findings between the previous reviews and this study.

Within the scope of the 3-horizon framework [15], most included studies in this paper (22/26, 85%) indicated that the process domain facilitated the development of AI analytics within horizon 2 [20,30,31,33-44,47,48,50-54]. Co-design with clinicians was the most commonly reported enabler in 46% (12/26) of the papers in horizon 2 [30,31,33,35,39-41,43,44,52-54]. The process domain was also highlighted as having a facilitative role in the creation of new care models with AI (horizon 3) in 35% (9/26) of the papers [30,34,44,46,48,50,51,53,54]. Training end users to adopt AI solutions and interpret the insights was reported in all these 9 studies as an enabling factor in horizon 3.

Technological factors were highlighted in 58% (15/26) of the studies as enablers within horizon 2 [20,32-35,39,40,43-45,48-51,54], with the most commonly reported factor being continuous learning capability of AI analytics [32,33,40,44,51] and containerization capability by providing separated development environments [34,35,40,43,44] and applying the interoperability techniques ensuring seamless integration of diverse formats of clinical data from different hardware and software sources [34,35,39,45].

Of all the included studies, 46% (12/26) [30,34,35,38-41,43,44,48,51,54] and 31% (8/26) [30,33,35,44,46,48,53,54] identified people-related enablers across horizons 2 and 3, respectively, with multidisciplinary teams in horizon 2 and trained end users in horizon 3 being the 2 most reported enablers.

Enabling factors related to the information domain were discussed in 35% (9/26) of the included studies in this review [32,35,37,39,40,45,49-51], with data quality being the most reported enabler of the successful implementation of AI in hospitals in >50% of these papers (5/9, 56%) [37,39,40,49,51]. The enablers of the AI adoption in hospitals were reported to include factors such as considerations of data security [35,45,51] and data visualization [32,50] in horizon 2 along with AI usability [38] solutions in horizon 3.

RQ 1B Findings: Barriers to AI Implementation in Hospitals

Overall, a total of 18 barriers to AI implementation in hospitals were extracted from both prior work and this study, with 5 (28%) found to be new in this study (Table 4). Most of these newly identified barriers (4/5, 80%) were related to the information and technology categories. A total of 50% (9/18) of the identified barriers were found to be shared findings between the previous work and this study. In our analysis, some factors played dual roles, acting as both enablers and barriers. For instance, “Seamless integration” served as an enabler (enabler 5; Table 3), whereas “Disruptive integration” acted as a barrier (barrier 3; Table 4). We reported both enablers and barriers with such reversed meanings to highlight the real-world complexities due to which such factors can exhibit this duality.

Regarding the 3-horizon framework [15], 58% (15/26) of the included studies in this review showed that the process domain hindered the development of AI within horizon 2 [20,31,37,40-43,45-47,50-54]. The lack of sufficient performance assessment within horizon 2 was the most commonly reported barrier in 27% (7/26) of the papers [37,41,42,46,47,50]. The factors related to the process domain were also reported as barriers to the implementation of AI within horizon 3, with 8% (2/26) of the papers reporting alert fatigue as an obstacle to AI adoption for creating new models of care [20,53].

Information-related factors were highlighted in 31% (8/26) of the studies as barriers within horizon 2 [20,35,36,46,51], with the most commonly mentioned one being poor data quality [20,35,36,46,51]. The challenge with data shift was reported as part of the information domain within horizon 3 [32].

Technology-related challenges in horizon 2 were identified in 19% (5/26) of the studies, including issues such as the lack of customization capability and computational limitations of hardware [35,43,48,50,52].

Within horizon 3, a total of 19% (5/26) of the included papers highlighted the barriers related to the people domain [20,30,41,50,53], with lack of trust by clinicians and inexperienced end users in using AI within their routine workflows being 2 barriers reported in these studies.

RQ 2 Findings: Mapping the Findings to the 3-Horizon Framework

The identified enablers and barriers to AI implementation in hospitals (RQ 1) were mapped to the 3-horizon framework [15] across 4 categories: people, process, information, and technology within horizons 2 and 3 (Figure 3 [15]).

In horizon 2, we identified a total of 21 enablers, with most associated with technology (n=9, 43%) and processes (n=7, 33%). Moving to horizon 3, a total of 7 enablers were identified, spanning the categories of people (n=3, 43%), processes (n=3, 43%), and information (n=1, 14%). Regarding barriers, horizon 2 presented a total of 13 barriers, with >50% (n=7, 54%) falling into the process category. In horizon 3, we identified a total of 5 barriers primarily distributed among the people (n=2, 40%), process (n=2, 40%), and information (n=1, 20%) categories.

‎

Discussion

Principal Findings

The health care industry needs to adopt new models of care to respond to the ever-growing demand for health services. Over the last decade, the academic community has shown considerable interest in the application of AI to explore new innovative models of care. Despite the numerous papers published each year exploring the potential of AI in various health domains, only a few studies have been implemented into routine workflows. Investigating the factors that lead to the success or failure of AI in health care could potentially provide actionable insights for the effective implementation of AI in clinical workflows. In this review, we explored the current state of the literature focusing on the implementation of AI in hospitals. Our review of 26 studies revealed several enablers of and barriers to the implementation of AI in digital hospitals. Although our search for studies dated back to 2010, all 26 case studies and guidelines found in our study were published from 2019 onward. This is not surprising considering the significant progress made in AI implementation across many fields in recent years. Given such substantial advancements, implementation science needs to be further developed to accommodate these new AI innovations in health care [19]. This paper can serve as a road map for decision makers, presenting key actionable items to translate AI into hospital settings and leveraging it for potential new models of care.

While this paper extends the findings of previous reviews by examining the factors associated with AI implementation in health care [22-24], a significant aspect found in both previous reviews and our study underscores the significance of process-related factors for creating AI analytics. A large number of papers identified in this study (22/26, 85%) reported process factors as enablers of their AI implementation, aligning with the factors found in all previous reviews (enablers 3-9; Table 3). This commonality indicates the significant opportunity for hospitals to leverage their existing workflows as a strategic approach to enable AI adoption. In the context of developing innovative care models through AI analytics, obstacles associated with people (barriers 14 and 15; Table 4) were identified in 19% (5/26) of the included studies, consistent with findings in 2 previous reviews [22,24]. This highlights the influence of human factors in facilitating the integration of AI in practice.

Apart from the common findings between this and previous reviews, there are several novel aspects to this study. First, it centered specifically on hospitals, the largest and richest source of clinical data. Second, it incorporated AI implementation guidelines from the included studies, allowing for a broader understanding of AI implementation. Third, our review identified new enablers of AI implementation regarding technology and information that can facilitate AI implementation, including quality of data, shared infrastructure for continuous development, and capabilities regarding hardware resources. Fourth, this paper identified new barriers to AI implementation, with most of them being within the domains of process, information, and technology. These barriers included challenges such as data privacy, dealing with heterogeneous data, limitations with the customization of AI analytics, and ambiguity surrounding the design of alert definitions. Finally, the study findings were mapped to a 3-horizon framework encompassing 4 key categories: people, information, process, and technology. This framework offers a clear and practical road map for health care organizations planning to create new AI analytics.

It is important to note that, while our primary focus was on hospital facilities, the findings of this review may exhibit variations across other health care settings. For example, the incorporation of AI in outpatient care may demand different technological infrastructures to enable AI development. Future research can expand upon this study by investigating the evidence of enablers and barriers associated with AI implementation in wider health care settings, including primary care and outpatient care, as we expect that the outcomes of this study may differ in other health care settings. Moreover, the incorporation of studies related to regulatory aspects can be a crucial component for a more comprehensive understanding of the trajectory of AI adoption within health care systems.

Toward AI Implementation in Hospitals

Actionable Recommendations

In this section, we consolidate the findings of this study and prior work within the scope of a 3-horizon framework [15] and provide recommendations for health care organizations that plan to implement AI analytics in hospitals (Textbox 2). These recommendations are not the ultimate solution but rather a flexible action plan to facilitate AI implementation and mitigate potential challenges regarding the digital transformation of hospitals.

Horizon 1: Establishing Digital Infrastructure

Data form the core of AI development to create clinical analytics. Some information barriers emerging in horizon 2, presented in Table 4, may be associated with challenges regarding EMR data, for example, quality of data (barrier 8), data heterogeneity (barrier 9), and data privacy (barrier 10). In the integration of EMR systems within hospital settings, careful attention must be paid to the functionality of the system to enable routine data collection to support the continuous development of AI analytics. Prioritizing the enhancement of data quality through the implementation of rigorous validation processes is a key factor in producing generalizable, reliable, and effective AI outputs. It is also imperative to ensure strict adherence to data privacy protocols during the EMR implementation, safeguarding sensitive patient information and maintaining ethical standards in handling health care data.

Horizon 2: Creating Analytics

Horizon 2 primarily focuses on data extraction and developing AI analytics. The successful implementation of AI in this horizon will be discussed within the following themes.

Form a Diverse Team of Experts

There is evidence suggesting that building a multidisciplinary team consisting of clinicians, nurses, end users, and data scientists can facilitate the successful design and implementation of AI in hospitals (enabler 1; Table 3). Experienced data scientists can potentially increase the success of AI in health care by ensuring accurate, reliable, and fair AI output in addition to identifying biases, handling complex medical data effectively, and optimizing AI algorithms (enabler 2; Table 3).

Enhance the Existing Processes

While horizon 2 revolves around technical aspects of AI implementation, the evidence indicates that involving clinicians, end users, and technical staff in the design and implementation stages is needed for successful integration (enabler 3; Table 3). The co-design strategy can alleviate challenges such as the lack of consensus on alert definitions (barrier 6; Table 4), leading to usability improvement (enabler 28; Table 3). Enhancing the understanding of AI output through training end users has the potential to alleviate concerns about the usability of AI output, fostering a smoother adoption of AI technologies in hospitals.

The studies recognized that minimizing workflow disruption is key for the successful implementation of AI (enabler 5; Table 3). To minimize workflow disruption and ensure a smooth transition when implementing new AI solutions in hospitals, it is important to engage end users from the early stage of the development process [56], although training and education should be provided to help staff members effectively incorporate the AI solution into their daily routines. For successful implementation of less disruptive technologies such as AI, it is recommended to establish a clear vision and communication by the leadership team (enabler 23; Table 3), have innovation champions (enabler 24; Table 3), and provide incentives (enabler 26; Table 3) to drive long-term adoption and habit formation [57].

Continuous AI development with the use of routinely collected data and clinicians’ feedback ensures that AI results accurately reflect the current clinical situations in hospital settings (enabler 13; Table 3). This can support clinicians in making more accurate diagnoses and treatment decisions by leveraging the latest insights derived from AI analytics. While insufficient assessment of AI performance in hospital settings is considered a prominent obstacle to successful implementation (barrier 1; Table 4), continuous development and monitoring helps avoid “data drift,” a phenomenon in which AI models lose accuracy over time due to changes in the data or environment [32,47].

Strive for Better Data Quality and Security

The studies indicated that the implementation of AI is hindered by data privacy concerns (barrier 10; Table 4). Hospitals can mitigate the risks associated with data handling and storage by adopting standardized data frameworks and interoperability techniques (enabler 15; Table 3). These measures help minimize vulnerabilities and enhance overall data security.

The quality of data in developing AI analytics refers to the accuracy, completeness, consistency, reliability, and relevance of the data used to implement AI analytics and is considered a crucial enabler for successful AI implementation in hospitals [58]. Hospitals are encouraged to improve their data quality by implementing robust data governance protocols [21,23,31,41,42], adopting standardized data protocols to facilitate interoperability [24,34,35,39,45], and actively validating and verifying the accuracy of the data with clinicians and data scientists [30,41,54].

Strengthen Technological Infrastructures

The use of third-party hardware and software in AI solutions can limit control and raise security and privacy concerns [43]. Open-source software can improve transparency and accountability by allowing experts to identify vulnerabilities, but it can also make it easier for malicious actors to exploit them [35]. To mitigate this risk, hospitals can adopt validated open-source software with appropriate security and privacy measures, such as standardized databases and interoperability protocols [24,34,35,39,45].

Horizon 3: New Models of Care

The objective of horizon 3 is to restructure the clinical care model by harnessing the insights generated from AI analytics. While the main focus of this horizon is on clinicians and processes, fewer practical experiences are available for health organizations to help in shaping the implementation strategy.

Training end users to understand AI output is suggested to enhance the adoption of AI in hospitals (enabler 25; Table 3). Hospital leadership plays a pivotal role in facilitating the adoption of AI by providing strategic guidance, allocating necessary resources, and fostering a supportive environment for the implementation of AI initiatives (enabler 23; Table 3). Hospitals are suggested to appoint innovation managers to actively promote and facilitate the applications of AI, fostering uptake and driving the implementation process in health care (enabler 24; Table 3). Resourcing is the crucial enabler of AI integration, in particular adequate skill sets. Experienced clinicians who can interpret AI results are essential for ensuring that AI systems are used effectively and responsibly in health care organizations (enabler 22; Table 3). As a result, this can redefine the traditional models of care by advocating for evidence-based practices, patient-centered care, collaborative care, and continuous quality improvement to enhance patient outcomes and the overall quality of the care provided by health care organizations.

Limitations

Our search strategy identified 26 studies that met the inclusion criteria. All 26 studies were conducted in high-income countries. As a result, the diversity and applicability of the findings to other health care systems were constrained.

By excluding regulatory frameworks from this review in the rapidly evolving regulatory landscape, we may have limited the important implementation guidelines that ensure patient safety and ethical use of AI provided by health care regulatory bodies.

We conducted a thorough examination of the reference lists in the included studies to ensure the inclusion of all relevant papers. Despite a valid research methodology, this approach may introduce publication bias, a factor to consider when appraising the study’s findings.

The methodological reporting of most studies included in this review was assessed as poor, potentially limiting the quality of the findings of this study. While consensus discussions were held after the quality assessment to mitigate potential discrepancies in the final evaluations, it is worth recognizing that this process is subjective and the perspectives of reviewers may evolve over time, resulting in variations when assessed by different individuals.

Although our intention was to identify successful implementations, it is possible that we missed significant enablers or barriers present in failed implementations.

Conclusions

This review incorporated the identified enablers of and barriers to the implementation of AI into a 3-horizon framework to guide future implementations of hospital AI analytics to evolve practice toward an LHS. Successful AI implementation in hospitals requires a shift in conventional resource management to support a new AI implementation and maintenance strategy. Using analytics to enable better decisions in hospitals is critical to enable the ever-increasing need for health care to be met.