Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at, first published .
Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review

Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review

Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review


1Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Brisbane, Australia

2Digital Health and Informatics Directorate, Metro South Health, Brisbane, Australia

Corresponding Author:

Sundresan Naicker, PhD

Australian Centre for Health Services Innovation and Centre for Healthcare Transformation

School of Public Health and Social Work, Faculty of Health

Queensland University of Technology

Q Block, 60 Musk Avenue, Kelvin Grove QLD

Brisbane, 4059


Phone: 61 3138 6454

Fax:61 0449 876 034


Background: Computerized clinical decision support systems (CDSSs) are essential components of modern health system service delivery, particularly within acute care settings such as hospitals. Theories, models, and frameworks may assist in facilitating the implementation processes associated with CDSS innovation and its use within these care settings. These processes include context assessments to identify key determinants, implementation plans for adoption, promoting ongoing uptake, adherence, and long-term evaluation. However, there has been no prior review synthesizing the literature regarding the theories, models, and frameworks that have informed the implementation and adoption of CDSSs within hospitals.

Objective: This scoping review aims to identify the theory, model, and framework approaches that have been used to facilitate the implementation and adoption of CDSSs in tertiary health care settings, including hospitals. The rationales reported for selecting these approaches, including the limitations and strengths, are described.

Methods: A total of 5 electronic databases were searched (CINAHL via EBSCOhost, PubMed, Scopus, PsycINFO, and Embase) to identify studies that implemented or adopted a CDSS in a tertiary health care setting using an implementation theory, model, or framework. No date or language limits were applied. A narrative synthesis was conducted using full-text publications and abstracts. Implementation phases were classified according to the “Active Implementation Framework stages”: exploration (feasibility and organizational readiness), installation (organizational preparation), initial implementation (initiating implementation, ie, training), full implementation (sustainment), and nontranslational effectiveness studies.

Results: A total of 81 records (42 full text and 39 abstracts) were included. Full-text studies and abstracts are reported separately. For full-text studies, models (18/42, 43%), followed by determinants frameworks (14/42,33%), were most frequently used to guide adoption and evaluation strategies. Most studies (36/42, 86%) did not list the limitations associated with applying a specific theory, model, or framework.

Conclusions: Models and related quality improvement methods were most frequently used to inform CDSS adoption. Models were not typically combined with each other or with theory to inform full-cycle implementation strategies. The findings highlight a gap in the application of implementation methods including theories, models, and frameworks to facilitate full-cycle implementation strategies for hospital CDSSs.

J Med Internet Res 2023;25:e45163




Data in tertiary health care settings, including hospitals, are now frequently digitized through the integration of electronic health records and other systems. The embedding of computerized clinical decision support systems (CDSSs) that leverage digital clinical information to support the safe and effective provision of health services has become increasingly widespread, particularly in hospitals [1-3]. There has been growing evidence to support the application of CDSS innovations to streamline clinical management decisions [4], improve risk-based decision-making, and provide personalized care [5] within acute health care settings. This includes computerized provider order entry (CPOE) systems for medical administration [6,7], interruptive alerts to promote patient safety [4], and artificial intelligence for risk prediction or diagnostic decision support [8-10], among other applications. The effective implementation of a CDSS rests upon a multifaceted interplay among various human intermediaries (such as patients, researchers, IT specialists, health care decision makers, and physicians), integrated computerized systems, the CDSS interface, the knowledge embedded within the decision support system, and the broader health care [11]. Although other types of health information technology systems may share similar determinants [12], the interactivity of CDSSs coupled with their role in supporting clinical decision-making, add additional layers of consideration to address potential barriers and facilitators to implementation. Even the implementation of a seemingly “simple” CDSS alert and reminder system necessitates a thoughtful assessment of alert design and clinical workflows as potential barriers to adoption [13].

Despite their widespread adoption in modern health care systems, studies have observed suboptimal use of CDSS innovations over time [14-17]. Furthermore, implementing a CDSS innovation in clinical practice did not ensure changes in prescribing choices, according to a 2010 comprehensive review (n=58 trials) [16]. This review identified that a lack of technical training and a low acceptance of CDSS outputs acted as barriers to end-user adoption of CDSS innovations [16]. A more recent 2022 systematic review and meta-analysis (n=11) on the impact of CDSS on provider behavior in inpatient settings found no statistically significant increase in clinician adoption of the desired practice behavior after CDSS implementation [17]. The authors suggested that contextual determinants such as increased workflow interruptions or “CDSS frustrations” may adversely change provider behavior and CDSS interaction during implementation [17]. The differences in how well CDSS innovations can change clinical behavior and consequently impact clinical practice and patient outcomes might stem from the ineffectiveness of the CDSS innovation on its own, the impact of wider contextual factors after implementation, or interactions between the 2. Several other systematic reviews across varied clinical contexts (ie, inpatient hospital care) have been conducted to measure clinical effectiveness, provider uptake, or change in clinical practice owing to CDSS implementation [18-23]. Overall, these systematic reviews have demonstrated either no significant effect or small effect sizes of CDSS innovations to improve clinical processes and patient outcomes [18-23].

A range of factors have been reported to hinder the adoption of CDSSs, including usability, clinical workflow disruption, constrained financing systems, and medico-legal concerns, among other wider organizational and contextual barriers [14,15,24,25]. The implementation process and the mechanisms for success or failure of CDSS uptake are likely multifactorial; however, the appropriate use of tools and processes to support the identification of these factors and subsequent adoption have typically lacked systematic planning and evaluation [24,26]. Consequently, CDSS innovations and associated implementation strategies may not be contextually tailored to local health system settings [24,27]. This may account for increases in inappropriate use [28], suboptimal uptake [22], and abandonment over time of the CDSS innovation and related implementation strategies [14]. Given that CDSS often include multiple intervention components that need to be contextually adopted in fast-paced tertiary health care settings such as hospitals, mitigating the factors reported to hinder the adoption of CDSS will require evidence-based decision-making and planning [22,23].

Theories, models, and frameworks from the field of implementation science may assist in providing translational guidance for CDSS implementation and promoting its adoption in complex hospital settings. Implementation science can be defined as “the scientific study of methods to promote the systematic uptake of research findings into routine practice to improve the quality and effectiveness of routine services” [29].

Nilsen [30] conducted a review of the types and uses of theory within the field of implementation science, which was used to create a taxonomy of 5 categories of theories, models, and frameworks. Their review led to the creation of a taxonomy comprising 5 categories encompassing theories, models, and frameworks. One category, referred to as “process models,” includes models that outline specific phases, steps, or stages for translating research into practice. An example of this is the 3-phase structure within the knowledge-to-action framework [31]. Another category involves “deterministic frameworks,” which encompass frameworks such as the Consolidated Framework for Implementation Research (CFIR) [32]. These frameworks incorporate descriptive categories that account for the factors influencing various aspects of the process, such as the intervention, adopters, end users, context, and strategy. The third category, “classic theories,” consists of well-established theories originating from fields such as psychology, sociology, and organizational management, for instance, the social cognitive theory [33]. The fourth category is “implementation theories,” which are novel concepts derived from multiple disciplines and are still in the process of empirical testing; an example of such a theory is the theoretical domain’s framework [34].

A separate and distinct component of the implementation methodology is “implementation evaluation.” This involves evaluating the effectiveness of the implementation strategies, and the methodologies designed for this purpose can be theories, models, or frameworks. Notable examples include the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework [35] and the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation–Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development model [36].

Despite the existence of a range of theories, models, and frameworks applied across health care settings to support digital health implementation strategies, they are not commonly used [37] in practice, particularly in hospitals [27,38]. The reasons for this gap are not well known but could be owing to a lack of awareness; uncertainty about using theories, models, or frameworks; or hesitancy to apply them in practice [39]. This is especially noted in the field of digital health, which often necessitates the adoption of multifaceted interventions in fast-paced and often resource-constrained health care settings [27,39]. To date, there has been 1 prior review of theoretical implementation frameworks for CDSSs that examined 16 studies published from 2005 to 2014 [40]. This review identified 15 different theoretical and conceptual frameworks influencing adoption, with the technology acceptance model being the most used framework [40]. However, the review did not focus on hospital (singular) settings, models and frameworks were not differentiated in the findings, and the rationale for using select theoretical approaches was not noted. Consequently, the use and application of a theory, model, or framework were not examined [40].


There has been a proliferation of models and frameworks specific to the adoption of health system interventions in practice, including health information technology [12,41]. However, it is unclear how frequently, if at all, these theories, models, and frameworks have been used to support the implementation of CDSS tools in tertiary health care settings. Furthermore, little is known about their contextual application or efficacy in promoting CDSS uptake. Even when a theory, model, or framework has been applied to a study, theoretical constructs have been found to be chosen inappropriately [42], not adhered to when applied [43-45], and often applied retrospectively after implementation when many approaches are designed to be applied prospectively [46]. As a theoretical or hypothetical understanding can shape implementation efforts, it is important to review not only what theory, model, or framework is being used (if at all) but also how it is being informed and used. This scoping review aims to identify which theories, models, or frameworks have been used to facilitate the prospective implementation and adoption of CDSSs in tertiary health care settings, including hospitals. Furthermore, it will also describe (only if mentioned within papers) any rationales or justifications for selecting a particular theory, model, or framework. This review will also identify the implementation phases for each included full-text paper using the “Active Implementation Framework” [47,48].


The review was appraised using the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist (Multimedia Appendix 1). The review was guided by the Joanna Briggs Institute Manual for Evidence Synthesis [49] and used the 5 steps from the methodological framework outlined by Arksey and O’ Malley [50]. The following activities were conducted: identifying the research question; identifying the relevant studies; selecting records; charting the data; and collating, summarizing, and reporting the results.

Identifying Relevant Studies

The search (date last searched: May 2021) was conducted in the following databases: CINAHL via EBSCOhost, PubMed, Scopus, PsycINFO, and Embase using relevant Medical Subject Heading headings and key terms, including “implementation,” “theory,” “model,” “framework,” “computerized clinical decision support system,” and “hospital.” The final search strategy was refined using the Institute for Evidence-Based Healthcare Systematic Review Accelerator Search Refinery [51] to ensure that relevant studies were being captured. The search terms used for each database can be found in Multimedia Appendix 2.

Eligibility Criteria

Records were included if they met all the eligibility criteria outlined in Textbox 1. No date or language limits were applied to exhaustively capture CDSS implementation studies. Abstracts were included in the review to capture the rapid pace of knowledge generation in the fields of digital health and implementation [52,53]. A CDSS was defined as software intended to be a direct aid to clinical decision-making. In this system, patient characteristics are matched to a computerized clinical knowledge base with specific assessments or recommendations, which are available to the clinician for a decision at the point of care. Examples of CDSSs include web applications, CPOE systems, desktops, smartphones, tablets, and devices for biometric monitoring or wearable health technology, which can be linked to electronic health record databases [1]. The inclusion criteria were restricted to prospective studies, as studies describing retrospective activities involving the mapping of theory, models, or frameworks were considered nonactive implementation activities, falling outside the purview of this review. Furthermore, retrospective assessments do not entail real-time analyses of deliberate implementation strategies in action. Instead, retrospective studies may involve evaluations facilitated by theories, models, or frameworks to distinguish effective from ineffective implementation or intervention adoption processes that have already occurred [30]. As a result, such studies do not lend themselves to exploring the advantages of using theories, models, or frameworks for implementation [46]. Rather, they contribute to understanding the mechanics of successful implementation and its outcomes.

The perspectives adopted for this review were those of end users within the health system, such as clinicians or researchers. Consequently, studies that focused on patient perspectives and feedback stemming from information retrieved during clinical management were excluded from this scoping review [16,54]. Typically, CDSSs forms part of tertiary health care infrastructure that is generally unseen by patients themselves. Consequently, decisions to adopt, expand, or use these tools are made by health service leaders, often without consumer consultation [16,54]. The categories of the taxonomy by Nilsen [30] were used to classify the approaches reported as either models, determinant frameworks, evaluation frameworks, or theories. Further operational definitions can be found in Multimedia Appendix 3.

Textbox 1. Inclusion and exclusion criteria.

Inclusion criteria

  • Examined implementation of a computerized clinical decision support system (CDSS) into clinical or routine practice
  • CDSS designed for use at the point of care
  • Prospective implementation of CDSS
  • Conducted in a hospital or tertiary health care setting
  • Reported using a theory, model, or framework in relation to the implementation of a CDSS (the definitions and elaboration of terms used to identify theories, models, or frameworks according to the taxonomy by Nilsen [30] are provided in Multimedia Appendix 3 [1,30,54-57])

Exclusion criteria

  • Conducted solely in a primary care or community setting such as general practice or allied health care clinics
  • Developed a theory, model, or framework but did not apply it to a CDSS implementation
  • Described the development of a CDSS
  • Retrospectively evaluated a CDSS
  • Studied patient decision-making as opposed to clinical decision-making

Study Selection

Search results from each database were imported into EndNote X9 (Clarivate), and duplicates were removed. The remaining records were then uploaded to Rayyan Beta (Rayyan Systems Inc) [58] for additional deduplication and screening. A title and abstract screen of 10% (687/6874) of the included records was conducted in Rayyan Beta by reviewers (MF, BA, and ZT) using the eligibility criteria and the definitions and elaboration document. Following this, the research team met to discuss discrepancies and make modifications to the eligibility criteria and the definitions and elaboration document. When ≥75% agreement was achieved, 2 reviewers (MF and TD) screened the remaining titles and abstracts. Conflicts were resolved by discussion or by an additional reviewer. Full-text publications were then sought for the remaining records.

Full-text records were screened in duplicate by a reviewer pair, with 1 reviewer (MF) reviewing all full texts paired with additional reviewers (TD, BA, ZT, and SN) who each reviewed a portion of the full texts. Conflicts were resolved by discussion or by reviewers BA or SN.

Charting the Data

The fields for data extraction were adapted from the Joanna Briggs Institute template found in the JBI Manual for Evidence Synthesis [49]. Extraction included the country of origin (where the study was conducted); year of publication; data type (quantitative, qualitative, or mixed); study design; types of CDSSs implemented; clinical context of implementation; implementation phase of the CDSS (as operationally defined by the Active Implementation Framework [47,48]); and the theory, model, or framework used.

The data extraction form was piloted and refined by reviewers SN and MF on 10% (5/42) of the included studies to ensure consistency and clarity in the data extracted. Following this, 1 reviewer (MF) independently extracted data for the 42 included records.

Collating, Summarizing, and Reporting the Results

The results were synthesized and reported by conducting a narrative synthesis of the extracted data from all included full-text publications and using graphical displays of the extracted information. Using the taxonomy by Nilsen [30], where described, implementation approaches were synthesized into models, determinant frameworks, evaluation frameworks, and theories. If the study used >1 theory, model, or framework, it was noted as a “mixed” approach and described appropriately, accounting for the approaches taken. Where included, rationales, strengths, and limitations of the approach were collated and summarized.

Implementation phases were categorized according to the Active Implementation Framework [47,48] using the following operational definitions: exploration (feasibility and organizational readiness), installation (organizational preparation), initial implementation (initiating implementation, ie, training), and full implementation (sustainment). This framework is a pragmatic categorization tool to identify the phases of digital health implementation using consistent, jargon-free language.

It is important to highlight that studies classified as “other” within the scope of this review were those that met the specified eligibility criteria but did not directly use a theory, model, or framework in the active implementation processes of translating a CDSS into a hospital setting. However, these studies may have used a theory, model, or framework to guide implementation strategies as a component of hybrid implementation-effectiveness trials or analogous experimental methodologies. This classification was applied to maintain a clear distinction between studies directly engaged in CDSS implementation within a hospital context and those adopting a theory, model, or framework to shape implementation strategies within hybrid trials or related experimental approaches in the same setting.

Figures were produced using R Statistical Software (v4.2.1.2; R Foundation for Statistical Computing) [59] and Wickham’s R package ggplot2 [60]. Results from the included abstracts were summarized separately using narrative synthesis.


The initial searches returned 3411 unique records after duplicate removal, of which 3228 (94.64%) were excluded after title and abstract screening (Multimedia Appendix 4). The full texts of 183 records were sought. The full text of 1 record could not be obtained despite contacting the author and was therefore included as an abstract. Of the 183 records, 44 (24%) met the inclusion criteria and were included in the review (21 abstracts and 23 full-text records).

An updated search returned 6874 unique records after duplicate removal, of which 6520 (94.85%) were excluded after title and abstract screening, as shown in Figure 1 [61]. The full texts of 354 records were sought. Of the 354 records, 19 (5.4%) full-text records and 18 (5.1%) abstracts from the updated search met the inclusion criteria and were included in this review. Thus, 81 records (39 abstracts and 42 full-text records) were included in this review, as shown in Figure 1.

Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram showing the review process of the updated search [61]. CDSS: computerized clinical decision support system; TMF: theories, models, and frameworks.

Characteristics of Full-Text Published Studies

Publication occurred in a wide range of academic journals, including clinical or discipline-specific journals and informatics or computer science–specific journals. The most frequent country of origin for the full-text studies in this review was the United States (18/42, 43%). The study designs varied considerably (Table 1); the most frequent designs were pre-post designs (15/42, 36%) and case studies or case series (11/42, 26%). There were 3 lower-middle–income countries, as defined by the World Bank [62] (Tunisia [63,64], Nepal [65], and Tanzania [66]) represented in the 42 included full-text studies.

Table 1. Characteristics of the included full-text published studies (n=42).
Taxonomy by Nilsen [30], study, and year of publicationCountryaData typeStudy design

Ashraf et al [67], 2022United StatesMixedPre-post

Ali et al [68], 2020CanadaQuantitativePre-post

Bernonille et al [69], 2011France, Denmark, and GreeceMixedCase series

Bertoni et al [70], 2019United StatesQuantitativePre-post

Cho et al [71], 2023KoreaQuantitativePre-post

Ellouzi et al [64], 2017TunisiaQuantitativeCase study

Flaherty et al [72], 2021United StatesQuantitativePre-post

Hanna et al [73], 2020United KingdomMixedPre-post

Jemal et al [63], 2019TunisiaQuantitativeCase study

Khazaei et al [74], 2015CanadaQuantitativeCase series

Klunk et al [75], 2021United StatesMixedPre-post

Leibowitz et al [76], 2018United StatesQuantitativePre-post

Maia and Lapão [77], 2017PortugalMixedCase series

Simões et al [78], 2018PortugalMixedCase series

Sinvani et al [79], 2020United StatesMixedCase series

Stanger et al [80], 2018United KingdomMixedPre-post

Streiff et al [81], 2012United StatesQuantitativeCase study

Tan et al [82], 2021SingaporeMixedPre-post
Determinant frameworks

Bomba and Land [83], 2006AustraliaQualitativeCase study

Brunner et al [84], 2020United StatesMixedPre-post

Chinman et al [85], 2019United StatesMixedStepped-wedge cluster randomized design

Förberg et al [86], 2016SwedenQuantitativeCluster randomized trial

Geva et al [87], 2021United StatesMixedObservational study

Kuo and Chang [88], 2014TaiwanQuantitativePre-post

Kawamoto et al [89], 2021United StatesMixedCase study

Mosch et al [90], 2022GermanyQualitativeQualitative exploratory study

Patapovas et al [91], 2013GermanyQuantitativeProspective 3-phase study

Sheehan et al [92], 2013United StatesQualitativeMultisite cross-sectional study

Shelley et al [93], 2018United StatesMixedCluster randomized controlled trial

Stablein et al [94], 2003United StatesMixedCross-sectional Study

van Engen-Verheul et al [95], 2016Netherlands and IrelandMixedPre-post

Wilk et al [96], 2013United KingdomMixedProspective cohort study
Evaluation framework

Fuller et al [97], 2020United StatesMixedCross-sectional study

Melnick et al [98], 2022United StatesQuantitativePragmatic cluster randomized controlled trial

Blandford et al [99], 2022United KingdomQualitativeQualitative exploratory study
Model and determinant framework

Afshar et al [100], 2023United StatesMixedPre-post
Model and evaluation framework

Jacobsohn et al [101], 2022United StatesMixedPre-post

Mehanni et al [65], 2019NepalMixedCase study
Evaluation framework and determinant framework

Paulsen et al [102], 2021NorwayQualitativeQualitative exploratory study

Vasudevan et al [66], 2020TanzaniaMixedEffectiveness-implementation hybrid study
Theory and determinant framework

Fujimori et al [103], 2022JapanMixedCross-sectional study

Hsiao and Chen [104], 2016TaiwanQuantitative Cross-sectional study

aCountry where the study was conducted.

Out of the 42 full-text studies, 25 (60%) used multiple methods of data collection (ie, quantitative and qualitative). Quantitative data collection solely occurred in 31% (13/42) of full-text records, with solely collected qualitative data being less common (4/42, 10%).

CDSS Characteristics in Full-Text Published Studies

A total of 45 separate CDSSs were reported in the 42 articles included as shown in Table 2. The most commonly reported CDSSs were electronic prescribing or order sets (12/42, 29%) [65,69,70,75,76,79-81,83,88,94,98], followed by checklists, forms, or clinical guidelines (11/42, 26%) [63,66,73,82,89,92,93,95,102,104] and electronic alerts or reminders (9/42, 21%) [67,68,77,78,84,86,91,100,101]. Commonly, CDSSs were implemented in pharmacy or prescription services (17/42, 41%).

Studies that used theory, model, and framework-informed approaches for the planned implementation of CDSS within a single implementation phase described the following: exploration (3/42, 7%), installation (9/42, 21%), initial implementation (4/42, 10%), and full implementation (2/42, 5%). The remaining studies described ≥2 phases: exploration and installation (3/42, 7%); exploration, installation, and initial implementation (4/42, 10%); installation and initial implementation (3/42, 7%); initial implementation and full implementation (1/42, 2%); installation, initial implementation, and full implementation (3 phases; 1/42, 2%); and conducting implementation activities across all 4 phases (3/42, 7%). Another 21% (9/42) of the studies were experimental rather than evaluating or conducting implementation activities directly. These hybrid effectiveness-implementation studies were considered pretranslational studies and classified as “other” [47,48].

The rationale for implementing a CDSS was described in 35 (83%) of the 45 CDSS studies. Rationales included removing the burden from clinicians in complex situations (eg, monitoring antibiotic use [77]). Other rationales cited for implementing a CDSS included the following: the CDSS as a separate intervention to address a clinical need or context (eg, reducing the incidence of hospital-acquired infections [64]), increasing workflow efficacy [79], improving adherence to clinical guidelines [82], transforming time-consuming paper versions of a CDSS to a digital version [81,87,104], engaging or collaborating with health care workers [78], and decreasing duplication of tests or health resource waste (eg, duplicates of magnetic resonance imaging scans) [84]. Other authors noted that their study was a scale-up or modification of an initial pilot of a CDSS [80,82,98], that there was existing evidence that validated the use of a CDSS [66], and contextual factors of the health system (eg, rapid turnover of clinicians) required the implementation of a CDSS [70].

Table 2. Characteristics of the computerized clinical decision support systems (CDSSs) implementations included in the full-text studies (n=42).
Taxonomy by Nilsen [30], study, and year of publicationClinical contextCDSS typeImplementation phaseTheory, model, or framework

Ashraf et al [67], 2022Medical imagingAlgorithm and alert linked to EHRaNontranslational effectiveness study PDSAb IHIc model

Ali et al [68], 2020AnestheticPrefilled electronic form and alertsNontranslational effectiveness studyPDSA IHI model

Bernonille et al [69], 2011Prescription Web app and EHR CPOEdInstallationGaston framework by de Clercq et al [105]

Bertoni et al [70], 2019PrescriptionElectronic order set in EHRFull implementationPDSA IHI model

Cho et al [71], 2023Falls screening toolAlgorithm linked to EHRNontranslational effectiveness studyPDSA

Ellouzi et al [64], 2017Hospital-acquired infection pilotClinical dashboardInstallationTheoretical framework for multiagent-based decision support

Flaherty et al [72], 2021Asthma prescriptionAlgorithm linked to EHRNontranslational effectiveness studyPDSA

Hanna et al [73], 2020Cancer clinical screeningElectronic form in EHRInitial implementationPDSA

Jemal et al [63], 2019Clinical deteriorationPrediction modelingInstallationMultiagent system and intuitionistic fuzzy logic

Khazaei et al [74], 2015Hospital-acquired infectionPrediction modeling linked to EHRExploration and installationResearch and clinical framework

Klunk et al [75], 2021PrescriptionElectronic order set in the EHRNontranslational effectiveness studyPDSA IHI model

Leibowitz et al [76], 2018Blood infectionsElectronic order set in EHRNontranslational effectiveness studyPDSA

Maia and Lapão [77], 2017Hospital-acquired infectionHAI-TooLe and alert and surveillanceExploration, installation, initial implementation, and full implementationDesign science research methodology

Simões et al [78], 2018PrescriptionHAI-Tool alerts and patient timelineExploration, installation, initial implementation, and full implementationDesign science research methodology

Sinvani et al [79], 2020PrescriptionElectronic order set in EHRExploration, installation, initial implementation, and full implementationFour phases

Stanger et al [80], 2018PrescriptionElectronic prescription and automated formInstallation, initial implementation, and full implementationPDSA

Streiff et al [81], 2012Blood infectionsCPOEExploration, installation, and initial implementationTRIPf model

Tan et al [82], 2021VaccinationElectronic nursing checklistNontranslational effectiveness studyPDSA
Determinant framework

Bomba and Land [83], 2006PharmacyElectronic prescribingExplorationTriangle of Dependencies Model by Sauer

Brunner et al [84], 2020Medical imagingClinical alertExploration, installation, and initial implementationSociotechnical framework

Chinman et al [85], 2019Opioid screeningElectronic dashboardInitial implementation and full implementationConsolidated Framework for Implementation Research

Förberg et al [86], 2016Blood infectionsElectronic reminder in EHRInstallation and initial implementationAlberta context tool

Geva et al [87], 2021Blood infectionsWeb-based, static checklist and EHRInstallationFive Rights of Clinical Decision Support

Kuo and Chang [88], 2014PrescriptionCPOE and mobile appInstallationTAMg for mobile service

Kawamoto et al [89], 2021Neonatal bilirubin management app (hospital-acquired infection), atrial fibrillation stroke calculator, diabetes pharmacotherapy outcome prediction app, lung cancer screening shared decision-making appApplication within EHR-clinical guidelines, chart, and recommendations; clinical calculators embedded in EHR; application within EHR with predictive modeling; and an EHR-integrated shared decision-making screening toolExploration, installation, and initial implementationExploration, Preparation, Implementation, and Sustainment Framework

Mosch et al [90], 2022Intensive care monitoring and prediction modelingTablet computer–based remote patient monitoring and electronic chartInstallationConsolidated Framework for Implementation Research

Patapovas et al [91], 2013PrescriptionElectronic template and alerts in EHRInitial implementationTAM 2h

Sheehan et al [92], 2013Medical imagingClinical prediction rules and guidelinesExplorationSociotechnical framework

Shelley et al [93], 2018DentalElectronic chart system and workflowInitial implementationAdaptation and modification framework by Stirman

Stablein et al [94], 2003PrescriptionCPOEExplorationCPOE readiness assessment tool

van Engen-Verheul et al [95], 2016CardiologyEHR and clinical decision support guidelinesInstallationHeuristic principles by Zhang et al [106]

Wilk et al [96], 2013AsthmaPrediction modelingInstallationTask-based support architecture and organization-based multiagent system engineering method
Evaluation framework

Fuller et al [97], 2020DischargeDashboardInstallation and initial implementationRE-AIMi framework

Melnick et al [98], 2022PrescriptionElectronic order set in EHRInstallation and initial implementationRE-AIM framework

Blandford et al [99], 2022Ophthalmology screening and referralTelehealth and artificial intelligenceNontranslational effectiveness studyNormalization process theory
Model and determinant framework

Afshar et al [100], 2023PrescriptionNatural language processing, artificial intelligence, and alertsInitial implementationPDSA and Consolidated Framework for Implementation Research
Model and evaluation framework

Jacobsohn et al [101], 2022Falls screening toolPrediction modeling and alert in EHRExploration and installationProcess model by Jacobsohn et al [101] and RE-AIM framework

Mehanni et al [65], 2019PrescriptionElectronic order set in the EHRExploration, installation, and initial implementationPDSA cycles and World Health Organization’s Systems Thinking Framework
Evaluation framework and determinant framework

Paulsen et al [102], 2021NutritionElectronic report generated on a website portalNontranslational effectiveness studyRE-AIM framework with Consolidated Framework for Implementation Research

Vasudevan et al [66], 2020Cancer digital case management systemClinical prediction rules and guidelinesFull implementationConsolidated Framework for Implementation Research and RE-AIM framework
Theory and determinant framework

Fujimori et al [103], 2022Cardiology emergency department prediction modelingNatural language processing algorithm in EHRExploration and installationUnified Theory of Acceptance and Use of Technology and Consolidated Framework for Implementation Research

Hsiao and Chen [104], 2016Not statedClinical decision support guidelinesInstallationActive theory with extended TAM

aEHR: electronic health record.

bPDSA: Plan-Do-Study-Act.

cIHI: Institute for Healthcare Improvement.

dCPOE: computerized provider order entry.

eHAI-Tool: Healthcare-associated Infections Tool.

fTRIP: translating research into practice.

gTAM: technology acceptance model.

hTAM 2: technology acceptance model 2.

iRE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance.

Theories, Models, and Frameworks Used to Inform CDSS Implementation in Full-Text Published Studies

The following approaches were used to inform CDSS implementation in the order of frequency: models (18/42, 43%), determinant frameworks (14/42, 33%), mixed approaches (7/42,17%), evaluation framework (2/42, 5%), and theory (1/42, 2%; Figure 2 [105,106]).

When a single approach was used, which was the case in most studies (35/42, 83%), a range of theories, models, or frameworks was used (21/35, 60% distinct theories, models, or frameworks). Of those, the Plan-Do-Study-Act (PDSA) cycle model (10/35, 29%) was the most used, followed by the CFIR (2/35, 6%) and Reach, Effectiveness, Adoption, Implementation, and Maintenance evaluation framework (2/35, 6%), with the remaining approaches very varied, as shown in Figure 2.

Most studies (24/42, 57%) reported the rationale for using a theory, model, or framework. The rationales reported included the formulation of knowledge or research questions [69,97]; addressing implementation factors leading to the use of CDSSs; and developing an implementation plan, including addressing modifications to an intervention [92,93,100]. Others cited the existing use or development of a theory, model, or framework [85,86,104] and the use of theory in aiding the scalability of intervention in different settings [74,77]. Some studies reported using a theory, model, or framework to generate possible solutions to a clinical need or to explore a complex system [84,96,102], whereas others cited synergy of the proposed framework with other implementation tools such as the Expert Recommendations for Implementing Change (ERIC) list of strategies [90].

The strengths and limitations of theories, models, and frameworks used for implementation were not reported in most studies. Most studies (28/42, 67%) did not report the strengths of a specific theory, model, or framework used, and 86% (36/42) of the studies did not report the limitations of applying a specific theory, model, or framework.

Studies that reported on the strengths of applying a specific theory, model, or framework referred to the lessons learned [80,90,97], implementation plans developed [102], ability to address barriers and facilitators for implementation within the intervention [92,94], adaptability of an intervention or increased scale-up [72,74,93], buy-in from stakeholders [65,76], use of interdisciplinary knowledge [64], and synergy with other implementation tools such as the ERIC [100].

Studies that reported the limitations of applying a specific theory, model, or framework referred to the need for more “finely tuned models” [83], including iterative validation of tools such as the CFIR-ERIC implementation tool [90] or iterating the scoring of items on a CPOE readiness assessment tool [94]; difficulty characterizing determinants that were too interdependent using a determinant framework (eg, the sociotechnical model) [92]; and unfamiliarity with theory, model, or framework processes (such as the use of quality improvement processes, eg, PDSA cycles by clinicians) [65].

Figure 2. Theories, models, and frameworks used in implementing computerized clinical decision support system (CDSS) in tertiary health care settings from the included full-text records (n=42). CDS: clinical decision support; CPOE: computerized provider order entry.

Findings From Study Abstracts

The use of 19 different theories, models, and frameworks was reported in 39 included abstracts (Multimedia Appendix 5 [107-145]). Of these 39 abstracts, 4 (10%) used a theory, model, or framework but lacked sufficient detail to classify this using the taxonomy by Nilsen [30]. Of the abstracts that could be classified, the majority used models (27/33, 81%), with the PDSA cycle model being the most used (19/33, 58%).

Principal Findings

This scoping review identified that the distinct theories, models, and frameworks (26/42, 62%) used in the implementation of CDSS in tertiary health care settings in included full-text studies were informed by implementation science and also by quality improvement and information technology fields. Although this highlighted the use of varying perspectives to inform CDSS implementation activities across the included full-text studies, this review indicated that process models were used most frequently.

Models assist with the practical application of an evidence-based intervention, within a system, by describing implementation-specific activities in practice [30,146]. In total, 56% (45/81) of the theories, models, and frameworks used in both full text and abstracts included in our review were categorized as models based on the taxonomy by Nilsen [30]. The frequent use of models is not unexpected, given that the use of such models is common in quality improvement, which is familiar to many working in hospital settings [147]. Process models, in particular, break implementation activities into smaller steps, making them useful in complex hospital settings with diverse stakeholders, psychosocial and behavioral influences, and resource constraints.

Despite advantages, solely using models to inform implementation approaches may hinder the long-term sustainment, monitoring, and iteration of CDSS innovations within hospital systems. Models alone, particularly simplified process models (ie, PDSA), do not typically examine the causal mechanisms for CDSS implementation or systematically investigate implementation factors, including barriers or facilitators [30,148]. A failure to account for implementation factors and mechanisms may misinform the planning and resource allocations required for subsequent implementation strategies and evaluations. In addition, without appropriate consideration of the barriers and facilitators to implementation (by conducting a context assessment using a determinants framework), the likelihood of implementation success may be adversely impacted.

Appropriate consideration of barriers and facilitators is particularly important for intersectional and multifaceted interventions comprising CDSSs and related digital innovations in health care settings. This includes integrated electronic medical record systems, which often incorporate a range of modules to inform prescribing practices, categorize patient risks, and chart vital signs. The application and use of these innovations typically require a range of system-level modifications. This could require engagement with various stakeholders within the hospital and the broader health care delivery system. These stakeholders may include informatics and health systems professionals, a range of clinical end users (such as nursing, allied health, and medical staff), and the provision of ongoing technical support [16]. Consequently, the sustainable ongoing adoption and appropriate use of CDSS requires targeted ongoing investment in upskilling staff, added labor, and the management of data outputs and related assets [24,25]. Theories and frameworks applicable to translational medicine, human factors, and implementation science were borne from the need to address this complexity, and their use in facilitating the uptake of digital innovations within health care remains relatively untapped [149].

This review highlighted that over half (29/44, 65%) of the included papers (full texts and abstracts) that only used a model reported using a quality improvement PDSA cycle to inform their implementation of a CDSS.

Although PDSA cycles may be beneficial for tailoring implementation activities within local health system settings, relying solely on quality improvement–informed processes may limit the effective implementation of novel or multisystem interventions (a frequent characteristic of CDSS tools) within hospital settings. Quality improvement initiatives are often small scale and context specific, which may undermine scalability to other settings [150,151]. Of note was the wide application of PDSA cycles within hybrid effectiveness-implementation studies. These experiments adopted PDSA cycles to inform implementation strategies within a protocolized methodology as part of an effectiveness trial. Given that this accounted for 21% (9/42) of the included studies, it is notable that various CDSS innovations are still undergoing effectiveness testing and may not yet be ready for systematic translation. Furthermore, the frequent use of PDSA cycles in both pretranslational and translational studies suggests the need to move beyond quality improvement models to capture implementation outcomes in translational implementation studies.

This review found that over half of the authors reported the rationale for using a theory, model, or framework (24/42, 57%), suggesting contextual knowledge in their approach. Conversely, less than half (18/42, 43%) did not do this, which may be owing to the potential time, expertise, and training required to apply the suite of these approaches. This cognitive burden may be considered impractical by those on the front lines of health care who must respond rapidly to circumstances that could endanger lives [38]. However, knowing the strengths of a particular theory, model, or framework can assist in building a case for use. It may help develop knowledge of practically applying theory to enhance practice [38]. Our findings corroborated previous research that using theories, models, and frameworks assists in facilitating implementation strategies, fostering engagement with stakeholders, and creating generalizable knowledge [46,152].

The limitations of applying theories, models, and frameworks in practice should also be investigated. Even specified preimplementation deterministic frameworks may not be applied with fidelity, making it difficult to establish the effect of a theoretical viewpoint and related strategies on implementation efforts. Interventions are also influenced by the contextual settings, with process evaluations demonstrating how theory-informed implementation may stray from the devised plan and be affected by contextual factors or determinants [153,154]. Failure to record this information may lead to duplication of implementation efforts in different hospital settings, which could otherwise be prevented [151,154]. Reporting the practical constraints of applying theories, models, or frameworks to inform CDSS implementation activities may benefit future implementers whether they are clinicians or researchers.

Implications for Practice and Research

This review found that only 7 (17%) of the 42 included full-text studies attempted to use a mix of theories, models, or frameworks. This is despite recommendations that a combination of theory, model, and framework-informed approaches may be needed to target a range of contextual domains throughout the implementation process for complex interventions such as CDSS adoption [40,149]. Using theoretically informed methodologies may also assist in reducing the duplication of research and implementation efforts, which may improve the current lag of CDSS implementation and uptake in hospital settings [151]. Theories and frameworks, in addition to models, allow the systematic evaluation of discrete implementation science principles across a range of contextually specific health care settings. This allows for generalizable recommendations for the uptake of evidence-based interventions across such settings [30,39,146]. Specifically, it has been suggested that a theory or a deterministic framework should be used in conjunction with familiar process models and quality improvement–informed strategies such as PDSA cycles [115,116,155]. Thus, using a quality improvement–informed process model will likely provide recognizable guidance for health care professionals in implementing CDSSs in practice, but the addition of a deterministic framework may assist in informing generalizable implementation across settings. The use of both a quality improvement process model and a deterministic framework may assist in bridging the often-siloed fields of implementation science and improvement practice to create engaging implementation plans for decision makers and policy makers [156].

Methodological Considerations

This scoping review had several methodological limitations. Although peer-reviewed abstracts were included in this review, little information apart from the type of theory, model, or framework could be identified from the included abstracts. The study settings were also limited to tertiary health care contexts to address the study aim; therefore, the findings of this review are not likely to be generalizable to other settings.

Studies that described the development of CDSS were also excluded from this review. However, given the increasing use of co-design to develop and implement stakeholder-approved CDSS tools, future reviews examining intervention development studies informed by implementation science theories, models, or frameworks may offer additional insights into the systematic application of implementation principles in health care settings.


This review of theories, models, and frameworks to inform implementation planning and adoption of CDSS interventions in tertiary health care settings identified the frequent use of quality improvement methods, classified as models, to inform implementation activities, evaluation, and experimental studies. Although pragmatically beneficial in sustaining improvements over certain periods within specific clinical or service delivery contexts, non–theory-informed and nondeterministic frameworks typically lack a multifactorial system-wide approach. Non–theory-informed and nondeterministic frameworks may also not be well suited to multifaceted, intersectional, and complex interventions such as CDSSs and related digital innovations, particularly within an acute health care context. Theories, models, and frameworks were borne from the need to address this complexity, but their potential to accelerate the adoption of digital breakthroughs in health care remains relatively untapped. Furthermore, this review highlighted the lack of consolidated or standardized reporting to describe the application of implementation approaches for CDSS interventions. Reporting implementation activities may benefit from the wider use of consolidated guidelines such as the Standards for Reporting Implementation Studies [157]. However, further work may be required to evaluate why guidelines such as Standards for Reporting Implementation Studies are not commonly used, particularly within the digital health context. Consequently, there is a need for ongoing work to facilitate a more systematic and transparent reporting of implementation strategies to diffuse what works versus what does not work in the digital health field.


This work was supported by the Digital Health Cooperative Research Centre (DHCRC). DHCRC is funded under the Commonwealth’s Cooperative Research Centre’s Program. SMM was supported by National Health and Medical Research Council–administered fellowship (#).

Authors' Contributions

All authors (MF, SN, TD, ZT, SMM, and BA) made significant contributions to the original paper. All authors contributed to the study design. MF, TD, SN, ZT, and BA identified and selected the relevant studies; SN and MF charted the data; and MF, SN, ZT, and BA analyzed, summarized, and reported the results. The manuscript was drafted by MF, SN, TD, ZT, SMM, and BA. All authors critically revised the paper in terms of important intellectual content. All authors have read and approved the final version of the manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

DOCX File , 85 KB

Multimedia Appendix 2

Search strategy.

DOCX File , 52 KB

Multimedia Appendix 3

Definitions and elaborations document.

DOC File , 47 KB

Multimedia Appendix 4

Primary search PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.

DOCX File , 50 KB

Multimedia Appendix 5

Abstracts included.

DOCX File , 24 KB

  1. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. Feb 06, 2020;3:17. [FREE Full text] [CrossRef] [Medline]
  2. Dash S, Shakyawar SK, Sharma M, Kaushik S. Big data in healthcare: management, analysis and future prospects. J Big Data. Jun 19, 2019;6(1):54. [FREE Full text] [CrossRef]
  3. Uslu A, Stausberg J. Value of the electronic medical record for hospital care: update from the literature. J Med Internet Res. Dec 23, 2021;23(12):e26323. [FREE Full text] [CrossRef] [Medline]
  4. Kwok R, Dinh M, Dinh D, Chu M. Improving adherence to asthma clinical guidelines and discharge documentation from emergency departments: implementation of a dynamic and integrated electronic decision support system. Emerg Med Australas. Feb 2009;21(1):31-37. [FREE Full text] [CrossRef] [Medline]
  5. Barrett M, Boyne J, Brandts J, Brunner-La Rocca HP, De Maesschalck L, De Wit K, et al. Artificial intelligence supported patient self-care in chronic heart failure: a paradigm shift from reactive to predictive, preventive and personalised care. EPMA J. Nov 22, 2019;10(4):445-464. [FREE Full text] [CrossRef] [Medline]
  6. Moja L, Polo Friz H, Capobussi M, Kwag K, Banzi R, Ruggiero F, et al. Effectiveness of a hospital-based computerized decision support system on clinician recommendations and patient outcomes: a randomized clinical trial. JAMA Netw Open. Dec 02, 2019;2(12):e1917094. [FREE Full text] [CrossRef] [Medline]
  7. Bertsche T, Pfaff J, Schiller P, Kaltschmidt J, Pruszydlo MG, Stremmel W, et al. Prevention of adverse drug reactions in intensive care patients by personal intervention based on an electronic clinical decision support system. Intensive Care Med. Apr 2010;36(4):665-672. [CrossRef] [Medline]
  8. Cresswell K, Callaghan M, Khan S, Sheikh Z, Mozaffar H, Sheikh A. Investigating the use of data-driven artificial intelligence in computerised decision support systems for health and social care: a systematic review. Health Informatics J. Sep 2020;26(3):2138-2147. [FREE Full text] [CrossRef] [Medline]
  9. Kunhimangalam R, Ovallath S, Joseph PK. A clinical decision support system with an integrated EMR for diagnosis of peripheral neuropathy. J Med Syst. Apr 2014;38(4):38. [CrossRef] [Medline]
  10. Schwartz JM, Moy AJ, Rossetti SC, Elhadad N, Cato KD. Clinician involvement in research on machine learning-based predictive clinical decision support for the hospital setting: a scoping review. J Am Med Inform Assoc. Mar 01, 2021;28(3):653-663. [FREE Full text] [CrossRef] [Medline]
  11. Randolph AG, Haynes RB, Wyatt JC, Cook DJ, Guyatt GH. Users' guides to the medical literature: XVIII. how to use an article evaluating the clinical impact of a computer-based clinical decision support system. JAMA. Jul 07, 1999;282(1):67-74. [CrossRef] [Medline]
  12. Scott P, Keizer N, Georgiou A. Applied Interdisciplinary Theory in Health Informatics: A Knowledge Base for Practitioners. Amsterdam, Netherlands. IOS Press; 2019.
  13. O'Sullivan D, Fraccaro P, Carson E, Weller P. Decision time for clinical decision support systems. Clin Med (Lond). Aug 06, 2014;14(4):338-341. [FREE Full text] [CrossRef] [Medline]
  14. Sheikh A, Cornford T, Barber N, Avery A, Takian A, Lichtner V, et al. Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in "early adopter" hospitals. BMJ. Oct 17, 2011;343:d6054. [FREE Full text] [CrossRef] [Medline]
  15. Cresswell KM, Bates DW, Williams R, Morrison Z, Slee A, Coleman J, et al. Evaluation of medium-term consequences of implementing commercial computerized physician order entry and clinical decision support prescribing systems in two 'early adopter' hospitals. J Am Med Inform Assoc. Oct 2014;21(e2):e194-e202. [FREE Full text] [CrossRef] [Medline]
  16. Moxey A, Robertson J, Newby D, Hains I, Williamson M, Pearson SA. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc. Jan 2010;17(1):25-33. [FREE Full text] [CrossRef]
  17. Ronan CE, Crable EL, Drainoni ML, Walkey AJ. The impact of clinical decision support systems on provider behavior in the inpatient setting: a systematic review and meta-analysis. J Hosp Med. May 2022;17(5):368-383. [CrossRef] [Medline]
  18. Mebrahtu TF, Skyrme S, Randell R, Keenan AM, Bloor K, Yang H, et al. Effects of computerised clinical decision support systems (CDSS) on nursing and allied health professional performance and patient outcomes: a systematic review of experimental and observational studies. BMJ Open. Dec 15, 2021;11(12):e053886. [FREE Full text] [CrossRef] [Medline]
  19. Rawson TM, Moore LS, Hernandez B, Charani E, Castro-Sanchez E, Herrero P, et al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately? Clin Microbiol Infect. Aug 2017;23(8):524-532. [FREE Full text] [CrossRef] [Medline]
  20. Klarenbeek SE, Weekenstroo HHA, Sedelaar JPM, Fütterer JJ, Prokop M, Tummers MJ. The effect of higher level computerized clinical decision support systems on oncology care: a systematic review. Cancers (Basel). Apr 22, 2020;12(4):1032. [FREE Full text] [CrossRef] [Medline]
  21. Taheri Moghadam S, Sadoughi F, Velayati F, Ehsanzadeh SJ, Poursharif S. The effects of clinical decision support system for prescribing medication on patient outcomes and physician practice performance: a systematic review and meta-analysis. BMC Med Inform Decis Mak. Mar 10, 2021;21(1):98. [FREE Full text] [CrossRef] [Medline]
  22. Kouri A, Yamada J, Lam Shin Cheung J, van de Velde S, Gupta S. Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake. Implement Sci. Mar 10, 2022;17(1):21. [FREE Full text] [CrossRef] [Medline]
  23. Damoiseaux-Volman BA, van der Velde N, Ruige SG, Romijn JA, Abu-Hanna A, Medlock S. Effect of interventions with a clinical decision support system for hospitalized older patients: systematic review mapping implementation and design factors. JMIR Med Inform. Jul 16, 2021;9(7):e28023. [FREE Full text] [CrossRef] [Medline]
  24. Liberati EG, Ruggiero F, Galuppo L, Gorli M, González-Lorenzo M, Maraldi M, et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci. Sep 15, 2017;12(1):113. [FREE Full text] [CrossRef] [Medline]
  25. Laka M, Milazzo A, Merlin T. Factors that impact the adoption of clinical decision support systems (CDSS) in healthcare settings for evidence-based decision making 2020. Research Square. Preprint posted online on June 22, 2020. 2023 [FREE Full text] [CrossRef]
  26. Cresswell K, Sheikh A, Franklin BD, Krasuska M, Nguyen HT, Hinder S, et al. Theoretical and methodological considerations in evaluating large-scale health information technology change programmes. BMC Health Serv Res. May 27, 2020;20(1):477. [FREE Full text] [CrossRef] [Medline]
  27. Greenhalgh T, Russell J. Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Med. Nov 02, 2010;7(11):e1000360. [FREE Full text] [CrossRef] [Medline]
  28. Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. Oct 11, 2007;2007:26-30. [FREE Full text] [Medline]
  29. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1(1):1-3. [FREE Full text] [CrossRef]
  30. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. Apr 21, 2015;10:53. [FREE Full text] [CrossRef] [Medline]
  31. Wilson KM, Brady TJ, Lesesne C, NCCDPHP Work Group on Translation. An organizing framework for translation in public health: the Knowledge to Action Framework. Prev Chronic Dis. Mar 2011;8(2):A46. [FREE Full text] [Medline]
  32. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. Aug 07, 2009;4:50. [FREE Full text] [CrossRef] [Medline]
  33. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. Mar 1977;84(2):191-215. [CrossRef] [Medline]
  34. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. Apr 24, 2012;7:37. [FREE Full text] [CrossRef] [Medline]
  35. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. Sep 1999;89(9):1322-1327. [CrossRef] [Medline]
  36. Green LW, Kreuter MW. Health Program Planning: An Educational and Ecological Approach. New York, NY. McGraw-Hill; 2005.
  37. Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation frameworks for artificial intelligence translation into health care practice: scoping review. J Med Internet Res. Jan 27, 2022;24(1):e32215. [FREE Full text] [CrossRef] [Medline]
  38. Rapport F, Smith J, Hutchinson K, Clay-Williams R, Churruca K, Bierbaum M, et al. Too much theory and not enough practice? The challenge of implementation science application in healthcare practice. J Eval Clin Pract. Dec 15, 2022;28(6):991-1002. [CrossRef] [Medline]
  39. Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. Jan 2020;283:112461. [FREE Full text] [CrossRef] [Medline]
  40. Khong PC, Holroyd E, Wang W. A critical review of the theoretical frameworks and the conceptual factors in the adoption of clinical decision support systems. Comput Inform Nurs. Dec 2015;33(12):555-570. [CrossRef] [Medline]
  41. Heinsch M, Wyllie J, Carlson J, Wells H, Tickner C, Kay-Lambkin F. Theories informing eHealth implementation: systematic review and typology classification. J Med Internet Res. May 31, 2021;23(5):e18500. [FREE Full text] [CrossRef] [Medline]
  42. Birken SA, Powell BJ, Shea CM, Haines ER, Alexis Kirk M, Leeman J, et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci. Oct 30, 2017;12(1):124. [FREE Full text] [CrossRef] [Medline]
  43. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. May 17, 2016;11:72. [FREE Full text] [CrossRef] [Medline]
  44. Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. Mar 2015;24(3):228-238. [FREE Full text] [CrossRef] [Medline]
  45. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. Dec 01, 2013;8:139. [FREE Full text] [CrossRef] [Medline]
  46. Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. "There is nothing so practical as a good theory": a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. Nov 14, 2018;18(1):857. [FREE Full text] [CrossRef] [Medline]
  47. Fixsen DL, Blase KA, Naoom SF, Wallace F. Core implementation components. Res Soc Work Pract. May 27, 2009;19(5):531-540. [FREE Full text] [CrossRef]
  48. Blanchard C, Livet M, Ward C, Sorge L, Sorensen TD, McClurg MR. The active implementation frameworks: a roadmap for advancing implementation of comprehensive medication management in primary care. Res Social Adm Pharm. Sep 2017;13(5):922-929. [CrossRef] [Medline]
  49. Peters MD, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. Oct 2020;18(10):2119-2126. [CrossRef] [Medline]
  50. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. Feb 2005;8(1):19-32. [FREE Full text] [CrossRef]
  51. Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol. May 2020;121:81-90. [CrossRef] [Medline]
  52. Aagja J, Shome S, Chandra A. A bibliometric analysis of digital health and mobile health related global research publications. Hosp Top. Apr 05, 2022:1-7. [CrossRef] [Medline]
  53. Sales AE, Wilson PM, Wensing M, Aarons GA, Armstrong R, Flottorp S, et al. Implementation science and implementation science communications: our aims, scope, and reporting expectations. Implement Sci. Aug 06, 2019;14(1):77. [FREE Full text] [CrossRef] [Medline]
  54. Osheroff JA, Teich J, Levick D, Saldana L, Velasco F, Sittig D, et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd edition. Chicago, IL. HIMSS Publishing; 2012.
  55. Flegel K. Tertiary hospitals must provide general care. CMAJ. Mar 03, 2015;187(4):235. [FREE Full text] [CrossRef] [Medline]
  56. Glauser J. The tertiary care hospital and academic setting. In: Mace SE, editor. Observation Medicine: Clinical Setting and Education. Cambridge, UK. Cambridge University Press; 2017;62-63.
  57. Liu JB, Kelz RR. Types of hospitals in the United States. JAMA. Sep 11, 2018;320(10):1074. [CrossRef] [Medline]
  58. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. Dec 05, 2016;5(1):210-210. [FREE Full text] [CrossRef] [Medline]
  59. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. URL: [accessed 1999-11-30]
  60. Wickham H. ggplot2: Elegant Graphics for Data Analysis. New York, NY. Springer; 2016.
  61. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [FREE Full text] [CrossRef] [Medline]
  62. Lower middle income. World Bank. 2022. URL: [accessed 2022-06-14]
  63. Jemal H, Kechaou Z, Ben Ayed M. Multi-agent based intuitionistic fuzzy logic healthcare decision support system. J Intell Fuzzy Syst. Sep 09, 2019;37(2):2697-2712. [FREE Full text] [CrossRef]
  64. Ellouzi H, Ltifi H, Ben Ayed M. Multi-agent modelling of decision support systems based on visual data mining. Multiagent Grid Syst. Apr 12, 2017;13(1):31-45. [FREE Full text] [CrossRef]
  65. Mehanni S, Jha D, Kumar A, Choudhury N, Dangal B, Deukmedjian G, et al. Implementing a quality improvement initiative for the management of chronic obstructive pulmonary disease in rural Nepal. BMJ Open Qual. Mar 01, 2019;8(1):e000408. [FREE Full text] [CrossRef] [Medline]
  66. Vasudevan L, Schroeder K, Raveendran Y, Goel K, Makarushka C, Masalu N, et al. Using digital health to facilitate compliance with standardized pediatric cancer treatment guidelines in Tanzania: protocol for an early-stage effectiveness-implementation hybrid study. BMC Cancer. Mar 29, 2020;20(1):254. [FREE Full text] [CrossRef] [Medline]
  67. Ashraf H, Rambarat CA, Setteducato ML, Winchester DE. Implementation effort: reducing the ordering of inappropriate echocardiograms through a point-of-care decision support tool. Am Heart J Plus. Jun 2022;18:100185. [FREE Full text] [CrossRef]
  68. Ali U, Tsang M, Campbell F, Matava C, Igbeyi B, Balakrishnan S, et al. Reducing postoperative pain in children undergoing strabismus surgery: from bundle implementation to clinical decision support tools. Paediatr Anaesth. Apr 2020;30(4):415-423. [CrossRef] [Medline]
  69. Bernonille S, Nies J, Pedersen HG, Guillot B, Maazi M, Berg AL, et al. Three different cases of exploiting decision support services for adverse drug event prevention. Stud Health Technol Inform. 2011;166:180-188. [Medline]
  70. Bertoni CB, Prusakov P, Merandi J, Bartman T. Clinical decision support to improve dosing weight use in infants with neonatal abstinence syndrome. Pediatr Qual Saf. Jul 28, 2019;4(4):e184. [FREE Full text] [CrossRef] [Medline]
  71. Cho I, Kim M, Song M, Dykes PC. Evaluation of an approach to clinical decision support for preventing inpatient falls: a pragmatic trial. JAMIA Open. Jul 2023;6(2):ooad019. [FREE Full text] [CrossRef] [Medline]
  72. Flaherty MR, Whalen K, Lee J, Duran C, Alshareef O, Yager P, et al. Implementation of a nurse-driven asthma pathway in the pediatric intensive care unit. Pediatr Qual Saf. Dec 15, 2021;6(6):e503. [FREE Full text] [CrossRef] [Medline]
  73. Hanna FW, Issa BG, Lea SC, George C, Golash A, Firn M, et al. Adrenal lesions found incidentally: how to improve clinical and cost-effectiveness. BMJ Open Qual. Feb 12, 2020;9(1):e000572. [FREE Full text] [CrossRef] [Medline]
  74. Khazaei H, McGregor C, Eklund JM, El-Khatib K. Real-time and retrospective health-analytics-as-a-service: a novel framework. JMIR Med Inform. Nov 18, 2015;3(4):e36. [FREE Full text] [CrossRef] [Medline]
  75. Klunk CJ, Barrett RE, Peterec SM, Blythe E, Brockett R, Kenney M, et al. An initiative to decrease laboratory testing in a NICU. Pediatrics. Jul 2021;148(1):e2020000570. [CrossRef] [Medline]
  76. Leibowitz M, Wolfe H, Flynn A, Waanders A, Burlingame C, Aumaier B, et al. Standardization of prophylactic platelet transfusion dosing in a pediatric oncology population: a quality improvement project. Transfusion. Dec 27, 2018;58(12):2836-2840. [CrossRef] [Medline]
  77. Maia M, Lapão L. HAITooL: An innovative antibiotic stewardship decision-supporting system to improve antibiotic prescription effectiveness. An Inst Hig Med Trop. Dec 2017;16:69-73. [FREE Full text] [CrossRef]
  78. Simões AS, Maia MR, Gregório J, Couto I, Asfeldt AM, Simonsen GS, et al. Participatory implementation of an antibiotic stewardship programme supported by an innovative surveillance and clinical decision-support system. J Hosp Infect. Nov 2018;100(3):257-264. [CrossRef] [Medline]
  79. Sinvani L, Goldin M, Roofeh R, Idriss N, Goldman A, Klein Z, et al. Implementation of hip fracture co-management program (AGS CoCare: Ortho®) in a large health system. J Am Geriatr Soc. Aug 11, 2020;68(8):1706-1713. [CrossRef] [Medline]
  80. Stanger S, Dahill M, Hillary C, Whitham R, Tasker A. Improving analgesia prescription for trauma inpatients. BMJ Open Qual. Dec 16, 2018;7(4):e000397. [FREE Full text] [CrossRef] [Medline]
  81. Streiff MB, Carolan HT, Hobson DB, Kraus PS, Holzmueller CG, Demski R, et al. Lessons from the johns Hopkins multi-disciplinary venous thromboembolism (VTE) prevention collaborative. BMJ. Jun 19, 2012;344:e3935. [FREE Full text] [CrossRef] [Medline]
  82. Tan HZ, Phang CC, Wu SY, Sim MH, Law MM, Foo MW, et al. Improving influenza and pneumococcal vaccination uptake among incident peritoneal dialysis patients: a quality improvement initiative. Int Urol Nephrol. Oct 2021;53(10):2167-2175. [CrossRef] [Medline]
  83. Bomba D, Land T. The feasibility of implementing an electronic prescribing decision support system: a case study of an Australian public hospital. Aust Health Rev. Aug 2006;30(3):380-388. [CrossRef] [Medline]
  84. Brunner MC, Sheehan SE, Yanke EM, Sittig DF, Safdar N, Hill B, et al. Joint design with providers of clinical decision support for value-based advanced shoulder imaging. Appl Clin Inform. Jan 2020;11(1):142-152. [FREE Full text] [CrossRef] [Medline]
  85. Chinman M, Gellad WF, McCarthy S, Gordon AJ, Rogal S, Mor MK, et al. Protocol for evaluating the nationwide implementation of the VA Stratification Tool for Opioid Risk Management (STORM). Implement Sci. Jan 18, 2019;14(1):5. [FREE Full text] [CrossRef] [Medline]
  86. Förberg U, Unbeck M, Wallin L, Johansson E, Petzold M, Ygge BM, et al. Effects of computer reminders on complications of peripheral venous catheters and nurses' adherence to a guideline in paediatric care--a cluster randomised study. Implement Sci. Jan 27, 2016;11(1):10. [FREE Full text] [CrossRef] [Medline]
  87. Geva A, Albert BD, Hamilton S, Manning MJ, Barrett MK, Mirchandani D, et al. eSIMPLER: a dynamic, electronic health record-integrated checklist for clinical decision support during PICU daily rounds. Pediatr Crit Care Med. Oct 01, 2021;22(10):898-905. [FREE Full text] [CrossRef] [Medline]
  88. Kuo MC, Chang P. A total design and implementation of an intelligent mobile chemotherapy medication administration. Stud Health Technol Inform. 2014;201:441-446. [Medline]
  89. Kawamoto K, Kukhareva PV, Weir C, Flynn MC, Nanjo CJ, Martin DK, et al. Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center. JAMIA Open. Jul 2021;4(3):ooab041. [FREE Full text] [CrossRef] [Medline]
  90. Mosch LK, Poncette AS, Spies C, Weber-Carstens S, Schieler M, Krampe H, et al. Creation of an evidence-based implementation framework for digital health technology in the intensive care unit: qualitative study. JMIR Form Res. Apr 08, 2022;6(4):e22866. [FREE Full text] [CrossRef] [Medline]
  91. Patapovas A, Dormann H, Sedlmayr B, Kirchner M, Sonst A, Müller F, et al. Medication safety and knowledge-based functions: a stepwise approach against information overload. Br J Clin Pharmacol. Sep 2013;76 Suppl 1(Suppl 1):14-24. [FREE Full text] [CrossRef] [Medline]
  92. Sheehan B, Nigrovic LE, Dayan PS, Kuppermann N, Ballard DW, Alessandrini E, et al. Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis. J Biomed Inform. Oct 2013;46(5):905-913. [FREE Full text] [CrossRef] [Medline]
  93. Shelley DR, Kyriakos C, Campo A, Li Y, Khalife D, Ostroff J. An analysis of adaptations to multi-level intervention strategies to enhance implementation of clinical practice guidelines for treating tobacco use in dental care settings. Contemp Clin Trials Commun. Jul 25, 2018;11:142-148. [FREE Full text] [CrossRef] [Medline]
  94. Stablein D, Welebob E, Johnson E, Metzger J, Burgess R, Classen DC. Understanding hospital readiness for computerized physician order entry. Jt Comm J Qual Saf. Jul 2003;29(7):336-344. [CrossRef] [Medline]
  95. van Engen-Verheul MM, Peute LW, de Keizer NF, Peek N, Jaspers MW. Optimizing the user interface of a data entry module for an electronic patient record for cardiac rehabilitation: a mixed method usability approach. Int J Med Inform. Mar 2016;87:15-26. [CrossRef] [Medline]
  96. Wilk S, Michalowski W, O'Sullivan D, Farion K, Sayyad-Shirabad J, Kuziemsky C, et al. A task-based support architecture for developing point-of-care clinical decision support systems for the emergency department. Methods Inf Med. 2013;52(1):18-32. [FREE Full text] [CrossRef] [Medline]
  97. Fuller TE, Pong DD, Piniella N, Pardo M, Bessa N, Yoon C, et al. Interactive digital health tools to engage patients and caregivers in discharge preparation: implementation study. J Med Internet Res. Apr 28, 2020;22(4):e15573. [FREE Full text] [CrossRef] [Medline]
  98. Melnick ER, Nath B, Dziura JD, Casey MF, Jeffery MM, Paek H, et al. User centered clinical decision support to implement initiation of buprenorphine for opioid use disorder in the emergency department: EMBED pragmatic cluster randomized controlled trial. BMJ. Jun 27, 2022;377:e069271. [FREE Full text] [CrossRef] [Medline]
  99. Blandford A, Abdi S, Aristidou A, Carmichael J, Cappellaro G, Hussain R, et al. Protocol for a qualitative study to explore acceptability, barriers and facilitators of the implementation of new teleophthalmology technologies between community optometry practices and hospital eye services. BMJ Open. Jul 20, 2022;12(7):e060810. [FREE Full text] [CrossRef] [Medline]
  100. Afshar M, Adelaine S, Resnik F, Mundt MP, Long J, Leaf M, et al. Deployment of real-time natural language processing and deep learning clinical decision support in the electronic health record: pipeline implementation for an opioid misuse screener in hospitalized adults. JMIR Med Inform. Apr 20, 2023;11:e44977. [FREE Full text] [CrossRef] [Medline]
  101. Jacobsohn GC, Leaf M, Liao F, Maru AP, Engstrom CJ, Salwei ME, et al. Collaborative design and implementation of a clinical decision support system for automated fall-risk identification and referrals in emergency departments. Healthc (Amst). Mar 2022;10(1):100598. [FREE Full text] [CrossRef] [Medline]
  102. Paulsen MM, Varsi C, Andersen LF. Process evaluation of the implementation of a decision support system to prevent and treat disease-related malnutrition in a hospital setting. BMC Health Serv Res. Mar 25, 2021;21(1):281. [FREE Full text] [CrossRef] [Medline]
  103. Fujimori R, Liu K, Soeno S, Naraba H, Ogura K, Hara K, et al. Acceptance, barriers, and facilitators to implementing artificial intelligence-based decision support systems in emergency departments: quantitative and qualitative evaluation. JMIR Form Res. Jun 13, 2022;6(6):e36501. [FREE Full text] [CrossRef] [Medline]
  104. Hsiao JL, Chen RF. Critical factors influencing physicians' intention to use computerized clinical practice guidelines: an integrative model of activity theory and the technology acceptance model. BMC Med Inform Decis Mak. Jan 16, 2016;16:3. [FREE Full text] [CrossRef] [Medline]
  105. de Clercq PA, Hasman A, Blom JA, Korsten HH. Design and implementation of a framework to support the development of clinical guidelines. Int J Med Inform. Dec 2001;64(2-3):285-318. [CrossRef] [Medline]
  106. Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform. 2003;36(1-2):23-30. [FREE Full text] [CrossRef] [Medline]
  107. Arenal LS, Najera S, Alvarez C, Casanova PS, García OP, Ramos-Mejía S. SMART ICU: digitalization of an adult intensive care unit. Intensive Care Med Exp. Oct 19, 2022:70-71. [FREE Full text] [CrossRef]
  108. Azul M, Hsu L, Krepel A. A quality improvement initiative to increase pediatric education in venous thromboembolism risk. Pediatr Blood Cancer. 2020;35(4):S277-S278. [FREE Full text] [CrossRef]
  109. Bosma A, Bechtold D, Cunningham A, Frye J, Stolle J. Scientific and clinical abstracts from WOCNext 2020 reimagined. J Wound Ostomy Cont Nurs. 2020;47:S32-SS3. [FREE Full text] [CrossRef]
  110. Buenger LE, Webber EC. Clinical decision support in the electronic medical record to increase rates of influenza vaccination in a pediatric emergency department. Pediatr Emerg Care. Nov 2020;36(11):e641-e645. [CrossRef] [Medline]
  111. Dalal AK, Fuller TE, Garabedian PM, Breuer DJ, Lemonias D, Harry E, et al. Systems engineering and human factors support of a system of novel EHR-integrated tools to prevent harm in the hospital. J Hosp Med. Jun 01, 2019;13(4):8-11. [FREE Full text]
  112. Disabato JA, Levisohn P, Hutton L, McDougall N. Improving transition to adult epilepsy care: facilitating team communication using clinical decision support in the EMR. Epilepsy Curr. Dec 05, 2013;14:151. [FREE Full text]
  113. Elshenawy S, O'Connor T, Kim A, Schwartz E, Kreger J, Fuchs T, et al. Using quality improvement methodology to reduce sedations for MRI in a quaternary care neonatal intensive care unit. Pediatrics. 2018;141:177. [FREE Full text] [CrossRef]
  114. Federico MJ, Redmond D, Swietlik D, Reese J, Thrasher J, Worack L, et al. Using quality improvement methods, clinical decision support tools, and a multidisciplinary team to improve compliance with joint commission asthma action plan standards. Am J Respir Crit Care Med. 2011;183(1):A2600. [FREE Full text] [CrossRef]
  115. Hassanain M, Zamakhshary M, Farhat G, Al-Badr A. Use of Lean methodology to improve operating room efficiency in hospitals across the Kingdom of Saudi Arabia. Int J Health Plann Manage. Apr 12, 2017;32(2):133-146. [CrossRef] [Medline]
  116. Hemmat SM, Wang J, Incze M, Ciccariello C, Pardee N, Bass E, et al. Go home with naloxone: a residentled, multidisciplinary approach to increase naloxone prescribing at discharge for patients at risk of overdose. Res Social Adm Pharm. Jun 2019;15(6):e35. [FREE Full text] [CrossRef]
  117. Hsu CN, Wang YC, Chen IL, Hsiao SC. Implementing external electronic medication records to promote effective hospital medication reconciliation. Int J Qual Health Care. Oct 14, 2016;28(suppl 1):12. [FREE Full text] [CrossRef]
  118. Hung WC, Yang CH, Cheng WL, Wu CH. A novel five “i” model of fracture liaison service: Successful experience in Taiwan. Osteoporos Int. Nov 30, 2019;30(11):2361-2362. [FREE Full text] [CrossRef]
  119. Keller DM, Thompson MA, Fuhlbrigge AL. Implementing project CORE (Coordinating Optimal Referral Experiences) in a large children's hospital. Pediatrics. 2021;147(3):31-32. [FREE Full text] [CrossRef]
  120. Ketcham EM, Fredella A, Malone M, Ketcham C, Larkin GL. 279 a novel six-sigma-based multidisciplinary administrative intervention reduces opioid-seeking patient emergency department recidivism. Ann Emerg Med. Oct 2014;64(4):S99. [FREE Full text] [CrossRef]
  121. Khatri SB, Carl JC, Brewer K, Ferrari M, Vandergriff MA, Piedimonte G. Utilizing change management principles to launch asthma care paths at a large academic medical center. Am J Respir Crit Care Med. 2015;191:A4251. [FREE Full text]
  122. Sanchez EL, Kanna B, Basir R. Quality improvement program using clinical decision support tool on the electronic medical record to enhance adherence to NAEPP guidelines. Am J Respir Crit Care Med. 2015;191:A3808. [FREE Full text]
  123. Mazloom A, Steffen K, Palmquist A, Godin M, Garnholz D. Using the Electronic Medical Record (EMR) to reduce Central Line Associated Blood Stream Infections (CLABSI) rates in the pediatric intensive care unit by improving maintenance bundle compliance. Pediatrics. 2019;144(2):160. [FREE Full text] [CrossRef]
  124. Miller CR, Haut C, Wilson J. Implementation of a neonatal hyperbilirubinemia clinical pathway in a pediatric emergency department. J Pediatr Health Care. Mar 2022;36(2):210-211. [FREE Full text] [CrossRef]
  125. Norris RS, Parekh J, Thandra A, Niraj Y, Loranger A, Baskaran J, et al. Leads for needs: optimization of cardiac monitoring. J Intern Med. 2018;13(4) [FREE Full text]
  126. Obringer E, Bartlett A. Improving influenza vaccination rates in pediatric subspecialty clinics. Pediatrics. 2018;142(1):384. [FREE Full text] [CrossRef]
  127. Vuong KT, Rungvivatjarus T, Garcia IP, Lewis C, Miller L, Kuelbs CL. Rapid implementation of remote online patient portal activation in a children's hospital during the COVID-19 pandemic. ACI Open. Jun 23, 2021;05(01):e17-e26. [FREE Full text] [CrossRef]
  128. Schwartzberg E, Selitzky S. Keeping the genie in the bottle-development and implementation of an electronic risk reduction decision support calculator for administration of high alert intravenous medications (CHAIM). Int J Clin Pharm. Feb 2012;34(1):162-163. [FREE Full text] [CrossRef]
  129. Sharif K, Natarajan MD, Monks S, Wadsworth J, Khawaja H. Increasing the detection of lung cancer on chest X-rays and improving communication of abnormal results to the clinician. Clin Radiol. Nov 2022;77(3):e38. [FREE Full text] [CrossRef]
  130. Karim F. Improving blood component ordering practices in a developing country by introducing computer physician order entry [CPOE] system. Vox Sang. May 2018;113(S1):75-76. [FREE Full text]
  131. Simões AS. Implementing effective antibiotic stewardship programs throughout a collaborative process between healthcare workers and researchers. Antimicrob Resist Infect Control. Jun 15, 2017;6(S3) [CrossRef]
  132. Singh V, Morrissey L, Science M, Ostrow O. Urinary tract infections in the paediatric emergency department: a quality improvement initiative to promote diagnostic and antimicrobial stewardship. CJEM. May 2, 2019;21(S1):S41. [FREE Full text] [CrossRef]
  133. Yeh DM, Dworsky ZD, Rungvivatjarus T, Chong AZ, Brada K, Beaston L, et al. Identifying and connecting food insecure families to resources in the inpatient setting. Pediatrics. Feb 2022;149(1):550. [FREE Full text] [Medline]
  134. Holdsworth LM, Kling SM, Smith M, Safaeinili N, Shieh L, Vilendrer S, et al. Predicting and responding to clinical deterioration in hospitalized patients by using artificial intelligence: protocol for a mixed methods, stepped wedge study. JMIR Res Protoc. Jul 07, 2021;10(7):e27532. [FREE Full text] [CrossRef] [Medline]
  135. Hunt C, Zanetti K, Kirkham B, Michalak A, Masanic C, Vaidyanath C, et al. Ontario concussion care strategy: cultivating the landscape to improve adult concussion care. Brain Inj. Dec 2016;30(5-6):665. [CrossRef]
  136. Jordan M, Hauser J, Cota S, Li H, Wolf L. The impact of cultural embeddedness on the implementation of an artificial intelligence program at triage: a qualitative study. J Transcult Nurs. Jan 08, 2023;34(1):32-39. [CrossRef] [Medline]
  137. Karim S, Fegeler C, Boeckler D, H Schwartz L, Kauczor HU, von Tengg-Kobligk H. Development, implementation, and evaluation of a structured reporting web tool for abdominal aortic aneurysms. JMIR Res Protoc. Aug 16, 2013;2(2):e30. [FREE Full text] [CrossRef] [Medline]
  138. Snader L, Soh YH, Berenbrok L, Wong A, Kane-Gill S. Identification of facilitators and barriers to clinicians' response to receiving clinical decision support alerts in the Intensive Care Unit (ICU). J Am Coll Clin Pharm. Sep 08, 2021;4(9):1207-1208. [FREE Full text] [CrossRef]
  139. Gross K, Thompson A, Lai A, Hom M, Liu K, Rennke S, et al. An implementation science approach to improving intravenous albumin stewardship in adult inpatients. J Am Coll Clin Pharm. Feb 03, 2020;3(1):145-391. [FREE Full text] [CrossRef]
  140. Macheel CA, Reicks P, Sybrant C, Evans C, Farhat J, West M, et al. Clinical decision support intervention improves adherence with rib fracture evidence-based practice and reduces morbidity and hospital length of stay. J Am Coll Surg. Oct 2019;229(4):e61. [FREE Full text] [CrossRef]
  141. Steinmo S, Voegele P, Comerford O, Hughes J, Moss H, Bayandor SJ, et al. An in-situ/automated sustained improvement prototype: physiological targets setting in critical care. Intensive Care Med Exp. Oct 19, 2022;10(Suppl 2):40. [FREE Full text] [CrossRef]
  142. Moreno-Franco P, Grek A, Maniaci M, Shirazi E, Starbuck A, Thompson K, et al. Sepsis and shock response team: a recipe for high quality surviving sepsis guidelines implementation. Crit Care Med. 2015;43(12):208. [FREE Full text] [CrossRef]
  143. DeNardo B, Renaud T, Chawla A. Successful implementation of a computerized physician order entry system for pediatric chemotherapy treatment protocols. Pediatric Blood & Cancer. Apr 03, 2017;64(S1):S59. [CrossRef]
  144. Kibert J, Vinluan K, Pardo K, Anderson M, Lucas J, Parra D. Implementation of a comprehensive population management model to improve anticoagulation clinic efficiency of monitoring the safety and efficacy of direct oral anticoagulants (DOACs). Pharmacotherapy. 2017;37(12):e216-e217. [FREE Full text]
  145. Thompson CM, Kell CC, Bingham AL, Sarram M, Ellis B, Hart R, et al. A multidisciplinary team approach to care transitions for hospitalized heart failure patients reduces hospital readmissions and improves outcomes Betty Irene Moore Nursing Initiative. European J of Heart Fail. May 22, 2015;17(S1):415. [FREE Full text] [CrossRef]
  146. Moullin JC, Dickson KS, Stadnick NA, Albers B, Nilsen P, Broder-Fingert S, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. Apr 30, 2020;1(1):42. [FREE Full text] [CrossRef] [Medline]
  147. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. Apr 2014;23(4):290-298. [FREE Full text] [CrossRef] [Medline]
  148. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. Mar 25, 2019;19(1):189. [FREE Full text] [CrossRef] [Medline]
  149. Camacho J, Zanoletti-Mannello M, Landis-Lewis Z, Kane-Gill SL, Boyce RD. A conceptual framework to study the implementation of clinical decision support systems (BEAR): literature review and concept mapping. J Med Internet Res. Aug 06, 2020;22(8):e18388. [FREE Full text] [CrossRef] [Medline]
  150. Nilsen P, Thor J, Bender M, Leeman J, Andersson-Gäre B, Sevdalis N. Bridging the silos: a comparative analysis of implementation science and improvement science. Front Health Serv. Feb 4, 2021;1:817750. [FREE Full text] [CrossRef] [Medline]
  151. Ogrinc G, Dolansky M, Berman AJ, Chambers DA, Davies L. Different approaches to making and testing change in healthcare. BMJ. Aug 17, 2021;374:n1010. [FREE Full text] [CrossRef] [Medline]
  152. Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. Feb 09, 2010;5:14. [FREE Full text] [CrossRef] [Medline]
  153. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: medical research council guidance. BMJ. Mar 19, 2015;350:h1258. [FREE Full text] [CrossRef] [Medline]
  154. Chinman M, Acosta J, Ebener P, Shearer A. "What we have here, is a failure to [replicate]": ways to solve a replication crisis in implementation science. Prev Sci. Jul 26, 2022;23(5):739-750. [CrossRef] [Medline]
  155. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. Nov 01, 2017;19(11):e367. [FREE Full text] [CrossRef] [Medline]
  156. Leeman J, Rohweder C, Lee M, Brenner A, Dwyer A, Ko LK, et al. Aligning implementation science with improvement practice: a call to action. Implement Sci Commun. Sep 08, 2021;2(1):99. [FREE Full text] [CrossRef] [Medline]
  157. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI) statement. BMJ. Mar 06, 2017;356:i6795. [FREE Full text] [CrossRef] [Medline]

CDSS: computerized clinical decision support system
CFIR: Consolidated Framework for Implementation Research
CPOE: computerized provider order entry
ERIC: Expert Recommendations for Implementing Change
PDSA: Plan-Do-Study-Act
PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews

Edited by A Mavragani; submitted 18.12.22; peer-reviewed by M Esdar, M Barwick; comments to author 23.03.23; revised version received 18.08.23; accepted 14.09.23; published 18.10.23.


©Manasha Fernando, Bridget Abell, Zephanie Tyack, Thomasina Donovan, Steven M McPhail, Sundresan Naicker. Originally published in the Journal of Medical Internet Research (, 18.10.2023.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.