Review
Abstract
Background: Recent advancements in artificial intelligence (AI) have changed the care processes in mental health, particularly in decision-making support for health care professionals and individuals with mental health problems. AI systems provide support in several domains of mental health, including early detection, diagnostics, treatment, and self-care. The use of AI systems in care flows faces several challenges in relation to decision-making support, stemming from technology, end-user, and organizational perspectives with the AI disruption of care processes.
Objective: This study aims to explore the use of AI systems in mental health to support decision-making, focusing on 3 key areas: the characteristics of research on AI systems in mental health; the current applications, decisions, end users, and user flow of AI systems to support decision-making; and the evaluation of AI systems for the implementation of decision-making support, including elements influencing the long-term use.
Methods: A scoping review of empirical evidence was conducted across 5 databases: PubMed, Scopus, PsycINFO, Web of Science, and CINAHL. The searches were restricted to peer-reviewed articles published in English after 2011. The initial screening at the title and abstract level was conducted by 2 reviewers, followed by full-text screening based on the inclusion criteria. Data were then charted and prepared for data analysis.
Results: Of a total of 1217 articles, 12 (0.99%) met the inclusion criteria. These studies predominantly originated from high-income countries. The AI systems were used in health care, self-care, and hybrid care contexts, addressing a variety of mental health problems. Three types of AI systems were identified in terms of decision-making support: diagnostic and predictive AI, treatment selection AI, and self-help AI. The dynamics of the type of end-user interaction and system design were diverse in complexity for the integration and use of the AI systems to support decision-making in care processes. The evaluation of the use of AI systems highlighted several challenges impacting the implementation and functionality of the AI systems in care processes, including factors affecting accuracy, increase of demand, trustworthiness, patient-physician communication, and engagement with the AI systems.
Conclusions: The design, development, and implementation of AI systems to support decision-making present substantial challenges for the sustainable use of this technology in care processes. The empirical evidence shows that the evaluation of the use of AI systems in mental health is still in its early stages, with need for more empirically focused research on real-world use. The key aspects requiring further investigation include the evaluation of the use of AI-supported decision-making from human-AI interaction and human-computer interaction perspectives, longitudinal implementation studies of AI systems in mental health to assess the use, and the integration of shared decision-making in AI systems.
doi:10.2196/63548
Keywords
Introduction
Background
There has been an increasing interest in the use of artificial intelligence (AI) systems in health care in recent years as part of supporting health care professionals and individuals with health problems to facilitate care processes. This technology is taking advantage of the abundance of health care data to assess and support decision-making across multiple dimensions. Numerous studies have investigated the prospective utility of AI systems in augmenting decision-making across various health care domains. These encompass early detection, diagnostics, treatment modalities, and self-care initiatives [
- ]. While extensive research has examined the ethics, performance, and potential applications of AI models [ - ], there remains a notable gap in the literature regarding the integrative and empirical use of implemented AI systems in health processes.AI systems for decision-making are equipped to enhance health care and self-care processes. These systems not only have the potential to improve the efficiency of health care processes but also act as a bridge, integrating health care with self-care by enabling seamless information exchange between the 2 domains [
]. From a health care viewpoint, AI systems hold transformative potential for the future, particularly in the identification and diagnosis of health problems, as it can introduce more objectivity and precision [ ]. Self-care, in contrast, refers to the personal management of health and daily life activities outside the formal health care system, while self-help encompasses the use of external resources for the management of health and well-being, both within and beyond the formal health care system [ ].From the standpoint of clinical workflows and end users, AI systems present several advantages, including personalized decision-making, enhanced patient-physician communication, and support for shared decision-making (SDM) [
- ]. However, concerns have been raised by both clinicians and patients regarding the widespread adoption of AI systems, underscoring the need for cultural adjustments within health care to facilitate acceptance [ ]. Clinicians have expressed reservations about integrating AI systems into decision-making processes, given the challenges around accepting a second opinion without validation from another clinician [ ]. Beyond issues of acceptability, the implementation of AI in SDM poses various challenges, including potential delays in decision-making, communication barriers between health care professionals and patients, and uncertainty surrounding the role of AI in decision-making [ ]. Research into the use of AI systems in the context of SDM is still in its nascent stages, emphasizing the need for further exploration into how AI systems can effectively support SDM processes in the future [ ].The potential use of AI for SDM holds particular significance in the context of mental health care given the challenges in the identification and management of mental health problems faced by the contemporary mental health care institutions. However, the complex and heterogeneous nature of mental health problems presents considerable obstacles for AI systems in delivering accurate diagnostics, effective treatment guidance, and tailored interventions. Instead, AI systems currently demonstrate greater efficacy in aiding the identification of symptoms of mental health problems rather than diagnosing specific disorders [
, ]. Given this context, it becomes imperative to understand the interplay between the complexities of mental health problems and AI decision-making support mechanisms, emphasizing the incorporation of the perspectives of end users. Such an inclusive approach is essential for guiding the effective implementation of AI systems in mental health settings [ ]. Numerous studies underscore the necessity for a nuanced understanding of AI system use within real-world mental health care to inform future implementation strategies [ , - ].Objectives
Despite the potential promises and inherent challenges associated with AI systems in supporting decision-making processes within mental health care, there remains a notable research gap concerning their practical utility. Fundamental questions persist regarding the types of decisions AI systems can support in real-world environments, the extent to which these systems engage patients in the decision-making process, and the methodologies used for their evaluation. Given this backdrop, this study aims to explore the empirical evidence for the use of AI systems to support different types of decision-making in mental health. This includes how it has been researched, applied, and evaluated for use and implementation, contributing to the understanding and improvement of future design, development, and implementation of AI systems in mental health.
Methods
Study Design
A scoping review approach was considered the preferable method to address the explorative nature of the study and to systematically map the available empirical studies within the study domain. The design of this scoping review was guided by the 5-phase framework put forward by Arksey and O’Malley [
] to ensure quality and a systematic approach to the study. The framework phases included defining the research questions (RQs); identifying relevant studies; selecting the study; charting the data; and collating, summarizing, and reporting results. The reporting process was conducted following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist [ ], as shown in .Identifying the RQs
A multidisciplinary research team comprising experts from health care, nursing, AI systems, and implementation research engaged in discussions in the initial phase of the study to identify the scope of the research and RQs. Four main concepts were defined to guide the study: mental health, AI, decision-making, and implementation (
).Concept | Operational definition | Specification |
Mental health |
|
|
AIa systems |
|
|
Decision-making |
|
|
Implementation |
| This study focuses on researching AI systems specifically in terms of their use and implementation within care settings, with an emphasis on how these technologies are used and adopted in practice. Thus, AI systems with a TRL of ≤5 were excluded, as they are not sufficiently developed to allow end-user interaction or for meaningful evaluation in real-world care environments. |
aAI: artificial intelligence.
bML: machine learning.
cTRL: technology readiness level.
The following three RQs were formulated to address the aim of the study:
- What are the characteristics of research on AI systems used in relation to support decision-making in mental health?
- Which types of technologies, decisions, actors, and user flows of AI systems to support decision-making are described?
- How were the AI-based decision support systems evaluated in research in the mental health context, and what discernible consideration might enable or hinder the adoption or implementation of these systems?
Identifying Relevant Studies
A search strategy was developed within the research team with the assistance of a librarian. Literature searches were carried out using the 5 main health research databases: PubMed, Scopus, PsycINFO, Web of Science, and CINAHL. Keywords used in the searches were guided by the following concepts and Medical Subject Headings terms: mental health, artificial intelligence, decision-making, and implementation. Synonyms were joined by the Boolean operator OR; next, we combined the search strings for each keyword with the Boolean operator AND (
). Eligibility criteria were followed for the search and study selection, as shown in .Inclusion criteria
- Written in English
- Peer-reviewed
- Publication date between January 2011 and September 2022
- All artificial intelligence (AI)–based decision support systems intended for use in relation to mental health interventions
- AI tools with user interaction with patients or health professionals aiming to support decision-making
- All mental health care settings and providers
- Empirical study design
Exclusion criteria
- Interventions that use AI for other purposes than to support decision-making
- Interventions that deliver support for other than mental health professionals and care receivers
- Proof-of-concept, viewpoints, or validation studies not related to the implementation and use of AI
Study Selection (Screening)
All articles identified from the searches were uploaded to EndNote (version 20.1; Clarivate) to remove duplicates. The remaining articles were then uploaded to Rayyan (Rayyan Systems Inc), a web-based tool used to screen articles on the abstract level and identify eligible studies for full-text screening. Rayyan was selected as it provides a systematic approach and facilitates organized teamwork [
]. In the first phase of screening, researchers (HA and MN) reviewed the abstracts and held regular follow-up meetings to ensure criteria consistency. In the second phase, full-text articles were reviewed by 2 reviewers in the research team and screened to meet the eligibility criteria and address the RQs. Throughout both phases, articles where there were uncertainties about meeting the inclusion criteria were discussed among the full research team until a consensus was reached. The screening process followed a decision tree process for excluding the studies that did not meet the inclusion criteria, then illustrated in a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram.Charting the Data (Data Extraction)
A template sheet was developed to extract data from each of the included articles regarding the characteristics of research to respond to the first RQs, including the author’s name, publication date, title, aim, country, study design, care settings, mental health problems, and types of AI. A descriptive summary was performed to answer RQ1 concerning the characteristics.
Collating, Summarizing, and Reporting Results
The reporting of the results for RQ2 was performed in 3 parts. First, we categorized the AI systems described in the included articles according to their utility in supporting decision-making in mental health. Second, we mapped the AI system’s utility of supporting decision-making in relation to the end user’s interaction and use. This mapping was carried out in terms of types of interaction and decision-making stages as per a conceptual model [
, ], which describe decision-making as a cyclical process spanning from situational awareness to the execution of decisions. Third, we conducted an explorative analysis, mapping data flow and user interaction in relation to the AI system’s basic structure as elucidated by Sremac et al [ ], comprising 3 primary phases: training data, AI model, and AI output (decision). In the identified articles, the study by Sadeh-Sharvit et al [ ] did not provide sufficient data in the reporting for RQs 2 and 3.An abductive analysis was conducted for RQ3 [
] combining inductive and deductive reasoning when looking at the data by firstly conducting inductive thematic analysis and connecting the emerged themes deductively to the human, organization, and technology-fit (HOT-fit) framework [ ]. This framework provides 3 essential components when an AI system is being used; the human factor focuses on the individual user experience of the AI system; the organization factor focuses on the structure and environment evaluation of use; and the technology factor focuses on the quality of use of the system and the information provided by the AI system. These factors consist of subcategories or dimensions, and each dimension includes several elements. First, the content in each article was collated by the first author (HA) and inductively abstracted into themes. The framework was then used to deductively map the themes into the factors and their included elements. Two new elements, trustworthiness and explainability, were found in the abductive analysis and were included because they were relevant to the study objectives. Explainability refers to the ability of the AI system to provide clear and understandable reasoning of its output or decision support provided to the intended end users [ ]. Trustworthiness refers to the extent to which the AI system is trusted by the end users to use, including being reliable, ethical, and dependable for use and in supporting decision-making [ ]. The analysis was iteratively and repeatedly discussed between all the authors until consensus was achieved.Results
Overview and Study Characteristics
The literature search resulted in 1773 articles between the years 2011 and 2022. Following the removal of duplicates, 1217 articles remained for abstract-level screening. Of these, 1159 (95.23%) articles were excluded based on the inclusion and exclusion criteria, leaving 58 (4.77%) articles for full-text screening. A total of 12 (0.99%) articles were found to meet our study eligibility criteria after the completion of the screening process as seen in the PRISMA flowchart (
).
The articles included were published between 2020 and 2022, and a variety of designs were used (
). Most of the articles were from the United States (7/12, 58%); 25% (3/12) were from Canada, 8% (1/12) were from the United Kingdom, and 8% (1/12) were from China. The included studies encompass 3 settings of investigation: health care, self-care (remotely), and simulation environment. The health care studies took place in primary care, emergency department, and specialized mental health care. The self-care studies focused on AI systems using mobile apps or wearables with sensors. Simulation studies were conducted in the psychiatry department within a university-based setting. The spectrum of severity and types of mental health problems found to be supported by the AI systems varied. Depression was the most common mental health problem targeted by the AI system support (5/12, 42%), followed by substance misuse (2/12, 17%), autism spectrum disorder (2/12, 17%), suicide and mental health crisis (2/12, 17%), and psychotherapy for different mental health problems (1/12, 8%).Study, year, and country | Study aim | Study design | Care setting | Mental health problems | Type of AIa (MLb, DLc, etc) |
Dosovitsky et al [ | ], 2020; United StatesTo understand how users engage and are redirected through a chatbot for depression (Tess) to provide design recommendations. | Descriptive study | Self-care and online access | Depression | Not specified (using AI or AI model) |
Benrimoh et al [ | ], 2021; CanadaTo explore the use of a simulation center environment in evaluating the usability of (Aifred), particularly its impact on the physician–patient interaction. | Simulation-based study design | Simulation in a specialized care context (research and development) | Depression (major depressive disorder) | DL |
Dimeff et al [ | ], 2021; United StatesTo evaluate the feasibility, acceptability, and effectiveness of (Jaspr Health) among suicide adults in emergency departments. | Pilot RCTd | Emergency department | Suicide | Not specified (using AI or AI model) |
Jacobs et ale [ | ], 2021; United StatesTo investigate the influence of ML and AI on clinician decisions in major depressive disorder treatment. | Experimental study design | Primary and specialized health care | Depression (major depressive disorder) | ML |
Popescu et ale [ | ], 2021; CanadaTo examine the feasibility of an AI-powered CDSSf, which combines the operationalized 2016 Canadian Network for Mood and Anxiety Treatments guidelines with a neural network–based individualized treatment remission prediction. | Longitudinal feasibility study | Primary and specialized health care | Depression (major depressive disorder) | DL |
Prochaska et al [ | ], 2021; United StatesTo evaluate the efficacy and use of Woebot-SUDsg in managing SUD relative to a waitlist group in an RCT. | RCT | Self-care and online access | SUD | Not specified (using AI or AI model) |
Prochaska et al [ | ], 2021; United StatesTo examine the feasibility, acceptability, and preliminary efficacy of Woebot-SUDs in managing SUD. | Development and usability study | Self-care and online access | SUD | Not specified (using AI or AI model) |
Deng et al [ | ], 2022; ChinaTo evaluate the effectiveness of sensory management recommendation system device for children with autism. | Experimental study | Self-care (wearables and app) | Autism | ML |
Garriga et al [ | ], 2022; United KingdomTo explore the added value of an implemented ML model in a clinical context. | Prospective cohort study | Specialized and primary care | A range of mental health problems | ML |
Megerian et al [ | ], 2022; United StatesTo test the accuracy of an AI-based software as a medical device designed to aid primary care health care providers in diagnosing autism spectrum disorder. | A double-blinded, multisite, prospective, active comparator cohort study | Primary care | Autism | ML |
Sadeh-Sharvit et al [ | ], 2022; United StatesTo explore therapists’ use of a standard component that is pertinent across most behavioral treatments—prompting clients to summarize their treatment session as a means for consolidating and augmenting their understanding of the session and the treatment plan. | Retrospective study | Specialized care—behavioral health treatments | A range of mental health problems | ML and NLP |
Tanguay-Sela et al [ | ], 2022; CanadaTo evaluate the utility of a CDSS as perceived by physicians participating in simulated clinical interactions. | Simulation-based study design | Simulation in a primary and specialized care context (research and development) | Depression (major depressive disorder) | DL |
aAI: artificial intelligence.
bML: machine learning.
cDL: deep learning.
dRCT: randomized controlled trial.
eJacobs et al [
] and Popescu et al [ ] did not explicitly state the study aim in the articles.fCDSS: clinical decision support system.
gSUD: substance use disorder.
Decision Support Utility, Data Flow, and User’s Interaction With the AI System
Decision Support Utility
In our findings, 3 types of AI systems emerged based on their decision support utility and the form of output they provide to support decision-making in mental health, as shown in
. These three types are as follows: (1) diagnostic and predictive AI, including AI systems providing support in the identification of mental health states with diagnostic or prognostic output in binary or categorical forms; (2) treatment selection AI, including AI systems providing options for treatment with information explaining each option, designed to assess health care practitioners and patients in the therapy selection process; and (3) self-help AI, including AI systems designed to support patient and individuals in managing their own mental health, either in health care setting or through self-care practices. The self-help AI in the reviewed studies were all based on conversational agents, which address various mental health problems through psychoeducation and behavioral therapy methods.These 3 types of AI systems differ in relation to the intended end user’s engagement with the AI output to make a decision and with the extent of actionability of decisions the AI output is assisting (
[ , - ]). In , awareness represents the extent to which the AI supports the end user’s understanding of the situation related to decision-making, while acting on a decision refers to more direct and explicit AI support on which decision to be made. It could be seen that the 3 systems have different interaction dynamics. The figure shows that different AI systems require varied engagement of end users, including patients and health care professionals, to finalize a decision. Moreover, the urgency and requirement to act on a health-related decision in a care process provided by the AI systems differ as well ( ).
The dot’s locations represent the positions of the intended primary end users who are expected to interact directly with the AI system interface, and the intended primary decision-making utility (as represented in the 2 dimensions). The dot size reflects the number of AI systems that share the same location. The transparent background highlights the potential variety of involvement of secondary end users who may use the AI output alongside the primary intended user to make a decision.
In reference to
, the findings show diagnostic and predictive AI systems had the broadest potential number of primary end users from the health care and patients that needed to follow-up with the same AI output until reaching a decision without the need to interact with the AI interface. Treatment selection AI systems had a broader need for interaction through the same interface by both the health care professionals and the patients. The systems in self-help AI varied between supporting psychoeducation (closer to awareness) and psychotherapy (closer to acting on a therapeutic decision). Compared to the other types of AI systems, this was the most multifunctional type in relation to supporting various decisions.Data Flow and User’s Interaction With the AI System
It can be seen in a closer analysis of the use of AI systems for mental health issues that the 3 systems are different in the flow of data, both as input and output to the AI model; user interaction with the AI system; and the dynamics of the decision makers using the AI output. Four data flow and interaction modalities were found as seen in
.
Mapping the structure of the AI system’s use journeys shows 3 key differences: the variations in AI input data sources, the use of AI output interface, and the different user interaction interfaces along the AI use journey. First, the sources of AI input data varied in the 3 systems, and a broader range of sources of data were used in the diagnostic and predictive AI. This indicates a more diverse array of new input being incorporated into the AI data analysis. In contrast, self-help AI systems, which were based on conversational interactions, incorporated the end user’s textual interaction with a loop of self-help outputs. Second, the use of the AI output interface differed in the 3 AI systems, with more complicated dynamics of use in the treatment selection AI, where both patients and health care professionals potentially needed to interact with the same AI output interface. This is in contrast to the other 2 AI system types, where the interface was only used by 1 type of user at a time, either the health care professional or patient. Third, the interfaces used in the user journey from data input to output revealed variations in the dynamics of interaction. The interfaces for data input and output in the diagnostic and predictive AI and the treatment selection AI were separated, indicating a clear distinction between how information is entered into the system and how it is presented. In the self-help AI systems in health care settings, the input and output were integrated into a single interface for the patient’s or individual’s use, which in addition showed the potential for providing an extended interface for output for the health care professional to follow-up with the patient interaction.
Evaluation of the Implementation and Use of the AI Systems to Support Decision-Making
Overview
The findings revealed a varied distribution in how the studies evaluated AI-based decision support systems. The human factor in the self-help AI had a predominant focus compared with the other 2 AI types. AI factors were more balanced in representation in diagnostic and predictive AI and treatment selection. The organization factor emerged as the least evaluated aspect in terms of frequency in all AI system types (
). The assessment of the AI systems using the HOT-fit elements, as detailed in and [ ], reveals the different focus of evaluation across the 3 types of AI systems, illustrating potential challenges and enablers for use and implementation.HOT-fit elements | AI system type | Key findings, challenges (–), and enablers (+) | |||
Human | |||||
Amount and duration | Diagnostic and predictive, treatment selection, and self-help |
| |||
Motivation to use | Self-help |
| |||
Acceptance | Self-help |
| |||
Recurring use | Self-help |
| |||
Expectations and belief | Self-help |
| |||
Resistance and reluctance | Self-help |
| |||
Percentage used | Treatment selection and self-help |
| |||
Voluntaries of use | Self-help |
| |||
Knowledge and expertise | Treatment selection |
| |||
Overall satisfaction | Diagnostic and predictive and self-help |
| |||
Perceived usefulness | Diagnostic and predictive, treatment selection, and self-help |
| |||
Satisfaction with specific functions | Treatment selection and self-help |
| |||
Decision making satisfaction | Treatment selection |
| |||
Organization | |||||
Communication | Treatment selection |
| |||
Clinical process | Diagnostic and predictive and treatment selection |
| |||
Localization | Diagnostic and predictive |
| |||
Technology | |||||
Ease of use | Diagnostic and predictive and self-help |
| |||
Technical support | Diagnostic and predictive |
| |||
Usefulness of system features and functions | Diagnostic and predictive, treatment selection, and self-help |
| |||
Data accuracy | Diagnostic and predictive |
| |||
Trustworthiness | Treatment selection |
| |||
Explainability | Diagnostic and predictive and treatment selection |
| |||
Usefulness | Diagnostic and predictive, treatment selection, and self-help |
| |||
Relevance | Diagnostic and predictive |
| |||
Reliability | Treatment selection |
| |||
Accuracy | Treatment selection |
| |||
Empathy | Treatment selection and self-help |
|
In relation to the 3 types of AI systems, the study identified the following findings.
Diagnostic and Predictive AI
In relation to the evaluation of the use and implementation of diagnostic and predictive AI systems, the articles had a greater focus on evaluating technology elements compared with a less detailed evaluation of human and organization factors.
The human factor was considered only in terms of the amount and duration of use of the AI system, overall satisfaction, and perceived usefulness in the studies by Garriga et al [
] and Megerian et al [ ] indicating positive outcomes in this regard.In relation to the investigation of the technology factor, the main elements were accuracy and ease of use. The accuracy of the AI system was associated with several issues and considerations in relation to the relevance of the data used, illustrating the importance of including different stakeholders in the development of AI models to ensure that relevant data are included. However, in the study by Garriga et al [
] with a predictive AI system based on electronic health record data, the health care professionals highlighted that there was outdated information fed to the AI model and important qualitative data from the patients were not included. While in the study by Deng et al [ ], the data used in the diagnostic AI model for children with autism were only based on the child’s input, excluding the input from their caregivers. The study illustrated that caregivers were concerned about not having their input for better outcomes.The environment in which the AI system is used is found to contribute to affecting the accuracy. Deng et al [
] found that the prediction of the autism symptoms was less accurate when using the AI system in a less comfortable environment for the patients, and it gave more accurate output when implementing strategies that adjust the environment for the use of the AI system. Another consideration found affecting accuracy was health care professionals’ confidence in their own decisions. Megerian et al [ ] showed that greater confidence in one’s decision in ruling out the disease over AI output was associated with less accuracy of negative diagnostics (compared with false positive accuracy, which was notably higher).The ease-of-use element was evaluated in 2 studies reporting that the AI system was relatively easy to use, with the possible need for technical support due to use difficulties [
, ]. The design and ease of use of the AI system was found to influence the health care professional’s adoption of decisions provided by the AI system. The low level of understanding of the AI output by a few health care professionals in the study by Garriga et al [ ] was a reason for them to not follow-up with the decisions of the AI system.The evaluation of the organization factor was mainly focused on the clinical process element. Garriga et al [
] reported that the predictive AI tool was used as an additional resource to identify cases, and it led to additional workload due to the need for contacting more patients, arranging visits, and reviewing patient cases in a multidisciplinary team for medical SDM. While in terms of the clinical process change in this paper, the health care professionals reported that the tool did not change the method they followed to assess risk; it rather notified them of the unseen cases that might been missed in the standard practice processes. The identified facilitators for the adoption of the AI system in the study by Garriga et al [ ] for the clinical processes were good training and technical support. In contrast, the barriers reported were related to concerns about the responsibility of acting on the AI output and the potential increase in workload and time when using the AI system.Treatment Selection AI
Several elements affecting the use and implementation of AI systems were evaluated in this AI type within the 3 factors of the HOT-fit model.
Duration of use, users’ knowledge, and perceived usefulness were the main elements in the human factor found affecting the use of AI. The willingness of health care professionals to use the treatment selection AI systems was ≤5 minutes per session in the study by Benrimoh et al [
]. Moreover, 40% of the physicians in the study by Benrimoh et al [ ] stated that they would only use the AI system in complicated or treatment-resistant patients, while 50% stated they would use it for all cases. The frequency and amount of use of the AI systems in the study by Jacobs et al [ ] regarding treatment selection AI were associated with the amount of clinician’s knowledge in machine learning (ML). The increased familiarity of the health care professionals about ML was associated with the reduction of use of the ML recommendation, despite the higher level of perceived utility of ML. The health care professionals, in the 4 studies looking at the contexts in which treatment selection AI was used, expressed two main ways in which they found the AI system useful: (1) improving patient-clinician communication, which included improving patient understanding for treatment options, making communication less personal, and improving trust in treatment and (2) improving clinical practice such as potentially saving time, providing more objectivity to the practice, confirming or suggesting options, using it as a guideline, or as a source of extra information.The main technology-related elements in this type of AI system were related to accuracy and trustworthiness. Consistent with the findings in the diagnostic and preventive AI system, the increased familiarity of ML was associated with a decrease in accuracy, despite the increase in clinician confidence in their decisions [
]. Three studies [ , , ] investigated trustworthiness of the AI system and most of the participants had a high level of trust in using the AI system. Trust in the AI model was reported in the study by Tanguay-Sela et al [ ] to be improved with sessions only for participants who reported their trust as unsure in the first session of use, while ratings of either low or high levels of trust did not change for participants who reported low or high levels of trust in the first session (ie, a low level of trust did not improve with increased sessions of use). A lack of trust in particular AI features were found to be an aspect affecting the overall trust of the AI system. The participants in the study by Tanguay-Sela et al [ ] reported that their trust in the AI system was reduced due to them finding one of the features of the AI analysis irrelevant. A lack of explainability and evidence of the used models were mentioned in the study by Tanguay-Sela et al [ ] as potentially affecting trust in the AI system.AI system use in terms of the organization factor has been found to affect clinical processes, particularly in patient-physician communication. Some health care professionals in the study by Tanguay-Sela et al [
] had negative opinions about the use of the AI tool stating that it was negatively interfering with the interaction flow with the patient, while others saw it as a valuable tool in the communication process. Benrimoh et al [ ] showed that physicians turned their screen to include the patients in the medication selection process even though it was not required in the study design. The AI system was reported in the study by Tanguay-Sela et al [ ] to be potentially used in clinical processes other than direct treatment selection for the health care professionals. It was mentioned that the AI system can support clinical processes such as in assisting communication and SDM by visualizing the different options and being able to motivate options for the patients. For this to be valid, the study illustrated that the AI tool needs to be well understood by the physicians, and with transparent explanation of the options, for them to be able to communicate it to the patients.Self-Help AI
The human factor was the center of focus of evaluation in this type of AI system. Duration of use, percentage used, frequency of use, acceptability, and user satisfaction were the main elements evaluated.
More consistent duration of use and better completion rates of self-help content were found in the conversational agents with health care–supervised type [
], compared with the unguided self-care AI agents [ , , ]. The duration of use in the study by Dosovitsky et al [ ] was inconsistent ranging from 31 minutes to 48 hours to finish 1 module of content. The frequency of use in the studies by Dosovitsky et al [ ] and Prochaska et al [ ] was inconsistent as well, with a connection found in relation to the intrinsic motivation and passive engagement of the users with the conversational agents [ ], where it was found that the users would be more likely to engage in a session when it is initiated by the conversational agent. Satisfaction scored relatively high in all studies looking at satisfaction and perceived usefulness, but in relation to acceptability of use for self-care conversational agents it was found to vary according to the severity of the mental health condition, the socioeconomic status, such as cultural or racial background, marital status, and education [ ].Technology elements such as ease of use, usefulness, and empathy were investigated only in score-based evaluation showing positive outcomes, while organization elements in this AI system type were not found to be investigated.
Discussion
Principal Findings
This study explored the empirical evidence surrounding the use and implementation of AI systems for decision-making support in mental health. The identified studies primarily focused on AI systems developed for health care and self-care settings, with articles predominantly sourced from high-income countries. The state of research on AI systems in mental health is in the preimplementation stage in clinical processes. Although studies have evaluated the use of AI systems and their potential impact on clinical processes, none of the studies have explored the full adoption or long-term implementation of AI systems in care settings. Of the 12 studies, 8 (67%) evaluated the use of AI systems in real-time care use, while 4 (33%) studies focused on simulations or clinical case studies.
A total of 3 main types of AI systems were categorized according to their utility in mental health: diagnostics and predictive AI, treatment selection AI, and self-help AI. These systems exhibited differences in their use process and interaction with end users. The findings about the types of AI systems and the variety of end users engaged in using these systems were found to be consistent with previous research [
- ], which underscored the potential use of AI in mental health, and emphasized the importance of understanding the sustainable design, which enables the collaboration in decision-making and the integration of these technologies into clinical processes. The AI systems provided support to a diverse range of actors both in health care, including primary care physicians, community psychiatric nurses, psychiatrists, and occupational therapists, as well as outside the health care services, including patients, individuals with mental health problems, and caregivers.Evaluating the AI systems through the lens of HOT-fit highlighted several challenges in the integration and use of AI systems in care flow. These challenges included potential issues affecting accuracy and use biases, increased demand, physician-patient communication, and engagement with the AI system in self-care. However, despite the existing insights, there remains a pronounced lack of evidence that can help assess the integrative use and implementation of AI systems in different care contexts.
AI System Design for Decision Support and Interaction
The studies demonstrated the diverse interaction dynamics between end users of AI decision-making support systems. Interactions were influenced by the types of users and the data flow within the AI systems. There were variations in how data flows through the 3 types of AI systems. From the methods used to input data, to the types of interfaces presented to users, to the inner functionality of the AI models, and finally, to the nature of the output data—each system painted a unique picture. These variations align with the 4 central human-computer interaction (HCI) aspects identified by Rundo et al [
] for investigating decision-support systems, emphasizing the necessity of integrating HCI principles into AI technology development and implementation to ensure they effectively serve user needs, enhance decision-making outcomes, and ensure the sustainable use of AI in decision-making in mental health care.Diagnostic and predictive AI systems in the articles showed the greatest diversity in data sources, which can complicate user understanding and create a black-box effect due to lack of explainability, which can reduce user acceptance and decision-making quality [
, ]. Furthermore, how the AI model functions to support decision-making based on data input is crucial for acceptance and sustainable use [ , ]. Nevertheless, none of the studies showed how the AI model is being sustainably trained or how the end user’s use of the AI systems contributed to the continual learning process of the AI model. This process is a fundamental part of AI or ML apps that provide continual adaptation of the AI model for improvement of its performance with further use [ ]. End user–inclusive strategies, such as the human-in-the-loop approach, where health care professionals and patients actively participate in providing feedback to enhance the AI model’s functionality, can be useful in addressing this knowledge gap. This ensures that both humans and technology are aligned toward common long-term goals, promoting their joint improvements and effectiveness [ , ].Three main types of interfaces for end users were discernible when examining the use and data flow within the AI systems: data collection interfaces, AI output visualization interfaces, and combined input and output interfaces in which interaction loops take place as in self-help AI type. These interfaces were designed to be used by a single user (patient or health care professional) or multiple end users. The differences in interface types and end-user interaction can pose challenges in examining these systems and ensuring fluency in care flows. For example, while self-help AI interaction loop interfaces can simplify the flow of use and enhance automation, compared with separated input and output interfaces, it can also complicate measuring interaction effectiveness in the health care [
- ]. Moreover, when multiple end users with different backgrounds use a single AI interface for decision-making, there is a growing need for interfaces to provide on-demand customized explanations to fit the understanding and reasoning of multiple stakeholders [ ]. This also points to the need for co-design approaches to involve stakeholders from the early stages of development [ - ].Using AI system interfaces to support decision-making presents varied dynamics of interaction, not only between the end users and the AI system but also among the end users themselves. Interestingly, none of the reviewed studies included an AI system explicitly designed to facilitate SDM between health care professionals and patients nor did they assess the role of SDM when using AI systems. This absence raises questions about the extent to which collaboration between health care professionals and patients, often highlighted as crucial for improving decision quality in mental health care, is being integrated into AI-assisted processes [
, , ]. Drake et al [ ] emphasized the need to incorporate SDM into the clinical workflows for effective implementation of patients’ involvement [ ]. As AI systems become more integrated into clinical decision-making workflows, they could alter traditional clinician-patient interactions, introducing new modalities such as clinician-AI, clinician-patient, and patient-AI interactions [ ]. Each of these modalities may bring normative challenges that affect the roles of agency, transparency, trustworthiness, and responsibility among end users. Future research should explore how to thoughtfully integrate SDM in the design, implementation, and practical use of AI systems in care processes, while also addressing the broader normative shifts that may arise in clinical workflows because of AI integration.Types of AI Systems for Supporting Decision-Making
The evaluation of the 3 types of AI systems described in the articles using the HOT-fit framework highlighted several challenges impacting the use and potential implementation of these systems in mental health care processes. General challenges were observed affecting the accuracy of the AI systems due to the use of nonrepresentative or skewed data input, in addition to the selective use of the AI system participating in potential future biases. Other challenges found more in one AI system than another included the increase in usual care process steps when using the predictive AI systems, the need for adjustment for patient-physician communication when using the treatment selection AI type, and user engagement issues when using the conversational agents in self-help AI.
It was noted in some studies that health care professionals had a selective use of the AI systems for cases with specific severity, or the systems were preferred to be used by specific health care professionals more than others. This selective use with the continuous learning of the AI model may lead to feedback loop biases. This type of bias refers to the distortion of the AI output affected by the skewed use for specific cases over an extended span of time. Consequently, the AI output becomes more accurate to the selected cases and less toward the general use, which can affect the overall accuracy [
]. Furthermore, users in some of the studies described that the AI systems are useful for purposes that deviate from the AI model’s primary objective. If the potential deviated use was not considered during the implementation efforts, this issue can lead to concept drift bias. This refers to the bias that may happen when the AI processing method differs from the actual use of the AI system leading to degradation of the AI output performance over time [ ]. This underscores the need for including health care professionals and patients in the development and implementation phases to ensure that the AI system is designed and implemented in alignment with the sustainable use dynamics [ , ]. Co-design methods for analyzing sociotechnical scenarios can facilitate this by engaging stakeholders in early phases using prototype drafts to validate key AI-related concepts such as its goal and potential risks [ ]. Early stakeholder involvement is also essential for building trust and ensuring implementation and adoption. Research shows that collaboration among a diverse, interdisciplinary team, including clinicians, potential AI users, hospital leaders, quality improvement teams, human resources, and IT departments is key to achieving effective implementation. This collective expertise ensures that the AI system meets clinical needs, aligns with organizational goals, integrates smoothly into workflows, and addresses technical, operational, and user concerns [ ].In diagnostic and predictive AI, the utility of the AI systems in detecting mental health state and identifying potentially hidden cases can disrupt common clinical processes by increasing demand and requiring additional steps or time to validate the decision support presented by the AI system. Nevertheless, none of the studies evaluated how this disruption of the care process can be adjusted to and integrated into the usual care routines. If not addressed in the future, this can lead to a dilemma of consequences between undertrusting or overtrusting the AI output in decision-making. Undertrusting of the AI output may require additional steps to validate the decisions provided by the AI system leading to additional workload and time consumption or abandoning the AI system use [
, ]. In contrast, the overreliance on AI systems, particularly when they have a high success rate, can lead to automation bias, where decision makers favor AI recommendations over other sources information including their own [ ]. This may result in increased dependency on AI, discarding both health care professionals’ and patients’ judgments and preferences. Such a shift could move clinical care away from person-centered care toward the paternalized style of decision-making raising ethical and safety challenges [ , ]. A prominent example is a commercial AI model implemented in the health care system in the United States, which led to a racial bias discriminating access to health care for millions of patients [ ]. These concerns highlight the need for caution in scaling and adopting AI for critical decision-making tasks without robust evidence. Initiatives such as the AI Act aim to address these ethical issues by setting requirements for AI development and implementation. Current recommendations emphasize that final decisions in diagnosis and care should remain with human judgment, with AI serving as an assistive tool, a view supported by numerous studies in mental health care [ , - ].Treatment selection AI systems were the only type evaluating the patient-physician relationship when using the AI system. Health care professionals used the AI system in their communication with the patients in some of the studies to discuss options and guidelines. However, this integration of AI decision support can disrupt the communication process, introducing sociorelational challenges into the clinical processes that need to be addressed in the design and implementation of the AI to avoid such challenges [
]. The average time willingness to use the AI system was limited to 5 minutes in one of the studies. This time constraint may present a challenge of integrating the AI support into the patient-physician communication. It highlights the need for efficient and explainable AI that can facilitate communication within a short span of time and tailored to the individualized care processes considering both technical and end-user preferences [ ].A main challenge identified in the studies with self-help AI systems based on conversational agents is inconsistency in user engagement, including frequency of use, duration, and completion rates. The self-help AI systems, primarily used for psychotherapy and psychoeducation, face distinct challenges in both self-care and health care settings. Low engagement can reduce the effectiveness of self-help AI systems, particularly because behavioral change and cognitive behavioral therapy (CBT) techniques require long-term engagement to attain effectiveness [
, ]. While conversational agents show great potential in facilitating mental health services [ , ], several studies indicate a lack of clarity about effectiveness and evidence-based design for therapy and mental health assessment [ - ]. This challenge highlights the need for additional research on long-term effectiveness and reliability of conversational agents across various mental health applications. Jabir et al [ ] examined how conversational agents are typically evaluated and found that effectiveness assessment usually includes at least 1 clinical outcome and 1 user experience outcome. Given the need for sustained engagement with CBT-based conversational agents and the findings of our study, we recommend that future studies incorporate a third type of outcome measurement to better validate effectiveness. This additional measurement should focus on behavioral change, which is a fundamental aspect of CBT and chronic health conditions [ , ]. Cole-Lewis et al [ ] demonstrated that engagement with technology can lead to real-world behavior changes and consequently supporting health outcomes. Including behavioral change as an outcome is important to ensure that the mediated behaviors are successfully targeted not just the clinical outcomes or the user engagement. This is crucial because long-term life changes may mislead the association between self-help AI effectiveness and other positive life-related factors. Using behavior change techniques can serve as validated instruments for assessing health or decision-making behavior change, thus guiding the evaluation of conversational agents [ - ].Overall, from the findings it is evident that successful integration and implementation of AI systems in existing mental health care workflows requires substantial efforts to create evidence regarding its sustainable use. There is a lack of empirical studies evaluating the long-term implementation of AI in mental health care. Nair et al [
] identified strategies and barriers for successful implementation that lead to sustainable use in 3 essential areas (planning, implementing, and sustaining the use). When integrating AI systems for decision-making support, the evaluation of changes in clinical workflows needs to be considered proactively during the planning phase to avoid unseen organizational or increased demand issues when put in place [ ]. This step requires the inclusion of multidisciplinary individuals who will engage with the workflow using simulations or prototypes to enable the detection of any barriers or potential risks before putting the integrated AI systems in place [ , ]. Regarding implementation strategies and ensuring sustainable use, it is important to investigate approaches relevant to the context of mental health care processes addressing resistance of change, staff training, and monitoring of AI performance. This can help create evidence participating in planning for sustainable AI systems. Future studies, including case studies and longitudinal implementation research, are necessary to generate evidence for how to successfully plan, implement, and integrate AI for sustainable use in mental health. These studies can support proposing strategies for integrating AI decision-making utility in mental health practices, with the consideration to, including all relevant stakeholders in the research process.Strengths and Limitations
Some methodological considerations are worth noting. The literature search was conducted using 5 relevant databases, which were selected based on the study’s focus and the research field. However, it is important to acknowledge that using different databases may yield varying results, potentially affecting the comprehensiveness of the review. The keywords used in the search strategy were thoroughly discussed within the research group and with librarian assistance to align with the study’s aims. Nonetheless, the evolving nature of interdisciplinary field, such as health care, implementation science, and AI technology means that the literature may use a broad range of terminology. This variability could impact the identification and inclusion of relevant studies, as different keywords might yield different sets of literature.
Furthermore, the decision to include only AI systems with technology readiness level ≥6 intended to focus the exploration on AI systems that are close to the implementation phase and investigating the AI use in care settings. While this criterion ensured relevance to real-world applications, it also narrowed the scope of the review, excluding studies that focus on the earlier developmental stages of AI models. This limitation means that the review does not address the potential capabilities of emerging AI systems that are still in the research or proof of concept phase but might have profound implications for mental health applications in the future. In addition, the limited number of studies available in implementation phase restricts the ability to draw generalized conclusions about the long-term effectiveness or scalability of these AI systems in mental health.
Moreover, 2 researchers independently conducted the selection and review, enhancing the reliability and reducing potential bias in study inclusion. However, assessing SDM in relation to these AI systems posed a challenge, as most studies did not report on the interaction dynamics between health care professionals, patients, and the AI system in a mutual decision-making context. This lack of detailed reporting on SDM limits the understanding of how AI influences or facilitates collaborative care in mental health.
Implications and Future Research
This study sheds light on the knowledge gaps regarding the empirical evidence of the use and integration of AI systems in mental health decision-making processes, offering insights that can guide the development and design of human-centered AI systems. It underscores the need to consider the utility of AI systems and end-user perspectives in the planning for the development and implementation of AI systems to support decision-making in mental health, in addition to the need for more empirical evidence to validate the factors affecting behavioral sustainable use. Furthermore, this study presents the current state of complexity in which the HCI perspective is fundamental in combination with co-design methodologies to ensure person-centered care when implementing AI systems in care processes.
Future research is advised to explore the integration of support for SDM in the AI system’s design and development, addressing the gap identified in the current literature. In addition, there is a need for conceptual refinement of the current SDM process due to the disruptive effect of AI use. As the care process is evolving from the traditional physician-patient SDM, potentially adding the AI component as a third decision maker in the SDM dynamics. This adjustment is leading to changes in the essential elements of the process, and the 3-talk model, in particular when it comes to the technology-human SDM dynamics [
, ]. Conducting implementation studies are needed for creating robust empirical evidence, and to address potential long-term challenges by identifying barriers and facilitators of AI implementation and sustainable use in mental health care.Conclusions
The scoping review demonstrated the diverse range of uses of AI in the mental health field, including health care settings, self-care contexts, and hybrid approaches. However, there was a lack of empirical evidence from an implementation perspective on how the AI systems should be integrated into clinical processes. None of the studies discussed how to adjust clinical processes to accommodate aspects such as patient-physician communication and SDM.
The results showed that the utility of AI systems in supporting decision-making in mental health varies and illustrates the complexity of evaluating the utility of decision-making, influenced by the type of actors involved in decisions, the level of decision actionability, and the dynamics of data flow through the AI system. The evaluation of the use of AI systems in care settings through human, organization, and technology lenses revealed several challenges for the implementation of AI systems for sustainable use in care settings. As a consequence, these use- and implementation-related challenges may impact the accuracy of the decisions supported by the AI system, disrupt physician-patient communication, pose trust issues, and present engagement problems impacting the effectiveness and adoption of AI decision support. Future studies are needed to address stakeholder’s needs in AI systems design, evaluate the implementation of AI systems in clinical processes for sustainable use, and assess the integration of SDM in AI systems.
Acknowledgments
This research study was part of Center for Applied Intelligent Systems Research (CAISR) Health profile and was funded by the Knowledge Foundation (grant 20200208 01H) and Halmstad University. The Knowledge Foundation was not involved in any aspect of study design, data collection, analysis, interpretation of data, or in the writing and publication of this study.
Conflicts of Interest
None declared.
PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist.
PDF File (Adobe PDF File), 498 KBSearch strategy.
DOC File , 43 KBArtificial intelligence systems characteristics.
DOCX File , 41 KBAll elements used in the human, organization, and technology-fit framework with reference to the articles.
DOCX File , 22 KBReferences
- Bertl M, Ross P, Draheim D. A survey on AI and decision support systems in psychiatry – uncovering a dilemma. Expert Syst Appl. Sep 2022;202:117464. [CrossRef]
- Maslej MM, Kloiber S, Ghassemi M, Yu J, Hill SL. Out with AI, in with the psychiatrist: a preference for human-derived clinical decision support in depression care. Transl Psychiatry. Jun 16, 2023;13(1):210. [FREE Full text] [CrossRef] [Medline]
- Russ TC, Woelbert E, Davis KA, Hafferty JD, Ibrahim Z, Inkster B, et al. How data science can advance mental health research. Nat Hum Behav. Jan 10, 2019;3(1):24-32. [CrossRef] [Medline]
- Shatte AB, Hutchinson DM, Teague SJ. Machine learning in mental health: a scoping review of methods and applications. Psychol Med. Jul 2019;49(9):1426-1448. [CrossRef] [Medline]
- Grzenda A. Artificial intelligence in mental health. In: Eyre HA, Berk M, Lavretsky H, Reynolds C, editors. Convergence Mental Health: A Transdisciplinary Approach to Innovation. Oxford, UK. Oxford Academic Press; 2021:137.
- Warrier U, Warrier A, Khandelwal K. Ethical considerations in the use of artificial intelligence in mental health. Egypt J Neurol Psychiatr Neurosurg. Oct 27, 2023;59(1):36. [CrossRef]
- Ji M, Xie W, Huang R, Qian X. Forecasting the suitability of online mental health information for effective self-care developing machine learning classifiers using natural language features. Int J Environ Res Public Health. Sep 24, 2021;18(19):10048. [FREE Full text] [CrossRef] [Medline]
- Graham S, Depp C, Lee EE, Nebeker C, Tu X, Kim HC, et al. Artificial intelligence for mental health and mental illnesses: an overview. Curr Psychiatry Rep. Nov 07, 2019;21(11):116. [FREE Full text] [CrossRef] [Medline]
- Richmond RS, Connolly M. A delineation of self-management and associated concepts. Int J Healthc Manag. Aug 25, 2020;14(4):1576-1588. [CrossRef]
- Lee EE, Torous J, de Choudhury M, Depp CA, Graham SA, Kim HC, et al. Artificial intelligence for mental health care: clinical applications, barriers, facilitators, and artificial wisdom. Biol Psychiatry Cogn Neurosci Neuroimaging. Sep 2021;6(9):856-864. [FREE Full text] [CrossRef] [Medline]
- Sauerbrei A, Kerasidou A, Lucivero F, Hallowell N. The impact of artificial intelligence on the person-centred, doctor-patient relationship: some problems and solutions. BMC Med Inform Decis Mak. Apr 20, 2023;23(1):73. [FREE Full text] [CrossRef] [Medline]
- Viswanathan S, Rollwage M, Harper R. Promises and challenges of AI-enabled mental healthcare: a foundational study. In: Proceedings of the 36th Conference on Neural Information Processing Systems. 2022. Presented at: NeurIPS '22; November 28-December 9, 2022:1-6; Virtual Event. URL: https://openreview.net/pdf?id=TfmEdrBTVs5
- Triberti S, Durosini I, Pravettoni G. A "third wheel" effect in health decision making involving artificial entities: a psychological perspective. Front Public Health. Apr 28, 2020;8:117. [FREE Full text] [CrossRef] [Medline]
- Abbasgholizadeh Rahimi S, Cwintal M, Huang Y, Ghadiri P, Grad R, Poenaru D, et al. Application of artificial intelligence in shared decision making: scoping review. JMIR Med Inform. Aug 09, 2022;10(8):e36199. [FREE Full text] [CrossRef] [Medline]
- Yan WJ, Ruan QN, Jiang K. Challenges for artificial intelligence in recognizing mental disorders. Diagnostics (Basel). Dec 20, 2022;13(1):2. [FREE Full text] [CrossRef] [Medline]
- Golden G, Popescu C, Israel S, Perlman K, Armstrong C, Fratila R, et al. Applying artificial intelligence to clinical decision support in mental health: what have we learned? Health Policy Technol. Jun 2024;13(2):100844. [CrossRef]
- Espejo G, Reiner W, Wenzinger M. Exploring the role of artificial intelligence in mental healthcare: progress, pitfalls, and promises. Cureus. Sep 2023;15(9):e44748. [FREE Full text] [CrossRef] [Medline]
- Park Y, Jackson GP, Foreman MA, Gruen D, Hu J, Das AK. Evaluating artificial intelligence in medicine: phases of clinical research. JAMIA Open. Oct 2020;3(3):326-331. [FREE Full text] [CrossRef] [Medline]
- Yin J, Ngiam KY, Teo HH. Role of artificial intelligence applications in real-life clinical practice: systematic review. J Med Internet Res. Apr 22, 2021;23(4):e25759. [FREE Full text] [CrossRef] [Medline]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. Feb 2005;8(1):19-32. [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [FREE Full text] [CrossRef] [Medline]
- Granlund M, Imms C, King G, Andersson AK, Augustine L, Brooks R, et al. Definitions and operationalization of mental health problems, wellbeing and participation constructs in children with NDD: distinctions and clarifications. Int J Environ Res Public Health. Feb 09, 2021;18(4):1656. [FREE Full text] [CrossRef] [Medline]
- Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J. Jun 2019;6(2):94-98. [FREE Full text] [CrossRef] [Medline]
- Johansen ML, O'Brien JL. Decision making in nursing practice: a concept analysis. Nurs Forum. 2016;51(1):40-48. [CrossRef] [Medline]
- Irani E, Hirschman KB, Cacchione PZ, Bowles KH. Home health nurse decision-making regarding visit intensity planning for newly admitted patients: a qualitative descriptive study. Home Health Care Serv Q. 2018;37(3):211-231. [FREE Full text] [CrossRef] [Medline]
- Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JM. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med. Jan 2020;102:101762. [CrossRef] [Medline]
- Gupta JN, Forgionne GA, Mora MT. Intelligent Decision-Making: Support Systems Foundations, Applications and Challenges. Cham, Switzerland. Springer; 2007.
- Elwyn G, Frosch D, Thomson R, Joseph-Williams N, Lloyd A, Kinnersley P, et al. Shared decision making: a model for clinical practice. J Gen Intern Med. Oct 2012;27(10):1361-1367. [FREE Full text] [CrossRef] [Medline]
- Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. Dec 05, 2016;5(1):210. [FREE Full text] [CrossRef] [Medline]
- Sremac S, Kazimieras Zavadskas E, Matić B, Kopić M, Stević Ž. Neuro-fuzzy inference systems approach to decision support system for economic order quantity. Econ Res Ekon Istraz. Jun 08, 2019;32(1):1114-1137. [CrossRef]
- Sadeh-Sharvit S, Rego SA, Jefroykin S, Peretz G, Kupershmidt T. A comparison between clinical guidelines and real-world treatment data in examining the use of session summaries: retrospective study. JMIR Form Res. Aug 16, 2022;6(8):e39846. [FREE Full text] [CrossRef] [Medline]
- Thompson J. A guide to abductive thematic analysis. Qual Rep. May 20, 2022;27(5):1410-1421. [FREE Full text] [CrossRef]
- Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform. Jun 2008;77(6):386-398. [CrossRef] [Medline]
- Gunning D, Stefik M, Choi J, Miller T, Stumpf S, Yang GZ. XAI-explainable artificial intelligence. Sci Robot. Dec 18, 2019;4(37):eaay7120. [CrossRef] [Medline]
- Thiebes S, Lins S, Sunyaev A. Trustworthy artificial intelligence. Electron Mark. Oct 01, 2020;31(2):447-464. [CrossRef]
- Dosovitsky G, Pineda BS, Jacobson NC, Chang C, Escoredo M, Bunge EL. Artificial intelligence chatbot for depression: descriptive study of usage. JMIR Form Res. Nov 13, 2020;4(11):e17065. [CrossRef] [Medline]
- Benrimoh D, Tanguay-Sela M, Perlman K, Israel S, Mehltretter J, Armstrong C, et al. Using a simulation centre to evaluate preliminary acceptability and impact of an artificial intelligence-powered clinical decision support system for depression treatment on the physician-patient interaction. BJPsych Open. Jan 06, 2021;7(1):e22. [FREE Full text] [CrossRef] [Medline]
- Dimeff LA, Jobes DA, Koerner K, Kako N, Jerome T, Kelley-Brimer A, et al. Using a tablet-based app to deliver evidence-based practices for suicidal patients in the emergency department: pilot randomized controlled trial. JMIR Ment Health. Mar 01, 2021;8(3):e23022. [FREE Full text] [CrossRef] [Medline]
- Jacobs M, Pradier MF, McCoy Jr TH, Perlis RH, Doshi-Velez F, Gajos KZ. How machine-learning recommendations influence clinician treatment selections: the example of the antidepressant selection. Transl Psychiatry. Feb 04, 2021;11(1):108. [FREE Full text] [CrossRef] [Medline]
- Popescu C, Golden G, Benrimoh D, Tanguay-Sela M, Slowey D, Lundrigan E, et al. Evaluating the clinical feasibility of an artificial intelligence-powered, web-based clinical decision support system for the treatment of depression in adults: longitudinal feasibility study. JMIR Form Res. Oct 25, 2021;5(10):e31862. [FREE Full text] [CrossRef] [Medline]
- Prochaska JJ, Vogel EA, Chieng A, Baiocchi M, Maglalang DD, Pajarito S, et al. A randomized controlled trial of a therapeutic relational agent for reducing substance misuse during the COVID-19 pandemic. Drug Alcohol Depend. Oct 01, 2021;227:108986. [FREE Full text] [CrossRef] [Medline]
- Prochaska JJ, Vogel EA, Chieng A, Kendra M, Baiocchi M, Pajarito S, et al. A therapeutic relational agent for reducing problematic substance use (Woebot): development and usability study. J Med Internet Res. Mar 23, 2021;23(3):e24850. [FREE Full text] [CrossRef] [Medline]
- Deng L, Rattadilok P. A sensor and machine learning-based sensory management recommendation system for children with autism spectrum disorders. Sensors (Basel). Aug 03, 2022;22(15):5803. [FREE Full text] [CrossRef] [Medline]
- Garriga R, Mas J, Abraha S, Nolan J, Harrison O, Tadros G, et al. Machine learning model to predict mental health crises from electronic health records. Nat Med. Jun 2022;28(6):1240-1248. [FREE Full text] [CrossRef] [Medline]
- Megerian JT, Dey S, Melmed RD, Coury DL, Lerner M, Nicholls CJ, et al. Evaluation of an artificial intelligence-based medical device for diagnosis of autism spectrum disorder. NPJ Digit Med. May 05, 2022;5(1):57. [FREE Full text] [CrossRef] [Medline]
- Tanguay-Sela M, Benrimoh D, Popescu C, Perez T, Rollins C, Snook E, et al. Evaluating the perceived utility of an artificial intelligence-powered clinical decision support system for depression treatment using a simulation center. Psychiatry Res. Feb 2022;308:114336. [CrossRef] [Medline]
- Ettman CK, Galea S. The potential influence of AI on population mental health. JMIR Ment Health. Nov 16, 2023;10:e49936. [FREE Full text] [CrossRef] [Medline]
- Miner AS, Shah N, Bullock KD, Arnow BA, Bailenson J, Hancock J. Key considerations for incorporating conversational AI in psychotherapy. Front Psychiatry. 2019;10:746. [FREE Full text] [CrossRef] [Medline]
- Shimada K. The role of artificial intelligence in mental health: a review. Sci Insights. Nov 29, 2023;43(5):1119-1127. [CrossRef]
- Rundo L, Pirrone R, Vitabile S, Sala E, Gambino O. Recent advances of HCI in decision-making tasks for optimized clinical workflows and precision medicine. J Biomed Inform. Aug 2020;108:103479. [FREE Full text] [CrossRef] [Medline]
- Khan B, Fatima H, Qureshi A, Kumar S, Hanan A, Hussain J, et al. Drawbacks of artificial intelligence and their potential solutions in the healthcare sector. Biomed Mater Devices. Feb 08, 2023:1-8. [FREE Full text] [CrossRef] [Medline]
- Joyce DW, Kormilitzin A, Smith KA, Cipriani A. Explainable artificial intelligence for mental health through transparency and interpretability for understandability. NPJ Digit Med. Jan 18, 2023;6(1):6. [FREE Full text] [CrossRef] [Medline]
- Higgins O, Short BL, Chalup SK, Wilson RL. Artificial intelligence (AI) and machine learning (ML) based decision support systems in mental health: an integrative review. Int J Ment Health Nurs. Aug 2023;32(4):966-978. [CrossRef] [Medline]
- Lee CS, Lee AY. Clinical applications of continual learning machine learning. Lancet Digit Health. Jun 2020;2(6):e279-e281. [FREE Full text] [CrossRef] [Medline]
- Chandler C, Foltz PW, Elvevåg B. Improving the applicability of AI for psychiatric applications through human-in-the-loop methodologies. Schizophr Bull. Sep 01, 2022;48(5):949-957. [FREE Full text] [CrossRef] [Medline]
- Heer J. Agency plus automation: designing artificial intelligence into interactive systems. Proc Natl Acad Sci U S A. Feb 05, 2019;116(6):1844-1850. [FREE Full text] [CrossRef] [Medline]
- Ooge J, Stiglic G, Verbert K. Explaining artificial intelligence with visual analytics in healthcare. Wiley Interdiscip Rev Data Min Knowl Discov. Nov 28, 2021;12(1):e1427. [CrossRef]
- Tudor Car L, Dhinagaran DA, Kyaw BM, Kowatsch T, Joty S, Theng YL, et al. Conversational agents in health care: scoping review and conceptual analysis. J Med Internet Res. Aug 07, 2020;22(8):e17158. [FREE Full text] [CrossRef] [Medline]
- Jacobs M, He JF, Pradier MF, Lam B, Ahn AC, McCoy TH, et al. Designing AI for trust and collaboration in time-constrained medical decisions: a sociotechnical lens. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021. Presented at: CHI '21; May 8-13, 2021:1-14; Yokohama, Japan. URL: https://dl.acm.org/doi/10.1145/3411764.3445385 [CrossRef]
- Patrickson B, Musker M, Thorpe D, van Kasteren Y, Bidargaddi N. In-depth co-design of mental health monitoring technologies by people with lived experience. Future Internet. May 25, 2023;15(6):191. [CrossRef]
- Hogg HD, Al-Zubaidy M, Technology Enhanced Macular Services Study Reference Group, Talks J, Denniston AK, Kelly CJ, et al. Stakeholder perspectives of clinical artificial intelligence implementation: systematic review of qualitative evidence. J Med Internet Res. Jan 10, 2023;25:e39742. [FREE Full text] [CrossRef] [Medline]
- Zicari RV, Ahmed S, Amann J, Braun SA, Brodersen J, Bruneault F, et al. Co-design of a trustworthy AI system in healthcare: deep learning based skin lesion classifier. Front Hum Dyn. Jul 13, 2021;3:96. [CrossRef]
- Gotz D, Borland D. Data-driven healthcare: challenges and opportunities for interactive visualization. IEEE Comput Grap Appl. May 2016;36(3):90-96. [CrossRef]
- Drake RE, Cimpean D, Torrey WC. Shared decision making in mental health: prospects for personalized medicine. Dialogues Clin Neurosci. 2009;11(4):455-463. [FREE Full text] [CrossRef] [Medline]
- Braun M, Hummel P, Beck S, Dabrock P. Primer on an ethics of AI-based decision support systems in the clinic. J Med Ethics. Apr 03, 2020;47(12):e3. [FREE Full text] [CrossRef] [Medline]
- Ferrara E. The butterfly effect in artificial intelligence systems: implications for AI bias and fairness. Mach Learn Appl. Mar 2024;15:100525. [CrossRef]
- Schwartz R, Schwartz R, Vassilev A, Greene K, Perine L, Burt A, et al. Towards a standard for identifying and managing bias in artificial intelligence. National Institute of Standards and Technology. URL: https://www.nist.gov/publications/towards-standard-identifying-and-managing-bias-artificial-intelligence [accessed 2024-04-29]
- Magrabi F, Ammenwerth E, McNair JB, de Keizer NF, Hyppönen H, Nykänen P, et al. Artificial intelligence in clinical decision support: challenges for evaluating AI and practical implications. Yearb Med Inform. Aug 2019;28(1):128-134. [FREE Full text] [CrossRef] [Medline]
- Byrne MF, Parsa N, Greenhill AT, Chahal D, Ahmad O, Bagci U. Making AI work: designing and evaluating AI systems in healthcare. In: Byrne MF, Parsa N, Greenhill AT, Chahal D, Ahmad O, Bagci U, editors. AI in Clinical Medicine: A Practical Guide for Healthcare Professionals. Hoboken, NJ. John Wiley & Sons; 2023.
- Nair M, Svedberg P, Larsson I, Nygren JM. A comprehensive overview of barriers and strategies for AI implementation in healthcare: mixed-method design. PLoS One. 2024;19(8):e0305949. [FREE Full text] [CrossRef] [Medline]
- Mahadevaiah G, Rv P, Bermejo I, Jaffray D, Dekker A, Wee L. Artificial intelligence-based clinical decision support in modern medical physics: selection, acceptance, commissioning, and quality assurance. Med Phys. Jun 17, 2020;47(5):e228-e235. [FREE Full text] [CrossRef] [Medline]
- Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121-127. [FREE Full text] [CrossRef] [Medline]
- Ferrario A, Gloeckler S, Biller-Andorno N. AI knows best? Avoiding the traps of paternalism and other pitfalls of AI-based patient preference prediction. J Med Ethics. Mar 2023;49(3):185-186. [CrossRef] [Medline]
- Lorenzini G, Arbelaez Ossa L, Shaw DM, Elger BS. Artificial intelligence and the doctor-patient relationship expanding the paradigm of shared decision making. Bioethics. Jun 2023;37(5):424-429. [CrossRef] [Medline]
- Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. Oct 25, 2019;366(6464):447-453. [CrossRef] [Medline]
- Göndöcs D, Dörfler V. AI in medical diagnosis: AI prediction and human judgment. Artif Intell Med. Mar 2024;149:102769. [FREE Full text] [CrossRef] [Medline]
- Rebelo AD, Verboom DE, dos Santos NR, de Graaf JW. The impact of artificial intelligence on the tasks of mental healthcare workers: a scoping review. Comput Human Behav. Aug 2023;1(2):100008. [CrossRef]
- Alhuwaydi A. Exploring the role of artificial intelligence in mental healthcare: current trends and future directions - a narrative review for a comprehensive insight. Risk Manag Healthc Policy. 2024;17:1339-1348. [FREE Full text] [CrossRef] [Medline]
- Amann J, Vetter D, Blomberg SN, Christensen HC, Coffee M, Gerke S, et al. To explain or not to explain?-artificial intelligence explainability in clinical decision support systems. PLOS Digit Health. Feb 17, 2022;1(2):e0000016. [FREE Full text] [CrossRef] [Medline]
- Reddy YJ, Sudhir PM, Manjula M, Arumugham SS, Narayanaswamy JC. Clinical practice guidelines for cognitive-behavioral therapies in anxiety disorders and obsessive-compulsive and related disorders. Indian J Psychiatry. 2020;62(8):230. [CrossRef]
- Lincoln T, Jung E, Wiesjahn M, Schlier B. What is the minimal dose of cognitive behavior therapy for psychosis? An approximation using repeated assessments over 45 sessions. Eur Psychiatry. Oct 23, 2016;38:31-39. [CrossRef] [Medline]
- Rollwage M, Habicht J, Juechems K, Carrington B, Viswanathan S, Stylianou M, et al. Using conversational AI to facilitate mental health assessments and improve clinical efficiency within psychotherapy services: real-world observational study. JMIR AI. Dec 13, 2023;2:e44358. [FREE Full text] [CrossRef] [Medline]
- Rollwage M, Habicht J, Juchems K, Carrington B, Hauser TU, Harper R. Conversational AI facilitates mental health assessments and is associated with improved recovery rates. BMJ Innov. Mar 01, 2024;10(1-2):4-12. [CrossRef]
- Li H, Zhang R, Lee YC, Kraut RE, Mohr DC. Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. NPJ Digit Med. Dec 19, 2023;6(1):236. [FREE Full text] [CrossRef] [Medline]
- Alfano L, Malcotti I, Ciliberti R. Psychotherapy, artificial intelligence and adolescents: ethical aspects. J Prev Med Hyg. Dec 2023;64(4):E438-E442. [FREE Full text] [CrossRef] [Medline]
- Saqib H, Sumyia AS, Saqib A. AI chatbots and psychotherapy: a match made in heaven? J Pak Med Assoc. Oct 30, 2023;73(11):2321. [CrossRef]
- Balcombe L. AI chatbots in digital mental health. Informatics. Oct 27, 2023;10(4):82. [CrossRef]
- Denecke K, Abd-Alrazaq A, Househ M. Artificial intelligence for chatbots in mental health: opportunities and challenges. In: Househ M, Borycki E, Kushniruk A, editors. Multiple Perspectives on Artificial Intelligence in Healthcare: Opportunities and Challenges. Cham, Switzerland. Springer; 2021:115-128.
- Jabir AI, Martinengo L, Lin X, Torous J, Subramaniam M, Tudor Car L. Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments. J Med Internet Res. Apr 19, 2023;25:e44548. [FREE Full text] [CrossRef] [Medline]
- Dean E, Söderlund A. What is the role of lifestyle behaviour change associated with non-communicable disease risk in managing musculoskeletal health conditions with special reference to chronic pain? BMC Musculoskelet Disord. Apr 13, 2015;16:87. [FREE Full text] [CrossRef] [Medline]
- Benzo RP. Self-management programs and the pursuit of behavior change. Respir Care. May 28, 2024;69(6):678-685. [CrossRef] [Medline]
- Cole-Lewis H, Ezeanochie N, Turgiss J. Understanding health behavior technology engagement: pathway to measuring digital behavior change interventions. JMIR Form Res. Oct 10, 2019;3(4):e14052. [FREE Full text] [CrossRef] [Medline]
- Kok G, Gottlieb NH, Peters GJ, Mullen PD, Parcel GS, Ruiter RA, et al. A taxonomy of behaviour change methods: an intervention mapping approach. Health Psychol Rev. Sep 2016;10(3):297-312. [FREE Full text] [CrossRef] [Medline]
- Agbadjé TT, Elidor H, Perin MS, Adekpedjou R, Légaré F. Towards a taxonomy of behavior change techniques for promoting shared decision making. Implement Sci. Aug 20, 2020;15(1):67. [FREE Full text] [CrossRef] [Medline]
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. Aug 2013;46(1):81-95. [FREE Full text] [CrossRef] [Medline]
- Marlatt GA. Taxonomy of high-risk situations for alcohol relapse: evolution and development of a cognitive-behavioral model. Addiction. Dec 1996;91(s12):S37-S49. [CrossRef]
- Guven E, Gokce G. How effective are efficiency studies?: an evaluation in the context of cognitive behavioral therapies. Turk Psikiyatri Derg. 2018;29(1):21. [CrossRef]
- Scheinker D, Brandeau ML. Implementing analytics projects in a hospital: successes, failures, and opportunities. INFORMS J Appl Anal. May 2020;50(3):176-189. [CrossRef]
- Sendak MP, Ratliff W, Sarro D, Alderton E, Futoma J, Gao M, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Med Inform. Jul 15, 2020;8(7):e15182. [FREE Full text] [CrossRef] [Medline]
- Makoul G, Clayman ML. An integrative model of shared decision making in medical encounters. Patient Educ Couns. Mar 2006;60(3):301-312. [CrossRef] [Medline]
Abbreviations
AI: artificial intelligence |
HCI: human-computer interaction |
HOT-fit: human, organization, and technology-fit |
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews |
RQ: research question |
SDM: shared decision-making |
Edited by T de Azevedo Cardoso; submitted 26.06.24; peer-reviewed by S Mahed Mousavi, RA El Arab, D Patel; comments to author 19.09.24; revised version received 28.10.24; accepted 25.11.24; published 24.01.25.
Copyright©Hassan Auf, Petra Svedberg, Jens Nygren, Monika Nair, Lina E Lundgren. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 24.01.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.