Characteristics and Outcomes of mHealth Interventions in Psychosis: Systematic Mapping Review

Introduction

Psychotic disorders are a group of serious mental illnesses consisting of “abnormalities in one or more of the following five domains: delusions, hallucinations, disorganized thinking (speech), grossly disorganized or abnormal motor behavior (including catatonia), and negative symptoms” [1]. The global burden of psychotic disorders on individuals, communities, and health care systems cannot be understated. Schizophrenia, the most well-known psychotic disorder, has been ranked among the top 20 causes of years lived with disability (YLDs) among all illnesses and injuries and afflicts 23.6 million individuals worldwide [2]. Schizophrenia also has an early age of onset in adolescence [3] and has a propensity for relapsing. Such acute relapses can leave deleterious impacts on the individual yet are difficult to predict. The chronic, relapsing-remitting illness trajectory of schizophrenia not only contributes to a high disability weight during acute psychotic episodes [4] but also incurs hefty societal costs—upwards of US $300 billion in the United States alone [5].

The debilitating nature of psychosis thus spells the need for scalable, cost-effective, and accessible solutions to augment traditional psychiatric management, which remains underpinned by intensive human-delivered care. Mobile health (mHealth), defined as the use of wireless mobile technologies for public health [6], stands out as an emerging possibility. This is especially so with smartphone ownership becoming more pervasive in the 21st century, including among persons living with psychotic disorders [7]. In an era where different smart devices permeate everyday life, mHealth can enable data-driven assessment of individuals’ lifestyles and well-being. The repertoire of personalized mHealth interventions has similarly been expanding, ranging from medication adherence tools [8] to internet-based cognitive behavioral therapy (iCBT) [9]. mHealth interventions, therefore, harbor much optimism in empowering persons living with psychosis toward proactive self-care with timely symptom management and targeted interventions [10].

With the exponential increase in popularity of mHealth in recent years [11], there is substantial interest in evaluating the efficacy of mHealth interventions for adult patients with psychosis. However, to the best of our knowledge, there are no systematic reviews evaluating mHealth interventions for adult patients with psychosis. A systematic meta-review of mHealth interventions for mental health in general [12] found that none of the included meta-analyses studied their effects on psychotic disorders. There have been a few reviews looking into mHealth technologies for psychosis, but these focused on the scope of technologies rather than the outcomes these technologies seek to achieve [13-15]. Reviews attempting to quantify the impact of mHealth interventions on psychosis outcomes included few articles at the time of publication [16], which may no longer represent the current body of mHealth interventions. Furthermore, Firth and Torous [17] assessed the impact of mHealth intervention only for feasibility, while Clarke et al [18] focused on their effectiveness in reducing psychotic symptoms without assessing other patient-centric outcomes.

It is therefore important to critically review prevailing outcome measures used to evaluate mHealth interventions for adults with psychosis, whether in terms of traditional clinical outcomes (eg, relapse prevention, reduction of hospitalization, or mortality) or person-centric attributes (eg, quality of life, subjective well-being, and various psychological constructs). This can be achieved through a systematic mapping review to “collate, describe and catalogue available evidence” while following “the same rigorous, objective and transparent processes as do systematic reviews” [19]. Although systematic reviews of randomized controlled trials (RCTs) are the perceived “gold standard,” there are significant hurdles in doing so, given the heterogeneity of mHealth interventions and study designs and lack of agreement over choices of outcomes and their measurement instruments. Against this backdrop, this systematic mapping review seeks to answer the following questions: (1) What are the characteristics of mHealth interventions for adults with psychosis? (2) What type of outcomes are assessed and reported in RCTs of mHealth interventions for adults with psychosis?

Methods

Overview

This mapping review was performed according to the methodology proposed by James et al [19] and reported in line with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) reporting guidelines (Multimedia Appendix 1). The protocol was registered in Open Science Framework Registries [20] in May 2023.

Setting Inclusion Criteria for Studies

Studies included in this mapping review were randomized controlled studies of any design, reporting the use of mHealth interventions for adults with psychosis. Our detailed inclusion and exclusion criteria are reflected in Textbox 1.

Searching for Evidence

An electronic literature search was performed on September 20, 2022, across PubMed, CINAHL (EBSCO), PsycINFO (EBSCO), Embase (Ovid), and Cochrane Library. We included English language reports published from January 1973 onwards, aligning with the advent of mobile technologies. The search strategy was developed in PubMed and adapted to other databases in consultation with a medical librarian. Search terms included a comprehensive list of words and phrases representing the intersection between mHealth interventions and psychosis (Multimedia Appendix 2). During the screening process, the citations of reports identified from bibliographical databases were also searched to include any other eligible publications that met all the inclusion criteria.

Screening Evidence

The search results from all databases were imported into a single EndNote (version 20; Clarivate) library, and duplicate records were removed. Subsequently, reviewers worked in pairs to select studies independently and in parallel using the online screening tool Covidence. This was performed in 2 stages: initial screening of title and abstract, followed by a second round of full-text screening. Discrepancies in any screening stage were resolved through a stepwise approach of mutual discussion, followed by engaging a third reviewer. The search and screening process was documented in a study selection flowchart [21].

Coding

Data were coded using a standardized data extraction form on Microsoft Excel developed by the review team. Variables that were extracted included the following: study methodology, participant baseline characteristics, study setting, characteristics of mHealth intervention and delivery, choice of outcomes, and corresponding measurement instruments. The data extraction form was piloted in 2 studies and amended based on feedback before it was used for data extraction. Furthermore, reviewers met up regularly to ensure concordance in the data extraction process. Like the screening stage, reviewers worked in pairs to extract data from included studies independently and in parallel. The extracted data was compared, and any discrepancies were resolved through mutual discussion or involving a third reviewer acting as the arbiter.

Describing and Visualizing the Findings

Data were visualized in a diagrammatic or tabular form accompanied by a narrative summary. Descriptive statistics were used to ascertain the choice of outcomes and measurement instruments reported in the included studies. Furthermore, data were classified and mapped according to the delivery medium (the digital platform used, eg, app, website, SMS), delivery channel (the physical device used, eg, smartphone, tablet), and delivery format. Based on recommendations by Lattie et al for digital mental health interventions, we modified their approach to subclassify delivery formats of mHealth interventions within this study based on the level of human support incorporated. Whereas self-guided interventions were defined to be fully automated, we consider guided interventions to “include human support as part of their delivery” (whether dyssynchronous or synchronous), while blended interventions involve the delivery of the digital modules “as part of face-to-face mental health interventions” [22].

Outcomes were grouped into 8 categories, namely severity of psychosis-related symptoms, functioning, well-being, medication adherence, adverse events, user experience, technical, and all other outcomes reported. These were derived directly or indirectly based on the International Consortium for Health Outcomes Measurement (ICHOM) patient-centered outcome measures for psychosis (eg, symptoms and functioning) [23], as well as the Core Outcome Measures in Effectiveness Trials (COMET) Initiative’s 38-item medical research outcome taxonomy (eg, 28: Emotional functioning/well-being, 32: Delivery of care, 38: Adverse events/effects) [24]. Instruments measuring any of these outcomes were also classified as objective or subjective, in line with the COSMIN (Consensus-based Standards for the selection of health Measurement INstruments) definition. [25] Objective measures include data passively obtained from phone usage or embedded sensors. Subjective measurement instruments include either observer-rated or self-report questionnaires that can be written or administered digitally, whether regarding their health status or experience using the mHealth intervention.

Results

Overview of Search Strategy

The initial search of databases yielded 2537 papers, of which a total of 1699 titles and abstracts were screened after removing duplicate records. Thereafter, 98 reports were assessed in full for eligibility, including 4 additional records, which were retrieved by searching the reference list of papers screened. This culminated in 29 publications reporting 23 studies being included in this review (PRISMA flow diagram in Figure 1).

‎

Characteristics of Included Studies

All included studies were published from 2017 onwards. Of these, nearly three-quarters of the included studies (17/23, 74%) were published from 2020 onwards [26-44]. Based on the 2021 World Bank Country and Landing Groups classification system, all except 2 studies [36,43] were conducted in high-income countries; 8/23 (35%) of the studies were conducted in the United States [32,38,44-52] and another 10/23 (43%) of the studies (including multicenter studies) were conducted in Europe [26,28,29,33-35,37,39-41,53,54]. The majority of the studies (16/23, 70%) were conducted in the outpatient and community setting [26,27,30-33,35-37,39-42,45-49,53,54]. Study designs included mostly RCTs with a variety of comparisons used. Particularly, two of the studies were 3-arm RCTs that featured both positive and negative controls [41,45].

There was also a mix of psychiatric diagnoses among recruited participants of the included studies. While over half (13/23, 57%) of included studies recruited only participants with psychotic disorders [26,28-32,34,36,39,40,42-44,47-49,52,53], 6 studies also recruited individuals with mood disorders (such as bipolar disorder or major depressive disorder) [27,33,38,45,46,50,51] and 1 study [35] included individuals with “Ultra-High Risk” of psychosis. By the American Psychiatric Association’s definition of adulthood, the mean participant age of most studies (16/23, 70%) 26-34,38,41-50,52] corresponded to middle adulthood (35 years and older) while that of remaining studies corresponded to young adulthood (35 years and younger). Table 1 presents a summary of the characteristics of the included studies, and further details can be found in Multimedia Appendix 3.

Characteristics of mHealth Interventions

A wide range of features were incorporated into the mHealth interventions investigated in included studies, spanning from psychological therapy (12/23, 52%) [26,28,29,32,38,40,43,45-49,52-54], psychoeducation (8/23, 35%) [32,36,38,40,42,45,50-53], active self-monitoring (8/23, 35%) [27,30,31,33-35,37,43,50,51], medication adherence (7/23, 30%) [33,34,36,40,41,44,45,53], personalized recommendations on coping strategies (5/23, 22%) [27,30,31,33-35] to peer support (3/23, 13%) [32,43,52]. Different types of psychological therapy were used, such as cognitive behavioral therapy (CBT) [26,28,29,32,45,46,52,54], cognitive training [38,40,53], social cognition training [43], or a mix of techniques [47-49]. Less frequently incorporated features included emergency assistance [33], behavioral activation and mindfulness [52], shared decision-making [39], and teleconsultations [40,53].

To further characterize the type of mHealth interventions, these features were, in turn, grouped into distinct themes based on the pillars of Engel’s biopsychosocial model [55]: medication-related features (eg, medication reminders, teleconsultations) representing the biological aspect, psychological therapy, and social support (eg, peer support, emergency assistance). A fourth intervention type—self-management—was also identified, encompassing psychoeducation, active self-monitoring and personalized recommendations on coping strategies. This was found to be a highly prevalent modality targeted by mHealth interventions (16/23, 70%) [27,30-38,40,42,43,45,50-53]. Many mHealth interventions were found to be multimodal, combining features targeting multiple pillars of Engel’s biopsychosocial model (10/23, 43%) [32-34,36,38,40,43,45,52,53]. Even among unimodal interventions, a combination of multiple features was commonly used, such as active symptom monitoring with correspondingly tailored coping strategies [27,34,35], or medication reminders along with teleconsultations [40,53]. Of these, there was an even spread between self-management, psychological therapy, and medication or treatment-related interventions without any interventions that exclusively delivered social support.

The majority of mHealth interventions were delivered through applications (14/23, 61%) as the main medium [27,30,31,34,35,37-42,46,50-54], and smartphones (17/23, 74%) as the main channel of delivery [27,30,31,34-45,50-54]. Most of the interventions were also blended with in-person sessions (11/23, 48%) [26,27,32,33,35,39,40,42-45,53] or guided remotely (6/23, 26%) [28-31,36,37,50-52] by persons such as health care providers or trained peer supporters (Table 2).

Outcome Measures

The most common primary outcome was the severity of psychosis-related symptoms, which was chosen in 11/23 (48%) of included studies [26-29,32,34,35,37,40,50-53]. This was also the most prevalently measured outcome overall, being reflected in almost all (21/23, 91%) included studies [26-32,34-43,45-54]. Notably, psychosis-related symptoms were consistently reported in conjunction with other outcomes, such as well-being (17/23, 74%) [26-33,36-39,42,43,46-52,54], user experience (15/23, 65%) [27-31,33,34,37-40,43,46-54] and functioning (12/23, 52%) [34,35,37-39,41-43,45,47-49,52,54].

Figure 2 illustrates the choice of outcome measures and their relative frequencies in included studies. The most frequent measures of psychosis-related symptoms were positive symptoms (13/23, 57%) [26,27,34,35,37-40,42,50-54], mood and anxiety (10/23, 43%) [26,27,34,35,37,38,40,50-54], and overall psychopathology severity (9/23, 39%) [30,31,35,40-43,45,47-51,53]. Less commonly were negative symptoms (7/23, 30%) [34,35,39,40,45,47-49,52,53], other psychosis-related symptoms such as rehospitalization, relapse, and insight (7/23, 30%) [28-32,36,39-41,53], and cognitive symptoms (3/23, 13%) [26,46-49]. Measurement instruments for positive symptoms were highly varied with a mix of observer-rated questionnaires [27-29,35,37,39,40,42,50-54], self-report questionnaires [26,28,29,34,38], and ecological momentary assessments (EMAs) [35] to gauge aspects such as overall positive symptom severity, hallucinations, paranoia, and intensity and distress of psychotic experiences. Similar diversity was observed in measurement instruments for negative symptoms, including overall negative symptom severity [34,35,39,40,53], motivation [47-49,52], and defeatist beliefs [45,52]. On top of self-report questionnaires used to rate mood and anxiety symptoms, EMAs were also leveraged to measure momentary mood [34,35]. Finally, overall psychopathology severity was uniformly observer-rated.

‎

User experience was another commonly measured outcome, with the main domains of interest being feasibility (7/23, 30%) [27,30,31,34,37,40,46,52,53], acceptability (7/23, 30%) [27,30,31,37-39,52,54], and engagement (6/23, 26%) [30,31,38,43,47-51,54]. Feasibility was mostly understood as an objective construct among studies that reported it [27,31,34,37,52,54], with the main measurement instrument being passive data collected from devices. Interpretation of acceptability was slightly more pleomorphic, with most studies using self-report questionnaires to ascertain participants’ views on the intervention [27,30,31,38,39,52,54] and some using objective phone data [37,39] and study retention rates [52]. Of the studies measuring engagement, most studied objective app engagement [30,31,38,40,43,45,50-54], although there were instances of measuring attendance of physical sessions accompanying the mHealth intervention [43,50,51] or subjective service engagement [30,31]. Other measures of user experience include satisfaction, usability, adherence, perceived usefulness, user-friendliness, and negative experiences.

Well-being was uniformly measured through self-report questionnaires. Although quality of life was the most measured construct of well-being (10/23, 43%) [26,28-33,37,43,47-51,54], there was marked diversity in the choice of outcomes within this category, including self-efficacy, recovery, social engagement and support, empowerment, hope, self-esteem, emotional distress, stigma, use of coping strategies and metacognitive beliefs. On the other hand, functioning was mostly measured with observer-rated questionnaires, though other types of measurement instruments were also used [34,35,38,43]. Most instruments measured global (7/23, 30%) [31,35,38,39,41,45,54] and social functioning (6/23, 26%) [34,35,39,42,43,54]. Almost all studies measuring medication adherence used the self-reported Medication Adherence Rating Scale [30,31,33,44,54] except for one study using the Medication Adherence Questionnaire [36] and another collecting data from sensors embedded in medication bottle caps [41]. Contrary to most outcomes, which were compared before and after the intervention, adverse events and technical outcomes were predominantly monitored throughout the study as and when they arose [26-31,37,40,43,45,46,53,54].

Choices of outcomes remained multidimensional even after stratifying by the type of intervention studied, as displayed in Figure 3. The severity of psychosis-related symptoms, functioning, and medication adherence were categories of reported outcomes that were common to all intervention types. In contrast, well-being, adverse events, and user experience outcomes were outcome choices common to psychotherapy-only, self-management-only, and mixed modality interventions but not reported in exclusively medication-related mHealth interventions for psychosis. Other outcomes refer to intervention-specific outcomes that cannot be encompassed in the other categories, such as participants’ recall of specific coping heuristics, auditory processing, and motivational incongruence.

‎

Discussion

Principal Findings

In this study, we rigorously reviewed the choice of outcomes and measurement instruments in randomized controlled studies evaluating the use of mHealth interventions for adults with psychosis in relation to intervention and control characteristics. The recency of the 23 included studies, particularly the surge in studies published from 2020 onwards, is concordant with the dramatic growth of research in digital mental health in general [12,56], with the COVID-19 pandemic probably being a key catalyst of further development [57,58]. It is encouraging that most mHealth interventions studied are tailored toward outpatient and community care, where sustaining accessibility to mental health services remains a challenge: World Health Organization statistics reveal that almost one-third of countries worldwide still have fewer than 1 psychiatrist in the workforce per 100,000 individuals [59]. The emergence of transdiagnostic mHealth interventions also aligns with a previous systematic review [60], which found a marked increase in studies on mental health applications capable of benefitting multiple groups of patients with different diagnoses. The focus on participants in mid-adulthood may impress upon the possibility of mHealth to manage psychosis in the long term. That being said, younger populations may arguably be a bigger benefactor of mHealth interventions with more pervasive smartphone use and lower engagement with traditional health services [61,62].

The findings of this mapping review strongly suggest that mHealth interventions for adults with psychosis are complex and typically multidimensional in nature. When reviewing the intervention features in isolation, psychotherapy was the most frequently deployed intervention, followed by psychoeducation and active symptom monitoring. Unsurprisingly, CBT was the most common type of psychotherapy delivered, in keeping with its established evidence base and showing its translation toward the mobile realm [63]. Upon further classifying intervention features by domain, most mHealth interventions for psychosis either traversed multiple pillars of Engel’s biopsychosocial model or minimally combined multiple features within the same pillar. Notably, supporting users’ self-management was found to be the most common single-mode intervention. With previous evidence demonstrating how self-management can improve outcomes of people living with severe mental illness [64], this corroborates the role of mHealth interventions in helping users navigate the chronic trajectory of psychosis. Furthermore, we found that the most common medium and channel of delivery were applications and smartphones. This is within expectations given the growing penetration of smartphone usage day-to-day and smartphone uptake among individuals with psychosis [65].

Outcomes reported were similarly numerous, comparable with findings from previous literature [14]. The severity of psychosis-related symptoms—the most reported outcome—was always measured together with other patient-centric measures such as well-being, user experience, or functioning. This bears testament to the transition toward patient-centered care [66] in both the physical and digital spheres and the ensuing importance of evaluating user-related factors [60,67,68] in adopting digital mental health tools. Overall, there needs to be better concordance between measurement instruments of the same outcome categories. We observed multiple studies using mHealth-enabled nascent tools to pick up objective data, such as EMAs [27,30,31,34,35,37,43,50,51] and embedded sensors [41]. This is exciting for refining our understanding of how users’ clinical pictures evolve temporally, but it would require proper validation against traditional measurement instruments [17,69]. Furthermore, included studies vastly differed in the choice and modality of questionnaires, whether observer-rated or self-reported, for variables including hallucinations and social functioning. This might further hinder the reproducibility of results, on top of the myriad of control designs in included studies. Such heterogeneity may directly impact subsequent evaluations of efficacy—for instance, Goldberg et al [12] noted that effect sizes of mHealth-based mental health interventions were smaller when compared with controls with therapeutic intent.

Interestingly, our review dispels the notion that the heterogeneity of outcomes directly reflects intervention diversity—in fact, the spectrum of reported outcomes remained broad despite stratifying by intervention type. This may put into question whether deciding outcomes to be studied based on interventions’ putative mechanisms (like what the ecological interventionist-causal model approach proposes [70]) might still apply when evaluating mHealth interventions. Taken from another perspective, this finding might just be another portrayal of how multifaceted mHealth interventions can be with its integration of multiple components. Their effects may transcend illness severity to simultaneously impact patients’ experiences of their illnesses. In turn, this would strengthen the impetus for holistically assessing clinical and functional outcomes when evaluating mHealth interventions for psychosis. Overall, the plethora of interventions, comparisons, outcomes, and outcome measurement instruments reinforces the need for more high-quality studies with standardized controls and evaluation frameworks to enable validity in further data synthesis. Considering the complexity of between-feature interaction and synergistically affected patient outcomes, the uncertainty still lingers if RCTs are the best evaluation framework for mHealth interventions despite being conventionally perceived as the gold standard of interventional trials.

Strengths and Limitations

This systematic mapping review has notable strengths. We used a comprehensive search strategy across 5 major bibliographic databases to identify publications reporting on the use of mHealth interventions in adults with psychosis. We have also attempted to minimize missing eligible records by searching the references of included studies. A robust methodology to screen identified articles and perform data extraction in pairs was used to ensure reliable review findings.

There are also some limitations. While including only randomized controlled study designs allowed greater strength in interpreting effects on outcomes, we may not have included nascent interventions in the early stages of development in the process. Furthermore, some potentially relevant studies may have been omitted due to the gamut of terms used in this area.

Conclusions

mHealth interventions for psychosis should be considered complex interventions probing multiple social and behavioral determinants of health. Randomized controlled studies in the field often report a remarkable breadth of outcomes regardless of the domains they seek to target. Marked variation in the choice of measurement instruments for said outcomes and comparison groups persist amongst studies included in this review. Hence, we recommend a holistic evaluation approach combining clinical and patient-centric outcomes to adequately account for the assortment of mHealth interventions and outcome measures. This would advance our understanding of individuals’ interactions with mHealth interventions that can enhance the person-centered delivery of mental health care for people with psychosis.