This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289) mHealth for Wellness, Behavior Change and Prevention (2871) Mobile Health in Psychiatry (426) Reviews in Digital Mental Health (243) Mental Health Issues in Adolescence (442)

Published on in Vol 23, No 1 (2021): January

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/16490, first published .
Smartphone-Based Interventions and Internalizing Disorders in Youth: Systematic Review and Meta-analysis

Smartphone-Based Interventions and Internalizing Disorders in Youth: Systematic Review and Meta-analysis

Smartphone-Based Interventions and Internalizing Disorders in Youth: Systematic Review and Meta-analysis

Authors of this article:

Adrian Buttazzoni1 Author Orcid Image ;   Keshbir Brar2 Author Orcid Image ;   Leia Minaker1 Author Orcid Image
Article Authors Cited by (20) Tweetations (4) Metrics

    Review

    1School of Planning, University of Waterloo, Waterloo, ON, Canada

    2School of Public Health and Health Systems, University of Waterloo, Waterloo, ON, Canada

    Corresponding Author:

    Adrian Buttazzoni, MSc

    School of Planning

    University of Waterloo

    200 University Avenue W

    Waterloo, ON, N2L 3G1

    Canada

    Phone: 1 519 888 4567 ext 46564

    Email: anbuttazzoni@uwaterloo.ca


    Background: Mental health disorders in youth are a global issue that have important implications for the future quality of life and morbidity of affected individuals. In the context of public health initiatives, smartphone-based interventions have been suggested to hold the potential to be an effective strategy to reduce the symptoms of mental health disorders in youth; however, further evaluation is needed to confirm their effectiveness. This systematic review and meta-analysis documents and synthesizes existing research on smartphone-based interventions targeting internalizing disorders in youth populations.

    Objective: This study aims to synthesize existing research on smartphone-based interventions targeting internalizing disorders in youth populations.

    Methods: PubMed and SCOPUS were searched in 2019, and 4334 potentially relevant articles were found. A total of 12 studies were included in the final synthesis. We used the Hedges g meta-analysis approach and a random effects model for analysis.

    Results: The results of this review note that depression and anxiety are the most commonly targeted symptoms, and unlike other similar topics, most studies reviewed were linked to a proven treatment. The overall pooled effect from the meta-analysis showed small but significant effects (κ=12; N=1370; Hedges g=0.20; 95% CI 0.02-0.38) for interventions in reducing the symptoms of internalizing disorders. In total, 4 subgroup analyses examining specific symptoms and intervention styles found varied small significant and nonsignificant effects.

    Conclusions: Future research should focus on developing robust evaluative frameworks and examining interventions among more diverse populations and settings. More robust research is needed before smartphone-based interventions are scaled up and used at the population level to address youth internalizing disorders.

    J Med Internet Res 2021;23(1):e16490

    doi:10.2196/16490

    Keywords

    mental health; meta-analysis; mobile phone; smartphone; systematic review; youth 


    Globally, between 10% and 20% of children and adolescents experience mental illness, with approximately half of all mental illnesses known to begin by the age of 14 years [1]. Poor mental health during these developmental years has been linked with unfavorable outcomes regarding employment, relationships, family formation, and health and disability in early adulthood [2]. Childhood and adolescence are key life stages for interventions with regard to mental health and well-being. However, at present, the screening and support for youth mental health issues in many primary health care systems continue to be inadequate, as even optimistic estimates suggest that only 50% of illnesses are detected by doctors [3]. New and innovative strategies are needed to help address mental health issues in youth. Smartphone-based interventions have been increasingly employed to track symptoms and provide support for individuals on a range of related health issues, such as concussions [4]. Taking this potential and the relatively high digital and tech literacy of younger populations into consideration, there has been a recent growth in the interest of delivering mental health interventions to youth cohorts via smartphones [5].

    Internalizing disorders are those in which individuals tend to express distress inwardly, such as anxiety disorders and mood disorders (eg, major depressive disorder [6]). Anxiety is one of the most common disorders in young people and is estimated to affect 4% to 20% of children and adolescents [7], whereas depression is said to affect 2% to 8% of children and adolescents [8] and is a leading cause of disability globally [9]. Similarly, stress is often internalized and can significantly impact an individual’s coping abilities, self-esteem, and social relations [10], whereas insomnia (ie, sleep problems) has a robust relationship with depression [11] and other internalizing symptoms including anxiety [12]. Disorders can begin early in childhood and can develop into chronic conditions that negatively impact an individual’s relationships, development, and daily functioning in the near future [13]. These disorders are associated with functional impairment, increased risk of depression and suicide [14], and substance abuse issues [15] in the long term. As a result, internalizing disorders carry the potential for high societal burdens [16].

    Recently, smartphones have become an essential tool in the targeted support, management, and monitoring of mental health disorders. Common mental health intervention strategies using smartphones include text messaging services [17] and smartphone apps [18]. Growing evidence supports the feasibility and potential of smartphone-based interventions to address mental health issues and disorders [19]. For example, positive effects have been observed regarding memory training for older adults [20] and for people with attention-deficit or hyperactivity disorder [21]. Consequently, smartphones are being increasingly used to address mental health issues of youth and adolescent populations [22]. However, with an increase in the use of smartphones as an intervention delivery strategy, there is a need to improve the evaluation aspect of such interventions [23].

    Justification for Review and Meta-Analysis

    A recent systematic review of meta-analyses that focused on internet- and mobile phone–based interventions for mental and somatic conditions among children found 8 relevant meta-analyses [24]. Of the included papers, 5 focused on web-based or computerized interventions, one was primarily concerned with psychological-based interventions and one searched for but did not include phone-based interventions. The single meta-analysis that included smartphone-based interventions only analyzed one mobile phone–based intervention study [25]. Another recent meta-review [22] broadly cataloged and synthesized existing reviews of all types focusing on digital health interventions for young people with mental health problems. This review did not report any other meta-analyses (although 2 additional relevant scoping reviews and 1 additional systematic review were reported) beyond those documented by Domhardt et al [24]. To date, no meta-analysis has been conducted that has sought to quantitatively, and solely, evaluate the effectiveness of smartphone apps with regard to mental health (specifically, internalizing disorders) of youth populations. This review and meta-analysis are the first to aim to address these points and present an evaluation.

    Review Question and Objective

    The primary research question guiding this review and meta-analysis is as follows: What are the study designs, intervention features (review), and effectiveness (meta-analysis) of smartphone-based interventions that aim to minimize or reduce the symptoms of youth internalizing disorders? By applying the population, intervention, comparison, outcome, and context model of Petticrew and Roberts [26], we operationalized our meta-analysis research question as presented in Textbox 1.

    Population, intervention, comparison, outcome, and context review research question breakdown (criteria and description).
    • Population: youth (in general, <18 years old; however, in some articles, youth was defined as ≤24 years)
    • Intervention: smartphone-based interventions specifically targeting an internalizing disorder in a youth population
    • Comparison: control versus intervention, group 1 versus group 2, time 1 versus time 2, etc
    • Outcome of interest: effectiveness of intervention in reducing symptoms or intensity of internalizing disorder (ie, anxiety, depression, insomnia, stress)
    • Context: any
    Textbox 1. Population, intervention, comparison, outcome, and context review research question breakdown (criteria and description).

    Following this question, the principal objective of the meta-analysis is to quantitatively evaluate the effectiveness of smartphone-based interventions with regard to youth internalizing disorders.


    Search Strategy

    Searches of electronic databases were used to identify and document the articles presented in this review and the meta-analysis in June 2019. Different variations of, and other common terms used for, each focal concept (ie, smartphone, internalizing disorder, and youth population) were discussed and developed by the authors, whereas specific terms were truncated as necessary (Textbox 2).

    Search strategy outline.
    • Smartphone

    “cell phone” OR “cellular phone” OR “mHealth” OR “mobile health” OR “mobile phone” OR “phone” OR “SMS” OR “short message service” OR “smartphone” OR “text” AND

    • Internalizing disorder

    “anxiety” OR “depression” OR “internalizing disorder*” OR “internalising disorder*” OR “insomnia” OR “stress” AND

    • Youth population

    “adolescent*” OR “child*” OR “teen*” OR “youth” OR “young adult”

    Textbox 2. Search strategy outline.

    When deciding on which electronic databases to select for this review, we needed to ensure that content from the fields of behavioral science, pediatrics, psychology, and public health are captured. Therefore, to best incorporate this diversity in content and research areas, we conducted the search strategy in 2 interdisciplinary databases—PubMed and SCOPUS.

    Eligibility Criteria

    A total of 5 specific criteria were applied to the article search. The criteria stipulated that each article must (1) be focused primarily on the implementation and evaluation of a smartphone-based intervention (ie, not web-based strategies, social media focused initiatives, etc); (2) have reported some description of the study design and sample, as well as the intervention strategy, implementation, evaluation, and targeted outcome of the intervention strategy; (3) have been a primary research study (ie, not a review, feasibility or acceptability study, proposal, technical report etc) in the design of a randomized controlled trial (RCT), case control, cohort, or cohort analytic design; (4) have focused on and reported the outcome for one or more internalizing disorder outcomes (ie, anxiety, depression, insomnia, stress); and (5) be written in English. There were no geographic or publishing date restrictions placed on the search.

    Study Selection and Review Process

    Initial searches of PubMed (n=1726) and SCOPUS (n=2608) returned a total of 4334 results (Figure 1). Removal of duplicates resulted in 1200 titles being discarded. The remaining 3134 titles were vetted next, after which 2420 articles were excluded. Abstracts were then screened, resulting in 593 more records being removed. After full-text assessments of the final 121 articles were completed, a total of 11 papers were deemed to meet all of the eligibility criteria. Having included studies that primarily examined adolescents but were mixed with young adult populations, we decided to include 3 studies [27-29] with similar sample demographics but had a mean sample age >18 years and did not explicitly refer to their participants as adolescents or youth. The most common reasons for exclusion in the full-text assessment phase were feasibility studies lacking a formal assessment, proposal articles only presenting concepts, research papers not including a youth population, and primary studies not reporting an internalizing disorder outcome. Of the total, 1 paper [30] met all of the above conditions; however, its reporting of the outcome of interest was not precise enough to be included in the analysis. Reference list searches of the included articles added 1 additional article to the review. Eventually, 12 studies were documented and synthesized in the systematic review and subsequently included in the meta-analysis. Initial title and abstract vetting was conducted by a team member (AB) with a second member (LM) spot checking the abstract vetting to ensure the consistency of the process. Similar protocols were carried out for the later full-text assessment and data extraction phases (AB and LM).

    ‎
    Figure 1. Flow diagram of study selection processes.
    View this figure

    Systematic Review Data Extraction

    Systematic review data extractions and meta-analysis coding data for all the included articles are available in the Multimedia Appendix 1. The following information was gathered from each study: citation information, study design information (eg, primary outcome evaluated), smartphone-based intervention details (eg, supporting resources, delivery methods, length), sample details, and internalizing disorder variable information (eg, outcome measure used).

    Variables and Concepts Included in Meta-Analysis

    Internalizing disorders are often defined as any depressive and anxiety condition [31]. However, other definitions of internalizing disorders have expanded the concept to include panic and stress [32,33], whereas insomnia has become an important symptom of depressive and anxiety disorders [31]. Therefore, this meta-analysis conceptualized internalizing disorders as anxiety, depression, insomnia, or stress. Smartphone-based interventions were defined as any app, initiative, program, promotion, service, or tool that is based or run primarily through a smartphone.

    Meta-Analysis Coding

    In addition to the data extractions for the systematic review portion of this manuscript, identified studies pooled in the meta-analysis were also coded to include statistical data. On the basis of the availability of reported results from each study, statistics coded specifically for the meta-analysis included intervention effect sizes (eg, available between- or within-group d values and their corresponding SDs), results of t tests (eg, differences in means) in cases where no d values were reported, scale reliability scores, scale anchors, data source report (eg, self, other), and, if applicable, any information pertaining to multiple measures being employed in the same study.

    Meta-Analysis Approach and Statistical Procedures

    To guide the meta-analysis, we drew from the outline for effect size meta-analyses of Hunter and Schmidt [34] and chose to use the Hedges g approach. In traditional d value meta-analysis approaches, such as the bare bones method, no corrections are made for any artifacts other than sampling error [34]; however, given some of the small sample sizes of the included studies, we elected to follow previous recommendations [35] which stipulate using Hedges g as a correction for effect size—a method that pools weighted SDs unlike Cohen d. Similar to others [36], we elected to use a more conservative approach given the relatively small number of primary articles being used to estimate the pooled intervention effects. The reported or calculated effect size for the interventions was used to assess the between-group or within-group effectiveness of the interventions on reducing or minimizing internalizing disorder symptoms. Importantly, among the 8 between-group studies, 5 were randomized. Once the d values from each study (uncorrected effect sizes) were derived, we applied the Hedges g formula to calculate corrected effect sizes [35]. A random effects model was used for this meta-analysis because, unlike fixed effects models, it allows for potential variation in underlying parameters. As such, random effects models have been argued to more accurately reflect the real-world heterogeneity of effects [37].

    As a part of our conservative approach, all intervention follow-up values used in the meta-analysis were the last date (eg, 2-week and 4-week follow-ups; 4-week selected) reported in each study. Statistics calculated in the meta-analysis included the mean unweighted observed effect, mean effect size weighted by sample size, SD of the sample-weighted effect, corrected weighted variance of the effect (ie, Hedges g), and percentage of variance because of sampling error. We also calculated the same statistics for 4 separate subgroups (specific symptoms and intervention style—2 subgroups each) of the studies. In cases where multiple outcomes of interest were reported in a single study, we calculated the average standardized difference across variables to ascertain the overall effect size [34]. If no effect size data were provided, we followed the method by Morris and DeShon [38] of pooling pre- and posttest means and dividing them by the pretest SD to calculate effects. Sensitivity analyses following the Cochrane Handbook for Systematic Reviews of Interventions method [39] were used to calculate effect sizes for articles that only reported baseline and follow-up means (ie, no changes in means data) and to subsequently impute an SD of the change for the experimental group. All analyses were completed in Microsoft Excel.

    Quality and Risk of Bias Assessments

    Quality assessments (QAs) of the 12 studies reviewed and analyzed are shown in Table 1. To conduct the assessments, we used the Effective Public Health Practice Project (EPHPP) QA tool [40]. Global ratings for each study were developed following the EPHPP dictionary guidelines; 2 reviewers (AB and KB) first calculated independent scores for each article and subsequently evaluations were compared [41]. Comparisons of the evaluations were undertaken to address and resolve any grading variability and other interpretation differences between the 2 reviewers. Reviewers graded each article on its selection bias, study design, confounders, blinding, data collection, withdrawals, and dropouts. Ex-post comparisons of scores between the 2 separately completed individual assessments resulted in an interrater reliability for the QAs of over 90%. The final results of the QA examination were mixed with 4 articles possessing a strong global rating, 5 with a moderate rating, and 3 with a weak rating.

    Table 1. Quality assessment of the articles included in meta-analysis.
    Study (reference)Selection biasStudy designConfoundersBlindingData collectionWithdrawals and dropoutsGlobal ratinga
    Clarke et al 2016 [27]ModerateModerateModerateWeakStrongWeakWeak
    Grassi et al 2009 [28]ModerateModerateStrongWeakStrongWeakWeak
    Kauer et al 2012 [42]ModerateStrongStrongModerateStrongStrongStrong
    Lee et al 2013 [43]WeakStrongStrongModerateStrongWeakWeak
    Ranney et al 2016 [44]ModerateModerateStrongWeakStrongStrongModerate
    Ranney et al 2018 [45]ModerateStrongStrongModerateStrongStrongStrong
    Reid et al 2011 [46]ModerateStrongStrongModerateStrongModerateStrong
    Stallard et al 2018 [47]ModerateModerateStrongWeakStrongStrongModerate
    Takahashi et al 2019 [29]ModerateModerateStrongWeakStrongStrongModerate
    Werner-Seidler et al 2019 [48]StrongModerateStrongWeakStrongModerateModerate
    Whittaker et al 2017 [49]ModerateStrongStrongStrongStrongStrongStrong
    Worthen-Chaudhari et al 2017 [50]ModerateModerateModerateWeakStrongStrongModerate

    aPer the global rating system, strong: no weak ratings; moderate: 1 weak rating; and weak: 2 or more weak ratings.

    More comprehensive risk of bias assessments was also completed for each study included in this review and meta-analysis. A total of 2 separate tools were employed in this regard: nonrandomized intervention papers’ risk of bias was assessed using the ROBINS-I (Risk Of Bias In Non-randomised Studies-of Interventions) tool for nonrandomized studies of interventions [51], whereas randomized interventions’ (ie, RCTs) risk of bias was assessed using the Cochrane Collaboration tool for assessing risk of bias in randomized trials [52]. Risk of bias assessments of the nonrandomized interventions concluded with all 7 papers being scored as at a serious risk of bias overall, whereas evaluations of the randomized trials resulted in 4 of 5 papers being rated as at a low risk of bias. Regarding the former result, this trend was largely a consequence of the rating system stipulating that one rating of serious risk in any category will be scored with an overall serious risk of bias judgment for an article [51]. More specifically, the most common source of potential bias among nonrandomized studies was related to uncertainties regarding the blinding of assessors with regard to the allocation or intervention status of participants, which is not possible in single group interventions (4 out of 7 papers in this portion of the review). Visual illustrations of the results can be found in Multimedia Appendices 2 and 3.


    General Characteristics of Included Studies

    The characteristics of the 12 included studies are presented in Multimedia Appendix 1. In total, 5 studies were conducted in Australia or New Zealand, 3 in the United States, 2 in Europe, and 2 in Asia. Regarding the evaluated sample populations, 10 of the 12 studies examined between 20 and 120 participants (mean 114), 8 studies had a majority of female participants, 2 had an even split, and 2 had a majority of male participants. Depressive symptoms were the most common specific symptom addressed (in 9 studies), followed by anxiety symptoms (in 6 studies). Insomnia and stress symptoms were infrequently targeted and were only included in studies with multiple outcomes. Among the studies, 9 explicitly provided relevant details or numbers; no intervention had a retention rate lower than 70%, with 7 noting rates above 90%.

    Systematic Review

    Several interventions reported multiple follow-ups during the course of their evaluation. The range of follow-up periods varied from same day postoperation [43] to 12 months [49]. A number of different scales were used across studies to measure the internalizing disorder outcomes. In fact, the only scales used multiple times were the Beck Depression Inventory-II, Depression Anxiety Stress Scale, and Center for Epidemiological Studies–Depression Child. Most studies were conducted in a real-world setting, such as at home or in school locations. Other contexts included emergency rooms [44,45], medical clinics [50] or hospitals [43], and commuting settings [28]. A variety of different program delivery methods (eg, apps, monitoring programs, text messages, videos) were reported among the included studies. Only 3 studies [28,43,49] did not examine populations particularly affected by or at risk of internalizing disorders (eg, at risk diagnosed disorder, history of self-harm, etc).

    Furthermore, 8 of the 12 studies reported an empirically proven treatment or guideline on which their intervention was based. Cognitive behavioral therapy (CBT) was the most commonly reported treatment, with 5 studies noting its use. Other treatments listed were attention bias modification [27], emotional self-awareness [42], and positive psychology, social interaction, and gameful design [50]. Among the CBT-centered interventions, specific features included concepts such as emotional regulation [45], thought modification [44], photo libraries of positive memories and physical activities [47], video diary messages [49], and psychoeducation [48]. Documented intervention delivery strategies in the non-CBT studies included visual relaxation narratives [28], smartphone app games [43], stress monitoring [46], attention bias modification task completion exercises [27], daily reporting of moods, substance use, sleep, and activities [42], symptom frequency and severity tracking [50], and videos and positive messaging [29].

    Effectiveness of Interventions (Meta-Analysis)

    The meta-analysis examined smartphone-based interventions as 1 total group and based on primary outcome and intervention style. Table 2 shows the results of the overall and subgroup meta-analyses. The pooled unweighted sample effect sizes of all studies (κ=12; N=1370) reflect a small-to-moderate effect (d=0.40). When sample weights are added to calculate a more credible estimate, a smaller but significant mean effect size (d=0.20; 95% CI 0.02-0.38) was observed. The sampling error variance explains approximately 3.5% of the variance in this corrected estimate. The subgroup analysis of intervention studies primarily assessing anxiety symptoms (κ=6; n=322) had a greater sample-weighted standardized effects mean compared with the overall group and was statistically significant (d=0.42; 95% CI 0.00-0.83). Among a larger sample, interventions targeting depressive symptoms (κ=9; n=1102) had a notably smaller effect that was similarly significant (d=0.16; 95% CI 0.01-0.31). When analyzing the interventions based on their predominant style and features, the sample-weighted mean effect was significant and greater for the group of all other styles (eg, monitoring, relaxation, support, not CBT) of program delivery (κ=7; n=380; d=0.42; 95% CI 0.09-0.75). Interventions using CBT features had a very small weighted mean effect size (κ=5; n=990; d=0.11; 95% CI −0.06 to 0.28) but were nonsignificant.

    Table 2. Effectiveness of smartphone-based interventions on reducing youth internalizing disorder symptoms, overall, by primary targeted outcome, and by intervention style.
    AnalysisκaNbgdSDwgfS*ggVariance samplingh (%)95% CIi
    Total1213701140.400.200.070.110.040.02 to 0.38
    Symptom

    		Anxietyj6322540.430.420.240.320.080.00 to 0.83
    Depressionj911021220.280.160.030.060.030.01 to 0.31
    Intervention style
    CBTk-based59901980.190.110.030.050.02−0.06 to 0.28
    Other (eg, monitoring, support, relaxation)7380540.550.420.120.200.080.09 to 0.75

    aκ: number of studies.

    bN: total sample size for studies combined.

    c: average sample for studies combined.

    dg: unweighted mean Hedges g.

    e: sample-weighted mean effect size (ie, weighted average of Hedges g).

    fSDwg: SD of the sampled-weighted effects.

    gS*g: corrected weighted variance of the effect (Hedges g).

    hPercentage of variance sampling: percentage of variance because of sampling error (ie, average sampling error variance).

    i95% CI: CIs at 95% for sample-weighted mean.

    jStudies with multiple outcomes were included in the subgroup analysis of both anxiety and depression.

    kCBT: cognitive behavioral therapy.


    Principal Findings

    This meta-analysis reviewed the features and explored the effectiveness of smartphone-based interventions aimed at addressing youth internalizing disorders. The review findings indicated that the majority of interventions were grounded in proven treatments, that depressive and anxiety symptoms were the predominant outcomes measured and evaluated, and that all studies were conducted in developed nations. Across the 12 studies and 1370 participants included in the meta-analyses, and after being corrected for sample error variance, a small but significant pooled effect was observed with regard to reducing internalizing disorder symptoms. When examining subgroups based on specific symptoms and intervention styles, potentially because of small κ values, mixed results were observed among the 4 analyses.

    Findings from this review indicate that research conducted on the effectiveness of smartphone-based interventions for youth internalizing disorders appears to be in the nascent stages but is growing. A global review assessing the overall state of mobile health apps documented 3673 mobile phone apps but found that only 247 corresponded with published papers [53]. However, although evaluative studies are still being conducted, this particular body of literature may be growing. A similar review conducted 6 years ago documented smartphone app interventions for depression among all populations and synthesized only 8 papers [54]; this review synthesized 12 studies for 1 specific demographic. Given that during our full-text assessments, we discarded over 15 potential papers for being in the pre-evaluation phase (ie, they were one of a protocol, feasibility, or acceptability manuscript), more evaluations on this topic appear likely in the coming years.

    Our review also documented that unlike findings in other related health fields, the majority of studies on this topic have been linked to proven treatments. Researchers studying smoking cessation, for instance, found that less than a third of web-based interventions were linked to proven treatment or guidelines [55]. Others examining depression apps [56], for instance, have described similar trends. Similarly, an examination of the National Health Service app library in the United Kingdom revealed that only 4 of 27 apps provided any form of evidence related to patient-reported outcomes in support of their claims of effectiveness [57], which has subsequently raised concerns about the reliability of these tools as support mechanisms. This review found that 66% (8/12) studies linked their intervention to a proven treatment, which represents an encouraging development in this regard and a chance to reiterate that such research practices should be applied in all appropriate future work.

    The results of the meta-analysis confirm the tendency of digital public health interventions to produce modest-to-low effect sizes. Similar trends were reported in a systematic review and meta-analysis of internet-based promotions for health behavior change by Webb et al [58], who concluded that such “interventions had a statistically small but significant effect on health-related behavior.” Typically, these outcomes are argued to be a result of a myriad of confounding variables (eg, genetic predispositions, culture, seasonality); however, the mixed effectiveness of interventions targeting youth populations has previously been considered to derive from poor or insufficient efforts to adapt the initiatives to children’s cognitive and developmental needs [59]. A lack of professional support, tailoring to individual needs, and formulation taking into account immediate family and school contexts have been specifically noted [60]. Given that tailoring or personalization of public health interventions has been found to be desirable [61], it may be worthwhile for future interventions dealing with younger populations to incorporate such strategies, especially those pertaining to cognitive abilities and app support. In fact, in a study of Australian adolescents who were enrolled in a digital intervention, participants professed to value the program’s relatability, narrative structure, and personal choice aspects [62]. Interestingly, however, across the reviewed studies in this review there were, generally, relatively high retention rates—unlike earlier reviews on similar topics such as computer-tailored intervention for behavior change (74.4%) [63]—although, this is likely because of many small-scale evaluations. Nevertheless, an enhanced emphasis on these aspects of program delivery going forward may help improve the effectiveness of smartphone-based interventions addressing youth populations.

    Cross-study assessments of specific internalizing disorder symptoms (eg, anxiety) and intervention style (eg, CBT based) in our subgroup analyses revealed mixed significant effects. Interestingly, studies that included interventions featuring CBT, which is a therapeutic approach that has been recommended for the treatment of mental health issues among adolescents [64], only produced a small, nonsignificant pooled effect size. This may possibly be a consequence of such interventions still being in their nascent stages and having only been tested on limited populations. For comparison, Webb et al [58] similarly observed small effects in their overall analyses of internet-based interventions; they also found no significant effects for smaller subgroup analyses (eg, smoking abstinence, model or demonstration behavior change techniques). There are a few other possible reasons for this outcome. First, this was likely a consequence of the Whittaker et al [49] article’s relatively large sample and correspondingly small effect exerting its influence over the other smaller studies. Second, known barriers to CBT include a lack of training, infrastructure, and funding [65], which may have been present in some studies. Finally, our conservative approach, specifically the decision to use the last follow-up reports and not the most recent postintervention reports resulted in the smallest calculations of Hedges g values (effect sizes) used in the final meta-analysis.

    Taking the review and meta-analysis results together, future studies are warranted to better understand the specific impacts of smartphone-based interventions on different internalizing disorder symptoms as well as their effectiveness as a public health program delivery method. Noting the suggested potential of theory-informed interventions from the review of Webb et al [58] and the small but significant effects found in this review, it would be prudent for future interventions targeting youth mental to continue including proven treatments and pair them with behavioral or social change theories in their delivery methods. A similarly important area will be developing robust evaluation frameworks. As we identified varying lengths of follow-up evaluations, a diverse range of assessment scales, and a variety of different delivery strategies (eg, text messages, apps, monitoring) among the included studies, developing methods to precisely understand and evaluate smartphone-based interventions for sustainability, efficacy or user satisfaction, and functionality should be a priority. On this point, Chan et al [66] have previously recommended that apps be evaluated based on 3 central criteria: integration or infrastructure, usability, and usefulness. Future evaluative frameworks may also consider criteria related to support in the form of self-help strategies, as the method has the potential to relieve some of the burden on existing health care services [67] and has provided positive results for mental health interventions in individual research studies [68].

    Limitations

    There are important caveats to note when interpreting the results of this meta-analysis. As an area of study, research on smartphone-based interventions targeting youth and young people’s mental health, specifically internalizing disorders, is still emerging when compared with other areas of public mental health research. Many of the corrected estimates presented in this meta-analysis were thus derived from small sample populations and a limited number of studies overall. In addition, owing to these small numbers, we did not correct for any other potential errors such as attenuation or dichotomization in the meta-analysis. Such analytical limitations are important to disclose given that the effects of public health interventions are, as previously noted, typically confounded by several variables that may not be captured in the measurement of the primary outcomes and additional corrections could enhance the insights of a meta-analysis. Similar to other recent meta-analyses of health behavior change interventions [69], our risk of bias assessments varied widely across evaluated studies and their intervention designs and should be considered in the context of potential assessor biases being present. Finally, several different scales and measures were used to assess internalizing disorders, potentially resulting in some discrepancy in the measurements of the outcomes used in the analysis. Although the observed between-study variance because of sampling noted in our results was rather modest, which is likely because of the overpowering impact of the Whittaker et al [49] sample and age restrictions of this review—there is likely a high level of variation in our findings derived from the clinical and methodological heterogeneity (eg, varying levels of randomization, types of interventions, controls in analyses) of included studies. On the basis of this, we encourage more specifically focused future meta-analyses that contain a greater number of studies to assess these forms of heterogeneity in their analyses.

    Conclusions and Future Recommendations

    Smartphone-based interventions targeting youth populations appear to be an efficacious strategy to address symptoms of internalizing disorders. This systematic review and meta-analysis found small but significant pooled effects sizes for smartphone-based interventions in reducing the symptoms of internalizing disorders among youth. However, the results also clarify the need for more research in this area. More empirical research studies conducted on a wider range of populations and settings and development of evaluative frameworks for smartphone-based intervention are recommended for future study. Furthermore, our meta-analysis confirms that only a few of the identified conference proceedings, feasibility studies, and other reports have been comprehensively and rigorously evaluated. By following these suggestions, it is possible to further improve not only the understanding of the impact of smartphone-based interventions on youth populations but also better assess the efficacy of smartphones as a mechanism of change for youth internalizing disorders.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    General characteristic of the included studies.

    DOCX File , 17 KB
    ‎
    Multimedia Appendix 2

    Risk of bias in randomized studies.

    PNG File , 314 KB
    ‎
    Multimedia Appendix 3

    Risk of bias in nonrandomized studies.

    PNG File , 428 KB
    1. Improving the mental and brain health of children and adolescents. World Health Organization.   URL: http://www.who.int/mental_health/maternal-child/child_adolescent/en/ [accessed 2019-06-11]
    2. The Mental Health Foundation. Childhood and adolescent mental health: understanding the lifetime impacts. Office of Health Economics. London, UK; 2005 Jan.   URL: https://www.mentalhealth.org.uk/sites/default/files/lifetime_impacts.pdf [accessed 2019-07-04]
    3. Haller DM, Sanci LA, Sawyer SM, Patton GC. The identification of young people's emotional distress: a study in primary care. Br J Gen Pract 2009 Mar;59(560):e61-e70 [FREE Full text] [CrossRef] [Medline]
    4. Suffoletto B, Wagner AK, Arenth PM, Calabria J, Kingsley E, Kristan J, et al. Mobile phone text messaging to assess symptoms after mild traumatic brain injury and provide self-care support: a pilot study. J Head Trauma Rehabil 2013;28(4):302-312. [CrossRef] [Medline]
    5. Grist R, Porter J, Stallard P. Mental health mobile apps for preadolescents and adolescents: a systematic review. J Med Internet Res 2017 May 25;19(5):e176 [FREE Full text] [CrossRef] [Medline]
    6. Cosgrove VE, Rhee SH, Gelhorn HL, Boeldt D, Corley RC, Ehringer MA, et al. Structure and etiology of co-occurring internalizing and externalizing disorders in adolescents. J Abnorm Child Psychol 2010 Aug 4;39(1):109-123. [CrossRef]
    7. Krain A, Ghaffari M, Freeman J, Garcia A, Leonard H, Pine D. Lewis's Child Adolescent Psychiatry. In: Lewis's Child and Adolescent Psychiatry: A Comprehensive Textbook. Philadelphia, Pennsylvania: Lippincott Williams and Wilkins; 2007:538-547.
    8. Hazell P. Depression in children and adolescents. BMJ Clin Evid 2009 Jan 07;2009:- [FREE Full text] [Medline]
    9. Depression. World Health Organization. 2018.   URL: https://www.who.int/news-room/fact-sheets/detail/depression [accessed 2019-07-06]
    10. Cairney J, Rigoli D, Piek J. Developmental coordination disorder and internalizing problems in children: the environmental stress hypothesis elaborated. Dev Rev 2013 Sep;33(3):224-238. [CrossRef]
    11. Gregory AM, Buysse DJ, Willis TA, Rijsdijk FV, Maughan B, Rowe R, et al. Associations between sleep quality and anxiety and depression symptoms in a sample of young adult twins and siblings. J Psychosom Res 2011 Oct;71(4):250-255. [CrossRef] [Medline]
    12. Madrid-Valero J, Ronald A, Shakeshaft N, Schofield K, Malanchini M, Gregory A. Sleep quality, insomnia, and internalizing difficulties in adolescents: insights from a twin study. Sleep 2020 Feb 13;43(2):-. [CrossRef] [Medline]
    13. Whalen DJ, Sylvester CM, Luby JL. Depression and anxiety in preschoolers: a review of the past 7 years. Child Adolesc Psychiatr Clin N Am 2017 Jul;26(3):503-522 [FREE Full text] [CrossRef] [Medline]
    14. Rapee RM, Schniering CA, Hudson JL. Anxiety disorders during childhood and adolescence: origins and treatment. Annu Rev Clin Psychol 2009;5:311-341. [CrossRef] [Medline]
    15. Compton SN, Burns BJ, Helen LE, Robertson E. Review of the evidence base for treatment of childhood psychopathology: internalizing disorders. J Consult Clin Psychol 2002 Dec;70(6):1240-1266. [CrossRef] [Medline]
    16. Konnopka A, Leichsenring F, Leibing E, König HH. Cost-of-illness studies and cost-effectiveness analyses in anxiety disorders: a systematic review. J Affect Disord 2009 Apr;114(1-3):14-31. [CrossRef] [Medline]
    17. Faurholt-Jepsen M, Bauer M, Kessing LV. Smartphone-based objective monitoring in bipolar disorder: status and considerations. Int J Bipolar Disord 2018 Jan 23;6(1):6 [FREE Full text] [CrossRef] [Medline]
    18. Torous J, Staples P, Barnett I, Sandoval LR, Keshavan M, Onnela J. Characterizing the clinical relevance of digital phenotyping data quality with applications to a cohort with schizophrenia. npj Digital Med 2018 Apr 6;1(1):1-9. [CrossRef] [Medline]
    19. Mehrotra S, Tripathi R. Recent developments in the use of smartphone interventions for mental health. Curr Opin Psychiatry 2018 Sep;31(5):379-388. [CrossRef] [Medline]
    20. Oh SJ, Seo S, Lee JH, Song MJ, Shin M. Effects of smartphone-based memory training for older adults with subjective memory complaints: a randomized controlled trial. Aging Ment Health 2018 Apr;22(4):526-534. [CrossRef] [Medline]
    21. Moëll B, Kollberg L, Nasri B, Lindefors N, Kaldo V. Living SMART — A randomized controlled trial of a guided online course teaching adults with ADHD or sub-clinical ADHD to use smartphones to structure their everyday life. Internet Interv 2015 Mar;2(1):24-31. [CrossRef]
    22. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual research review: digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef] [Medline]
    23. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
    24. Domhardt M, Steubl L, Baumeister H. Internet- and mobile-based interventions for mental and somatic conditions in children and adolescents. Z Kinder Jugendpsychiatr Psychother 2020 Jan;48(1):33-46 [FREE Full text] [CrossRef] [Medline]
    25. Pennant ME, Loucas CE, Whittington C, Creswell C, Fonagy P, Fuggle P, Expert Advisory Group. Computerised therapies for anxiety and depression in children and young people: a systematic review and meta-analysis. Behav Res Ther 2015 Apr;67:1-18. [CrossRef] [Medline]
    26. Petticrew M, Roberts H. Systematic Reviews in the Social Sciences: A Practical Guide. Malden, MA, USA: Blackwell Pub; 2006:1-335.
    27. Clarke PJ, Bedford K, Notebaert L, Bucks RS, Rudaizky D, Milkins BC, et al. Assessing the therapeutic potential of targeted attentional bias modification for insomnia using smartphone delivery. Psychother Psychosom 2016;85(3):187-189. [CrossRef] [Medline]
    28. Grassi A, Gaggioli A, Riva G. The green valley: the use of mobile narratives for reducing stress in commuters. Cyberpsychol Behav 2009 Apr;12(2):155-161. [CrossRef] [Medline]
    29. Takahashi K, Takada K, Hirao K. Feasibility and preliminary efficacy of a smartphone application intervention for subthreshold depression. Early Interv Psychiatry 2019 Feb;13(1):133-136. [CrossRef] [Medline]
    30. Keith DJ, Rinchuse DJ, Kennedy M, Zullo T. Effect of text message follow-up on patient's self-reported level of pain and anxiety. Angle Orthod 2013 Jul;83(4):605-610. [CrossRef] [Medline]
    31. Wilkinson P. Conceptualization about internalizing problems in children and adolescents. Cien Saude Colet 2009;14(2):373-381 [FREE Full text] [CrossRef] [Medline]
    32. Krueger RF. The structure of common mental disorders. Arch Gen Psychiatry 1999 Oct;56(10):921-926. [CrossRef] [Medline]
    33. Watson D, Clark LA, Carey G. Positive and negative affectivity and their relation to anxiety and depressive disorders. J Abnorm Psychol 1988 Aug;97(3):346-353. [CrossRef] [Medline]
    34. Hunter J, Schmidt F. Methods of Meta-analysis: Correcting Error and Bias in Research Findings. Thousand Oaks, CA, USA: Sage; 1990.
    35. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol 2013 Nov 26;4:863 [FREE Full text] [CrossRef] [Medline]
    36. Allen TD, Eby LT, Poteet ML, Lentz E, Lima L. Career benefits associated with mentoring for protégeé: a meta-analysis. J Appl Psychol 2004 Feb;89(1):127-136. [CrossRef] [Medline]
    37. Field AP. Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods. Psychol Methods 2001 Jun;6(2):161-180. [CrossRef] [Medline]
    38. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods 2002 Mar;7(1):105-125. [CrossRef] [Medline]
    39. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0.: The Cochrane Collaboration; 2011.   URL: https://handbook-5-1.cochrane.org/ [accessed 2020-12-24]
    40. Quality Assessment Tool For Quantitative Studies. Effective Public Health Practice Project. 2017.   URL: https://merst.ca/wp-content/uploads/2018/02/quality-assessment-tool_2010.pdf [accessed 2018-11-29]
    41. Quality Assessment Tool for Quantitative Studies Dictionary. Effective Public Health Practice Project. 2017.   URL: https://merst.ca/wp-content/uploads/2018/02/qualilty-assessment-dictionary_2017.pdf [accessed 2018-11-29]
    42. Kauer SD, Reid SC, Crooke AHD, Khor A, Hearps SJC, Jorm AF, et al. Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012 Jun 25;14(3):e67 [FREE Full text] [CrossRef] [Medline]
    43. Lee J, Jung H, Lee G, Kim H, Park S, Woo S. Effect of behavioral intervention using smartphone application for preoperative anxiety in pediatric patients. Korean J Anesthesiol 2013 Dec;65(6):508-518 [FREE Full text] [CrossRef] [Medline]
    44. Ranney ML, Freeman JR, Connell G, Spirito A, Boyer E, Walton M, et al. A depression prevention intervention for adolescents in the emergency department. J Adolesc Health 2016 Oct;59(4):401-410 [FREE Full text] [CrossRef] [Medline]
    45. Ranney M, Pittman S, Dunsiger S, Guthrie K, Spirito A, Boyer E, et al. Emergency department text messaging for adolescent violence and depression prevention: a pilot randomized controlled trial. Psychol Serv 2018 Nov;15(4):419-428 [FREE Full text] [CrossRef] [Medline]
    46. Reid SC, Kauer SD, Hearps SJC, Crooke AHD, Khor AS, Sanci LA, et al. A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial. BMC Fam Pract 2011 Nov 29;12:131 [FREE Full text] [CrossRef] [Medline]
    47. Stallard P, Porter J, Grist R. A smartphone app (BlueIce) for young people who self-harm: open phase 1 pre-post trial. JMIR Mhealth Uhealth 2018 Jan 30;6(1):e32 [FREE Full text] [CrossRef] [Medline]
    48. Werner-Seidler A, Wong Q, Johnston L, O'Dea B, Torok M, Christensen H. Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms. BMJ Open 2019 May 27;9(5):e026502 [FREE Full text] [CrossRef] [Medline]
    49. Whittaker R, Stasiak K, McDowell H, Doherty I, Shepherd M, Chua S, et al. MEMO: an mHealth intervention to prevent the onset of depression in adolescents: a double-blind, randomised, placebo-controlled trial. J Child Psychol Psychiatry 2017 Sep;58(9):1014-1022. [CrossRef] [Medline]
    50. Worthen-Chaudhari L, McGonigal J, Logan K, Bockbrader MA, Yeates KO, Mysiw WJ. Reducing concussion symptoms among teenage youth: evaluation of a mobile health app. Brain Inj 2017;31(10):1279-1286 [FREE Full text] [CrossRef] [Medline]
    51. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016 Oct 12;355:i4919 [FREE Full text] [CrossRef] [Medline]
    52. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Br Med J 2011 Oct 18;343:d5928 [FREE Full text] [CrossRef] [Medline]
    53. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis. J Med Internet Res 2013 Jun 14;15(6):e120 [FREE Full text] [CrossRef] [Medline]
    54. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013 Nov 15;15(11):e247 [FREE Full text] [CrossRef] [Medline]
    55. Bock BC, Graham AL, Whiteley JA, Stoddard JL. A review of web-assisted tobacco interventions (WATIs). J Med Internet Res 2008 Nov 06;10(5):e39 [FREE Full text] [CrossRef] [Medline]
    56. Huguet A, Rao S, McGrath PJ, Wozney L, Wheaton M, Conrod J, et al. A systematic review of cognitive behavioral therapy and behavioral activation apps for depression. PLoS One 2016;11(5):e0154248 [FREE Full text] [CrossRef] [Medline]
    57. Leigh S, Flatt S. App-based psychological interventions: friend or foe? Evid Based Ment Health 2015 Nov;18(4):97-99. [CrossRef] [Medline]
    58. Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010 Feb 17;12(1):e4 [FREE Full text] [CrossRef] [Medline]
    59. Adelman CB, Panza KE, Bartley CA, Bontempo A, Bloch MH. A meta-analysis of computerized cognitive-behavioral therapy for the treatment of DSM-5 anxiety disorders. J Clin Psychiatry 2014 Jul;75(7):e695-e704. [CrossRef] [Medline]
    60. Simons L, Valentine AZ, Falconer CJ, Groom M, Daley D, Craven MP, et al. Developing mHealth remote monitoring technology for attention deficit hyperactivity disorder: a qualitative study eliciting user priorities and needs. JMIR Mhealth Uhealth 2016 Mar 23;4(1):e31 [FREE Full text] [CrossRef] [Medline]
    61. Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull 2007 Jul;133(4):673-693. [CrossRef] [Medline]
    62. Cheek C, Bridgman H, Fleming T, Cummings E, Ellis L, Lucassen MF, et al. Views of young people in rural Australia on SPARX, a fantasy world developed for New Zealand youth with depression. JMIR Serious Games 2014 Feb 18;2(1):e3 [FREE Full text] [CrossRef] [Medline]
    63. Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010;51(3-4):214-221 [FREE Full text] [CrossRef] [Medline]
    64. Connolly SD, Bernstein GA, Work Group on Quality Issues. Practice parameter for the assessment and treatment of children and adolescents with anxiety disorders. J Am Acad Child Adolesc Psychiatry 2007 Feb;46(2):267-283. [CrossRef] [Medline]
    65. Gunter RW, Whittal ML. Dissemination of cognitive-behavioral treatments for anxiety disorders: overcoming barriers and improving patient access. Clin Psychol Rev 2010 Mar;30(2):194-202. [CrossRef] [Medline]
    66. Chan S, Torous J, Hinton L, Yellowlees P. Towards a framework for evaluating mobile mental health apps. Telemed J E Health 2015 Dec;21(12):1038-1041. [CrossRef] [Medline]
    67. Jorm AF, Griffiths KM. Population promotion of informal self-help strategies for early intervention against depression and anxiety. Psychol Med 2006 Jan;36(1):3-6. [CrossRef] [Medline]
    68. Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MF. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. Br Med J 2012 Apr 19;344(apr18 3):e2598-e2598. [CrossRef]
    69. Maher CA, Lewis LK, Ferrar K, Marshall S, De BI, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res 2014 Feb;16(2):e40 [FREE Full text] [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    EPHPP: Effective Public Health Practice Project
    QA: quality assessment
    RCT: randomized controlled trial

    Edited by G Eysenbach; submitted 03.10.19; peer-reviewed by M Domhardt, J Bender; comments to author 26.10.20; revised version received 11.11.20; accepted 18.11.20; published 11.01.21

    Copyright

    ©Adrian Buttazzoni, Keshbir Brar, Leia Minaker. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.01.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.