This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514) Web-based and Mobile Health Interventions (3297) Reviews in Digital Mental Health (243)

Published on in Vol 24, No 9 (2022): September

This is a member publication of Imperial College London (Jisc)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/37776, first published .
Trends in Effectiveness of Organizational eHealth Interventions in Addressing Employee Mental Health: Systematic Review and Meta-analysis

Trends in Effectiveness of Organizational eHealth Interventions in Addressing Employee Mental Health: Systematic Review and Meta-analysis

Trends in Effectiveness of Organizational eHealth Interventions in Addressing Employee Mental Health: Systematic Review and Meta-analysis

Authors of this article:

Elizabeth Stratton1, 2 Author Orcid Image ;   Amit Lampit3 Author Orcid Image ;   Isabella Choi1, 2 Author Orcid Image ;   Hanna Malmberg Gavelin3, 4 Author Orcid Image ;   Melissa Aji1 Author Orcid Image ;   Jennifer Taylor1 Author Orcid Image ;   Rafael A Calvo5 Author Orcid Image ;   Samuel B Harvey6, 7, 8 Author Orcid Image ;   Nick Glozier1, 2 Author Orcid Image
Article Authors Cited by (15) Tweetations (17) Metrics

    Review

    1Central Clinical School, Faculty of Health and Medicine, University of Sydney, Camperdown, Australia

    2ARC Centre of Excellence for Children and Families over the Life Course, Sydney, Australia

    3Academic Unit for Psychiatry of Old Age, University of Melbourne, Parkville, Australia

    4Department of Psychology, Umeå University, Umeå, Sweden

    5Dyson School of Design Engineering, Imperial College London, London, United Kingdom

    6School of Psychiatry, University of New South Wales, Sydney, Australia

    7Black Dog Institute, Sydney, Australia

    8St George Hospital, Sydney, Australia

    Corresponding Author:

    Rafael A Calvo, PhD

    Dyson School of Design Engineering

    Imperial College London

    Imperial College Rd, South Kensington,

    London, SW7 9EG

    United Kingdom

    Phone: 44 20 7594 888

    Email: r.calvo@imperial.ac.uk


    Background: Mental health conditions are considered the leading cause of disability, sickness absence, and long-term work incapacity. eHealth interventions provide employees with access to psychological assistance. There has been widespread implementation and provision of eHealth interventions in the workplace as an inexpensive and anonymous way of addressing common mental disorders.

    Objective: This updated review aimed to synthesize the literature on the efficacy of eHealth interventions for anxiety, depression, and stress outcomes in employee samples in organizational settings and evaluate whether their effectiveness has improved over time.

    Methods: Systematic searches of relevant articles published from 2004 to July 2020 of eHealth intervention trials (app- or web-based) focusing on the mental health of employees were conducted. The quality and bias of all studies were assessed. We extracted means and SDs from publications by comparing the differences in effect sizes (Hedge g) in standardized mental health outcomes. We meta-analyzed these data using a random-effects model.

    Results: We identified a tripling of the body of evidence, with 75 trials available for meta-analysis from a combined sample of 14,747 articles. eHealth interventions showed small positive effects for anxiety (Hedges g=0.26, 95% CI 0.13-0.39; P<.001), depression (Hedges g=0.26, 95% CI 0.19-0.34; P<.001), and stress (Hedges g=0.25, 95% CI 0.17-0.34; P<.001) in employees’ after intervention, with similar effects seen at the medium-term follow-up. However, there was evidence of no increase in the effectiveness of these interventions over the past decade.

    Conclusions: This review and meta-analysis confirmed that eHealth interventions have a small positive impact on reducing mental health symptoms in employees. Disappointingly, we found no evidence that, despite the advances in technology and the enormous resources in time, research, and finance devoted to this area for over a decade, better interventions are being produced. Hopefully, these small effect sizes do not represent optimum outcomes in organizational settings.

    Trial Registration: PROSPERO CRD42020185859; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=185859

    J Med Internet Res 2022;24(9):e37776

    doi:10.2196/37776

    Keywords

    eHealth; mental health; employee; systematic review; mobile phone 


    Mental health conditions are to be considered the leading cause of disability, sick leave, and long-term work incapacity in most developed countries [1]. Furthermore, poor mental health has a substantial impact on employee well-being, productivity, absenteeism, compensation claims, and social welfare systems [2,3]. Evidence supports the increased demand for workplace interventions, highlighting that working conditions and the workplace environment can influence employees’ mental health and well-being [4].

    The nature of mental health symptoms fluctuates on a continuum between thriving and struggling [5]. Most mental ill health that is seen in the workforce is because of common mental disorders, most notably, depression, anxiety, and stress-related conditions [1,6,7]. Employees who have a mental health condition and become too unwell to continue working rarely move straight from being healthy to needing sick leave [8]. There is usually a course that an employee might experience as they develop worsening symptoms [9], and different interventions may be required at different stages of this course.

    International approaches suggest multilevel organizational approaches targeting (1) healthy workers via universal prevention interventions; (2) those with subclinical conditions (symptomatic or at-risk workers), such as those experiencing high stress via indicated interventions; and (3) those workers who have disclosed a mental health condition with tertiary interventions (treatments) [8]. Furthermore, the UK Thriving at Work Review into Mental Health at Work recommends the inclusion of the use of therapeutically tailored interventions based on individual-specific needs [5].

    The 2 potential rate-limiting steps for organizations in implementing mental health intervention programs are the budget and logistics of delivering universal interventions to all staff at scale and the ability to target indicated and early interventions to those who are at risk or unwell before disclosure. The internet offers a unique opportunity to address these rate-limiting steps by delivering eHealth interventions with components such as cognitive behavioral therapy (CBT) and stress management to a broad audience. eHealth interventions provide employees with access to psychological help when they are not employed in a typical working environment, such as shift workers or those working from home, or when they may be sick-listed from work. As a result, eHealth interventions have been widely implemented and provided. eHealth is an emerging field in public health and business and provides health services and information delivered or enhanced through the internet and related technologies, such as smartphone apps. Recently, eHealth has been seen as a popular approach in organizations as it provides an inexpensive and anonymous way of addressing common mental disorders [10], including apps linked to wearable devices, and guided meditation programs [11].

    There is a plethora of evidence for the short- and long-term benefits of eHealth-delivered CBT for treating anxiety and depressive conditions in both the general population and clinical settings [12-15]. Evidence has also emerged for the effectiveness of mindfulness-based eHealth interventions in improving symptoms for both the general population and individuals who are symptomatic [16].

    However, we know that employed individuals differ systemically from both general and clinical populations used in most eHealth studies; for instance, employees have much better mental health (fewer symptoms) than general and clinical populations [17], for whom many eHealth interventions and their content have been developed. As such, there are likely to be floor effects and other efficacy modifiers. The delivery in, and by, organizations will be different from just open access to interventions in the general population. This influences uptake and engagement, which are known determinants of digital health efficacy [18].

    A previous meta-analysis considering eHealth interventions from 2004 (first identified) to 2017 [19] found randomized controlled trial (RCT) evidence for only 23 eHealth interventions delivered to employees. Overall, there was a small pooled effect of reducing depression, anxiety, and stress symptoms after the intervention (Hedges g=0.24), which was sustained at follow-up (Hedges g=0.23). Similar effect sizes (ESs) were found in a contemporaneous review [20]. Only 2 years later, a meta-analysis [21] identified 50% (n=34) more studies of such interventions and suggested stronger effects on stress (Hedges g=0.54), insomnia (Hedges g=0.70), and burnout (Hedges g=0.51) but not symptoms of depression or anxiety.

    None of the previous reviews have addressed the key issue of efficacy, which is affected by the digital placebo effect. Smartphone app users can experience significantly reduced mental health symptoms, even if the app does not contain any direct therapeutic intervention, similar to the placebo effect seen in pharmaceutical trials when using “active” or “attention” controls [22], or may reflect regression to the mean of fluctuating symptoms. Even simply tracking the symptoms of depression in an app can lead to significant reductions in depression [23]. Both processes result in an apparent “intra-group” effect without any therapeutic intervention. In a trial, the type of control used is a primary determinant of the “between-group” ES. In a recent systematic review of smartphone apps for anxiety in clinical samples, the between-group ESs were lower in trials that used active controls than in those that used passive controls [24]. This inflates the apparent efficacy of the interventions evaluated in trials with passive controls, such as the wait-list. It is not known whether and to what extent this is observed in working populations or for more preventive approaches.

    Considering the increasing number of interventions being developed, implemented, and potentially evaluated, this review aimed to address the following questions by systematically reviewing the current state of evidence for the efficacy of eHealth interventions in reducing depression, anxiety, and stress in employees (Textbox 1).

    Research questions.

    Research questions addressed by this review

    • Is there evidence for the improvement in effectiveness over time (ie, are the interventions getting better)?
    • What factors, if any, moderate the efficacy?
      • Intervention approach; indicated, tailored, tertiary or universal
      • Type of control (active vs wait-list)
      • Type of intervention: cognitive behavioral therapy, mindfulness, stress management, and other
      • Presence of in-person support
    Textbox 1. Research questions.

    Overview

    This review complies with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [25]. We aimed to identify all published and unpublished, peer-reviewed, controlled clinical trials of eHealth interventions targeted at employees reporting outcomes on a standardized mental health measure of depression, anxiety, and stress. The systematic review protocol was registered on PROSPERO (International Prospective Register of Systematic Reviews; CRD42020185859).

    Search Strategy

    A systematic literature search was conducted using MEDLINE, PsycINFO, Cochrane Register of Controlled Trials, and Embase electronic databases for relevant peer-reviewed articles in English of controlled trials and RCTs published from 2004 (when the first eHealth intervention was identified) [19] to July 2020. Keywords used related to “workplace,” “intervention,” “outcome,” and “study design.” An example of this search strategy is presented in Multimedia Appendix 1. The search terms were developed from our previous systematic review [19]. The tables of contents of the Journal of Medical Internet Research, Journal of Internet Interventions, Occupational and Environmental Medicine, and the Journal of Occupational and Environmental Medicine, as well as the reference lists of included studies, were manually searched.

    Study Selection Criteria

    Eligibility Criteria

    The PICOS (Patient/Population, Intervention, Comparison, and Outcomes) framework commonly used to identify components of clinical evidence for systematic reviews in evidence-based medicine, endorsed by the Cochrane Collaboration [26], was used in this review.

    Participants

    Participants had to be in current paid employment and working-age adults aged between 18 and 65 years. Studies were excluded if the sample was defined as volunteer workers, unemployed participants, or general or clinical populations.

    Interventions

    Any eHealth intervention, defined as a therapeutic intervention delivered through a website, smartphone, tablet, or mobile app and designed to improve mental health, was included.

    Controls

    Studies were required to have a control group, defined as either passive (care as usual, no contact, or wait-list) or active (eg, another eHealth intervention as the comparison group).

    Outcomes

    The study had to report on at least one common mental disorder outcome: depression, anxiety, or stress. All eligible outcomes for each study and domain were included. The outcomes were standardized mean difference (SMD) from preintervention baseline score to immediate postintervention use and the follow-up time point.

    The following measurements of common mental disorders were regarded as eligible:

    1. Diagnostic interview
    2. Self-report diagnosis by a physician, psychologist, or other qualified health professional
    3. Self-administered rating scale for mental health (anxiety, depression, or stress)
    Studies

    Studies were excluded from the review if they were not specifically limited to employees and delivered in a workplace setting or used in-person, telephone, and email interventions only.

    Identification of Studies

    After duplicates were removed, 3 independent authors (ES, IC, and MA) screened all titles and abstracts to identify potentially relevant studies. Abstracts and full-text versions of potentially eligible studies were independently assessed by 2 investigators (ES and IC). Eligible studies with individual citations were scanned to ensure that all relevant studies were identified. Disagreements were adjudicated in conjunction with the senior author (NG).

    Data Extraction and Coding

    We extracted the mean and SD of standardized measures of stress or distress, depression, and anxiety and the sample size (n) in each arm (intervention and control) at baseline and at each follow-up time point. When sufficient details were not reported, the authors were contacted. Additional data concerning the type of design, intervention and control details, participant characteristics, study duration, length of follow-up, organizational population, and general outcomes were recorded (see the Results section). Studies were also coded by intervention type: universal interventions targeting relatively healthy participants; indicated interventions targeting subclinical symptoms or syndromes; and tertiary treatments for explicit diagnoses.

    Studies reporting results of the same intervention and sample in different papers (eg, a postintervention and follow-up study) were treated as 1 study, and we used the first follow-up outcome point in the analysis.

    We categorized studies according to their mental health outcome measures: anxiety, depression, or stress. Studies with multiple outcomes appear in >1 mental health outcome result.

    Quality Assessment and Risk of Bias Within Studies

    The risk of bias in RCTs was assessed using the revised Cochrane risk of bias tool for RCTs (RoB version 2.0) [27], in which 5 domains were independently evaluated by 2 authors: randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result. Each domain was assessed for the risk of bias. Studies were graded as (1) “low risk of bias” when a low risk of bias was determined for all domains, (2) “some concerns” if at least one domain was assessed as raising some concerns but not to be at high risk of bias for any single domain, or (3) “high risk of bias” when a high risk of bias was determined for at least one domain or there were bias concerns in multiple domains [27]. If a study contained >1 intervention arm, each intervention was assessed as 1 study.

    Statistical Analysis

    The summary ES was the SMD (calculated as Hedge g, with 95% CI) between the intervention and control groups for each outcome measure (anxiety, depression, and stress) [28]. A positive ES (SMD) indicated that the intervention was more efficacious than the control. The level of significance was set at P<.05 and 95% CIs. The magnitude of the effect was categorized as large (SMD>0.8), moderate (SMD 0.5-0.8), small (SMD 0.2 to <0.5), or trivial (SMD<0.2) [29]. Pooling of ESs across studies was performed using a random-effects model in Comprehensive Meta-Analysis (Biostat Inc; version 3). We detected and accommodated outliers in the meta-analysis using the random-effects variance shift model and the likelihood ratio test. As the different types of eHealth interventions and approaches used may result in large heterogeneity [30], between-study heterogeneity was quantified using τ2 (variance of true effects) and further assessed using the Higgins I2 statistic, which estimates the proportion of observed variance not because of sampling error. An I2 value of 75% was considered large, 50% was considered moderate, and 25% was considered low [31].

    We estimated the results separately for each mental health outcome measure (anxiety, depression, and stress) at the postintervention and first follow-up time points. Meta-regressions were performed for each mental health outcome to identify whether the intervention approach (eg, universal), type of intervention (eg, CBT), type of control (eg, wait-list), or intervention delivery method (web-based vs smartphone) moderated the observed effect.

    To evaluate differences in effectiveness over time, we used only within-group ESs from the intervention arms for anxiety, depression, or stress outcomes. ESs were calculated by dividing the difference between pre- and posttreatment means by the pooled SD of the difference [32]. ANOVA was used to assess whether there were any differences between the mean ESs for each year. For a test for trends, the within‐group ESs with positive signs indicated improvement, and negative signs indicated worsening. This method allowed for the difference in effects over time to be accurately calculated, allowing us to explore whether interventions improved with technology.

    To assess small study effects (in cases where at least 10 studies were available for analysis), we used a funnel plot for the overall effects and each subgroup analysis, which compared the outcome effects with their SEs. We used the Egger regression test to examine further asymmetry of the funnel plot [33] with statistical significance based on P<.10. In cases where at least 10 studies were available and a small study effect was found, we used a Duval and Tweedie trim and fill analysis to quantify the magnitude of the small study effect [34].


    Search Results

    The search strategy identified 3411 titles (Figure 1). Of the 3411 articles, after the removal of 371 (10.88%) duplicates, 3040 (89.12%) titles and abstracts were reviewed by the authors (ES, IC, and MA). Of these 3040 articles, 2893 (95.16%) articles were excluded based on the eligibility criteria, leaving 147 (4.84%) articles that were potentially relevant to the research question. Their full texts were examined by 2 independent researchers (ES and MA), and discrepancies were decided by the senior author (NG). Of the 147 articles, 74 (50.3%) were excluded for reasons provided in Figure 1. Of the remaining 73 studies, data were missing for 8 (11%) studies. The authors were contacted, and 5 responded with relevant data [35-39]. A total of 3 authors did not respond or were unable to obtain data; thus, these studies were excluded from the meta-analysis [35,40,41]. Of the 73 studies, 4 (5%) articles reported on the same study at different follow-up time points and were merged [42-45], leaving 68 (93%) studies, of which 2 (3%) were excluded as they used face-to-face treatment for control groups [46,47]. The remaining 97% (66/68) of articles were identified as meeting the criteria for quality assessment. Of the 66 studies, 5 (8%) used 2 intervention arms [48-52], and 2 (3%) studies used 3 intervention arms [44,53]. In this case, the intervention arms were treated as individual trials reported as author names; year of publication; and the letters a, b, or c (eg, Smith [a]). The number of participants in the control group was split evenly as a comparative arm to ensure that participants were not counted twice. This process resulted in 75 trials (Figure 1) for the meta-analysis. Multimedia Appendix 2 [36-39,42-45,49-53] provides the list of references.

    ‎
    Figure 1. Flow Diagram of included studies.
    View this figure

    Risk of Bias

    The individual studies included in the meta-analysis showed that, overall, 12% (9/75) had a low risk of bias, 3% (2/75) had ≥1 concern, and 85% (64/75) had a high risk of bias. Most of the bias was observed in deviations from the intended interventions and missing outcome data, which is not uncommon and is an area for improvement in the eHealth field. An important aspect common to behavioral interventions is that 55% (41/75) of the studies were rated as having a high risk of bias because of the use of a wait-list control group. As such, it was impossible for users to remain blinded (Multimedia Appendix 3 [36-39,42-45,49-53]).

    All the included trials used web-based self-report measures to collect the baseline and follow-up data. Multimedia Appendix 4 provides an overview of the scales used and their prevalence.

    Meta-analysis Results

    Overall Summary of the Identified Interventions

    The 75 trials reported postintervention data for 14,747 participants. Most trials used universal approaches (47/75, 63%), with fewer indicated (10/75, 13%), tertiary (11/75, 15%), or tailored (7/75, 9%) intervention approaches. Most interventions were delivered via web-based platforms as opposed to smartphone apps. Most of the trials used a wait-list control group. The 3 most common types of interventions described by the authors were CBT, mindfulness, and stress management. The studies were conducted in 15 countries: the United States (16/75, 21%), Japan (15/75, 20%), Germany (13/75, 17%), the United Kingdom (7/75, 9%), the Netherlands (5/75, 7%), Australia (4/75, 5%), Sweden (4/75, 5%), China (2/75, 3%), Europe combined (2/75, 3%), Italy (2/75, 3%), Brazil (1/75, 1%), Finland (1/75, 1%), Hong Kong (1/75, 1%), Singapore (1/75, 1%), and the United States and Canada combined (1/75, 1%). The most common types of participants were health care professionals (18/75, 24%). Interventions for the insurance industry (7/75, 9%), managers (6/75, 8%), information technology (6/75, 8%), male-dominated industries (5/75, 7%), telecommunications (5/75, 7%), schools (3/75, 4%), universities (3/75, 4%), marketing and sales (3/75, 4%), banking (1/75, 1%), and human resources (1/75, 1%) were also evaluated (Figure 2). The term general employee is used in Multimedia Appendix 2 when the study did not mention the organization type. A total of 15 trials were conducted across several industries, and only 2 were conducted in employees on sick leave. Further summarized and detailed descriptions of the studies are presented in Table 1 and Multimedia Appendix 5 [36-39,42-45,49-53].

    ‎
    Figure 2. Effects on anxiety symptoms.
    View this figure
    Table 1. Summarized description of the selected studies.
    StudyTypeCountry and population (sample size)Intervention and duration (sample size)Control (sample size)Type of intervention
    Grimea [54]CBTbUnited Kingdom; Occupational health department (48)
    • “Beating the Blues” web-based program for depression and anxiety
      • ‎
    • 8 web-based sessions, which last approximately an hour per week (24)
      • ‎
    		WLCc (24)Tertiary
    Hasson et ala,d [55]CBTSweden; IT and media companies (317)
    • 12-month open access
      • ‎
    • Web-based self-help exercises (129)
      • ‎
    		WLC+information (174)Universal
    Shimazu et ala [56]CBTJapan; construction machinery company (225)
    • 1-month web-based psychoeducation based on social cognitive theory
      • ‎
    • Self-based program with 3 phases (5 chapters; 112)
      • ‎
    		WLC (113)Universal
    Cook et ala [57]Stress managementUnited States; human resource employees (419)
    • Web-based multimedia health promotion program “Health Connection” (209)
      • ‎
    		WLC+paper-based information(210)Universal
    Ruwaard et ala,d [58]CBTThe Netherlands; general employees (239)
    • 7-week web-based program
      • ‎
    • Supported by trained therapists with 10 personalized feedback sessions
      • ‎
    • 1 module per week: awareness, relaxation, worrying, positive self-verbalization, positive assertiveness, and time management
      • ‎
    • In-person and audio training (177)
      • ‎
    		WLC (62)Tailored
    Billings et ale [59]CBTUnited States; technology company (309)
    • 3-month open-access web-based program (154)
      • ‎
    		WLC (155)Tailored
    Suzuki et ala [60]CBTJapan; university staff (43)
    • A 2-week program with 4 modules
      • ‎
    • Daily monitoring and feedback and sleep diary
      • ‎
    • Emailed weekly summary and advice (21)
      • ‎
    		WLC (22)Universal
    Yamagishi et ala [61]Assertion trainingJapan; shift working nurses (60)
    • 9 weeks of 60-minute web-based training provided weekly (30)
      • ‎
    		WLC (30)Universal
    Abbott et ala [62]CBTAustralia; industrial organization (53)
    • Web-based program
      • ‎
    • 7 core modules (26)
      • ‎
    		WLC (27)Universal
    Bennett et ala [63]Behavior changeUnited States; general managers (145)
    • Web-based open-access ExecuPrev
      • ‎
    • At least 10 hours over 6 months (72)
      • ‎
    		WLC (73)Universal
    Glück and Maerckera [64]MindfulnessAustria, Germany, and Switzerland; universities, car dealerships, broadcasting stations, and health care consulting (50)
    • Web-based program for 13 days
      • ‎
    • 2 modules, with each module lasting for 6 days with 20 minutes per day (28)
      • ‎
    		WLC (21)Universal
    Borness et ala [65]Cognitive trainingAustralia; public sector general employees (135)
    • 16 weeks of web-based cognitive training based on memory, attention, language, and executive function
      • ‎
    • The program was called “Spark!” with three 20-minute sessions per week (67)
      • ‎
    		Active control program (68); general knowledge informationUniversal
    Feicht et ala [66]Positive psychologyGermany; insurance company (147)
    • 7-week web-based happiness training
      • ‎
    • Weekly modules that took 10 to 15 minutes
      • ‎
    • Emailed instructions once weekly (85)
      • ‎
    		WLC (62)Universal
    Ketelaar et al [38]aHealth Surveillance ModelThe Netherlands; nurses and allied health professionals (367)
    • Tailored, assessed by screening Psyfit: well-being based; healthy participants
      • ‎
    • Strong at work: stress management; learning skills to cope with work stress
      • ‎
    • Color your life: depressive symptoms
      • ‎
    • Don’t Panic Online: panic symptoms for subclinical and mild cases of panic disorder
      • ‎
    • Drinking less: reducing risky alcohol drinking behavior (178)
      • ‎
    		WLC (188)Tailored
    Lappalainen et ala,d [67]CBT and ACTfFinland; men aged 28 to 58 years with depression (23)
    • “P4Well”—a 3-month program, including 3 group meetings, an internet or web portal, mobile phone apps, and personal monitoring devices (11)
      • ‎
    		WLC (12)Tertiary
    Villani et ala,d [68]Stress managementItaly; female oncology nurses with high stress (30)
    • 4-week program, with eight 5-minute video clips twice weekly, with a narrative
      • ‎
    • After work on the study phone (15)
      • ‎
    		Active control with 8 video clips (15)Indicated
    Bolier et ala,d [69]Health Surveillance ModelThe Netherlands; nurses, allied health professionals, and general employees (423)
    • Tailored, assessed by screening Psyfit: well-being based; healthy participants
      • ‎
    • Strong at work: stress management; learning skills to cope with work stress
      • ‎
    • Color your life: depressive symptoms
      • ‎
    • Don’t Panic Online: panic symptoms for subclinical and mild cases of panic disorder
      • ‎
    • Drinking less: reducing risky alcohol drinking behavior (212)
      • ‎
    		WLC (211)Tailored
    Deitz et ala [70]Behavior changeUnited States; hospital employees with cardiovascular risk (210)
    • 6-week web-based program with weekly modules
      • ‎
    • On the basis of increasing knowledge and reducing risk (105)
      • ‎
    		WLC (105)Universal
    Ebert et ala,d [10]Problem-solving trainingGermany; teachers (150)
    • 6 weeks, 5 lessons
      • ‎
    • Web-based program with 1 lesson per week and practice between each lesson (75)
      • ‎
    		WLC (75)Tertiary
    Geraedts et al [42,43] (2 papers)a,dProblem-solving training and cognitive therapyThe Netherlands; banking (231)
    • Web-based Happy@Work program
      • ‎
    • 6 weekly sessions, participants had to complete an assessment each week to move on (116)
      • ‎
    		WLC (115)Tertiary
    Imamura et al [44,45] (2 papers)a,dCBTJapan; IT (762)
    • 6-week web-based program
      • ‎
    • 6 lessons, 1 lesson per week, approximately 30 minutes each
      • ‎
    • Each lesson had homework (381)
      • ‎
    		WLC+information (381)Tertiary
    Ly et ala,d [71]ACTSweden; middle managers in the private sector (73)
    • 6-week smartphone app
      • ‎
    • 6 modules, 1 per week
      • ‎
    • Audio lecture, text, and exercises supported by student psychologist (36)
      • ‎
    		WLC (37)Universal
    Mori et ala,d [72]CBTJapan; IT engineers with high computer literacy (168)
    • A 4-week program with homework
      • ‎
    • Web-based
      • ‎
    • 150-minute group class; web-based entries to log daily stresses (85)
      • ‎
    		WLC (83)Universal
    Phillips et ala [73]CBTUnited Kingdom; transport, health, and communication sectors (637)
    • 5-week web-based “MoodGYM” program
      • ‎
    • 5 modules of 1 hour for preventing and coping with depression (318)
      • ‎
    		WLC+information (319)Tertiary
    Umanodan et ala [74]Stress managementJapan; manufacturing company (266)
    • 6-week web-based program
      • ‎
    • 6 lessons (1 per week); self-paced
      • ‎
    • 2-phased learning process (142)
      • ‎
    		WLC (121)Universal
    Carissoli et ale [75]MindfulnessItaly; general employees (56)
    • 3-week smartphone app
      • ‎
    • Practice 2 meditations per day, lasting 15 minutes each (20)
      • ‎
    		Music; 2 songs per day, lasting approximately 15 minutes each, while doing nothing else (18)Universal
    Cook et ala [76]Stress managementUnited States; older IT employees (278)
    • “HealthyPast50,” a web-based open-access program
      • ‎
    • 3 months, 5 modules (138)
      • ‎
    		WLC (140)Universal
    Ebert et ala [77]Behavior changeGermany; teachers (128)
    • 6-week internet-based weekly sessions (64)
      • ‎
    		WLC (64)Universal
    Guille et al [36]aCBTUnited States; medical interns (199)
    • MoodGYM program, comprising 4 weekly, web-based sessions lasting approximately 30 minutes each (100)
      • ‎
    		4 weekly mental health information emails (99)Universal
    Mak et al [51]aMBSRgChina; university (321)
    • 2-arm intervention: 8-week web-based mindfulness training, 1 lesson per week that took 23 to 30 minutes (107)
      • ‎
    • The second group had the identical training plus health action process approach(107)
      • ‎
    		WLC (107)Universal
    Praseka [78]MindfulnessUnite States; university (192)
    • 7-week self-guided, web-based mindfulness program called Sherman Project (101)
      • ‎
    		WLC (91)Universal
    Stansfeld et ala,d [79]Stress managementUnited Kingdom; NHSh Mental Health Trust (275)
    • Web-based team-based health promotion program
      • ‎
    • 6 fortnightly modules for 3 months (216)
      • ‎
    		WLC (59)Universal
    Volker et al [39]aCBT and PSTiThe Netherlands; sick-listed (773)
    • Return@Work, a web-based program with 5 modules
      • ‎
    • Up to 17 sessions, ranging from 6 to 17 (131)
      • ‎
    		WLC (89)Tertiary
    Yuana [80]PSTHong Kong
    • Happy@Work, a web-based program with 4 modules (162)
      • ‎
    		WLC (159)Universal
    Allexandre et al [53]aMindfulnessUnited States; general employees from a corporate call center (91)
    • 8-week web-based program with 1 session per week; audio guided; daily articles available and 2 email reminders sent (54)
      • ‎
    • Access to the above plus meeting in groups for 1 hour once a week; group session deep breathing exercise for 2 minutes, 10-minute audio recording, 20- to 30-minute guided meditation, and 20 minutes of discussion questions (37)
      • ‎
    • Same as above plus weeks 3, 6, and 8 were facilitated by a licensed clinical (33)
      • ‎
    		WLC (37)Universal
    Birney et ala [81]CBTUnited States (300)
    • 6-week CBT mobile phone app “MoodHacker”
      • ‎
    • Brief daily interactions (150)
      • ‎
    		WLC (150)Tertiary
    Bostock et ala [82]CBTUnited States; office based (270)
    • 8-week web-based Sleepio.com intervention
      • ‎
    • Animated virtual therapist (“The Prof”), sleep diary data, email or SMS text message prompts, and moderated web-based community (135)
      • ‎
    		WLC (135)Universal
    Dyrbye et ala [83]Positive psychologyUnited States; practicing physicians (290)
    • 6-week web-based program
      • ‎
    • One 5-minute domain per week (145)
      • ‎
    		WLC (145)Universal
    Ebert et ala,d [84] (a)Stress managementGermany (264)
    • 7-week web-based intervention GET.ON Stress
      • ‎
    • Weekly modules, 45 to 60 minutes each
      • ‎
    • Daily stress diaries and e-coach (psychologist; 132)
      • ‎
    		WLC (132)Indicated
    Ebert et ala,d [85] (b)Stress managementGermany (264)
    • 7-week web-based intervention GET.ON Stress
      • ‎
    • Weekly modules, 45 to 60 minutes each
      • ‎
    • Daily stress diaries (132)
      • ‎
    		WLC (132)Indicated
    Heber et ala,d [86]Stress managementGermany (264)
    • 7 sessions, with 1 to 2 sessions per week; web-based (132)
      • ‎
    		WLC (132)Indicated
    Hersch et ala [87]Stress managementUnited States; nurses and nurse managers in public hospitals (104)
    • Web-based intervention BREATHE, with open access for 3 months and 7 modules (52)
      • ‎
    		WLC (52)Universal
    Imamura et ala [44]CBTJapan (1236)
    • 4-week internet access to UTSMed, comprising text and illustrations; 90 pages (276)
      • ‎
    • Same as above but for moderate depression (291)
      • ‎
    • Same as above but for high depression (51)
      • ‎
    		WLC no depression (285); same as above but for moderate depression (290); same as above but for high depression (43)Tailored
    Jonas et ala,d [88]CBTGermany; employees with burnout (59)
    • A 4-week web-based program “Beratung Hilft” (“counselling helps”)
      • ‎
    • Structured and therapist guided
      • ‎
    • Daily stress diary (18)
      • ‎
    		WLC (21)Indicated
    Beiwinkel et ala,d [89]CBTGermany (180)
    • “HelpID,” a 12-week web-based program with weekly sessions of 30 to 45 minutes
      • ‎
    • Weekly reminder emails (100)
      • ‎
    		WLC+information (80)Tertiary
    Boß et al [48] (a)a and (b)a,dCBTGermany (434)
    • 5-week web-based program with 1 module per week (146)
      • ‎
    • Identical to the GET.ON intervention plus additional adherence-focused guidance by e-coaches (trained psychologist; 144)
      • ‎
    		WLC (144)Universal
    Carolan et al [49]a,dCBTUnited Kingdom (84)
    • WorkGuru, a web-based program with 7 weekly modules (28)
      • ‎
    • Same as above plus weekly web-based, guided discussion group (28)
      • ‎
    		WLC (28)Indicated
    Shirotsuki et al [52]aCBTJapan; office workers in hospitality and sales company (87)
    • 6-week web-based program where participants watched weekly e-learning movie segments (5-10 minutes long); recorded daily mood and weekly homework (29)
      • ‎
    • Same as above plus participants consumed 1 bottle of the supplement soft drink (29)
      • ‎
    		Recorded their mood state every day on a weekly monitoring sheet (homework; 29)Universal
    Zhang et ala [90]Awareness trainingSingapore; health care workers (80)
    • 4-week smartphone-based program
      • ‎
    • 20-minute briefing session and a daily SMS text messaging service (40)
      • ‎
    		Information from the Health Promotion Board (40)Universal
    Zwerenz et ala,d [91]PsychoeducationGermany; sick-listed and in inpatient rehabilitation (652)
    • 12-week web-based program with weekly reflective 45-minute blogs instructed by a therapist
      • ‎
    • Reminder emails, individualized feedback, audio-guided stress management, homework, and forum (303)
      • ‎
    		Active control; regular email reminders to use selected information posted on the web about stress management and coping (329)Tertiary
    Bostock et ala [92]MindfulnessUnited Kingdom; pharmaceutical and high-tech employees (238)
    • 8-week Headspace smartphone app
      • ‎
    • 45 days of daily 10- to 20-minute mindfulness meditation (128)
      • ‎
    		NHS web-based advice for work stress (110)Universal
    Eriksson et ala [93]Mindful self-compassion programSweden; practicing psychologists (101)
    • 6 weeks web-based program
      • ‎
    • Videos with guided instructions (51)
      • ‎
    		WLC (49)Universal
    Gollwitzer et al [50]aMental contrastingGermany; general nurses (129)
    • Web-based 3-week program; daily written reflections on “what if”; identifying obstacles in the way (41)
      • ‎
    • Same as above plus structured daily time for use (41)
      • ‎
    		WLC (47)Universal
    Hamamura et ale [94]CBTJapan; marketing company (557)
    • A 4-week smartphone app called “jibun kiroku”
      • ‎
    • Daily activities on an hourly basis
      • ‎
    • Evaluate the quality of their sleep, mood, and energy level (306)
      • ‎
    		WLC (251)Indicated
    Imamura et al [37]a,dCBTJapan; telecommunications (706)
    • 6-week 6-lesson web-based program with 30-minute lessons per week
      • ‎
    • Voluntary homework and reminder emails (353)
      • ‎
    		WLC (353)Universal
    Lilly et ala [95]MBSRUnited States and Canada; emergency telecommunication (323)
    • 7-week web-based (Destress 9-1-1) program, with 1 module per week for 30 minutes; introduction video, texts, and a moderated discussion board; outside practice for up to 45 minutes of daily mindfulness homework
      • ‎
    • Guided audio (163)
      • ‎
    		WLC (160)Universal
    Mistretta et ala [96]MBRTjUnited States; hospital employees (38)
    • 6-week smartphone app (23)
      • ‎
    		WLC (15)Indicated
    Oishi et ala [97]CBTJapan; teachers (240)
    • 12-week web-based “Mind Skill Up Training” program
      • ‎
    • 7 modules plus 1 group session, including homework, mood tracking, audio, and visual narrator, with 6 reminder emails (120)
      • ‎
    		WLC (120)Universal
    Persson Asplund et ala,d [98]Stress managementSweden; middle managers in health care, education, IT, or communications sectors (117)
    • 8 weekly modules
      • ‎
    • 2 to 3 hours per week to complete
      • ‎
    • Personalized written feedback via email from a coach (psychologist; 59)
      • ‎
    		Active control; weekly mail contact, homework, and access to a moderated discussion forum with other users (58)Indicated
    Querstret et ala [99]MBCTkUnited Kingdom (118)
    • 4-week web-based program (60)
      • ‎
    		WLC (58)Universal
    Song et ala [100]Health Surveillance ModelJapan (1526)
    • 16-week smartphone app “Karada-no-kimochi”
      • ‎
    • Records weekly and daily moods (612)
      • ‎
    		WLC (914)Universal
    Coelhoso et ala,d [101]Mindfulness meditationBrazil; female private hospital employees (490)
    • 8-week smartphone app 4 with classes per week
      • ‎
    • Each class contained a brief theoretical portion and a 15-minute guided practice
      • ‎
    • Participants wrote reflections in a gratitude journal for 20 minutes per week
      • ‎
    • Pop-ups, notifications, and feedback scores (250)
      • ‎
    		Active control; similar program (240)Universal
    Stratton et ala [102]Disclosure decision aid toolAustralia (107)
    • READY web-based disclosure decision aid tool
      • ‎
    • 2-week access; 7 modules, requiring approximately 60 minutes to complete (53)
      • ‎
    		Information provided about disclosure on leading NGOl website (54)Tertiary
    Weber et ala [103]Behavior changeGermany, England, and Northern Ireland (532)
    • 4-week smartphone app “Kelaa Mental Resilience App”
      • ‎
    • 28 sessions daily with 2 key modules
      • ‎
    • Tracks mood and health and provides feedback; 6 to 7 daily sessions, each approximately 2 to 4 minutes (210)
      • ‎
    		WLC (322)Universal
    Deady et ala [104]Behavior activation and mindfulnessAustralia; male-dominated industries (2257)
    • HeadGear, a smartphone app
      • ‎
    • 30-day “challenge” daily (5-10 minutes per day)
      • ‎
    • Risk calculator with personalized feedback, mood tracker, a toolbox of skills, and support service helplines (1131)
      • ‎
    		Active control same intervention but without risk calculator and mood tracker (1144)Universal

    aRandomized controlled trial.

    bCBT: cognitive behavioral therapy.

    cWLC: wait-list control.

    dGuided intervention.

    eControlled trial.

    fACT: acceptance and commitment therapy.

    gMBSR: mindfulness-based stress reduction.

    hNHS: National Health Service.

    iPST: problem-solving therapy.

    jMBRT: mindfulness-based resilience training.

    kMBCT: mindfulness-based cognitive therapy.

    lNGO: nongovernmental organization.

    Efficacy for Anxiety

    Overall, the 29 trials that measured anxiety outcomes showed a significant, small positive effect at the postintervention time point (Hedges g=0.26, 95% CI 0.13-0.39; P<.001), with high heterogeneity (I2=77.49%; τ2=0.08). Approximately 21% (16/75) of studies reported follow-up outcomes, showing a small positive effect (Hedges g=0.32, 95% CI 0.15-0.50; P<.001), with similarly large heterogeneity (I2=76.55%; τ2=0.08; Figure 3).

    ‎
    Figure 3. Effects on anxiety symptoms at postintervention and follow-up time points.
    View this figure
    Efficacy for Depression

    An overall small positive effect on depression at the postintervention time point (Hedges g=0.26, 95% CI 0.19-0.34; P<.001) was estimated by combining 46 trials. Moderate heterogeneity was observed (I2=66.96%; τ2=0.04). At the postintervention time point, one outlier was detected. By removing this outlier with a very large ES, a statistically significant but small magnitude positive effect remained (Hedges g=0.24, 95% CI 0.17-0.32; P<.001). Approximately 60% (45/75) of the studies reported follow-up outcomes. The follow-up effects on depression in 37% (28/75) of studies were very similar (Hedges g=0.23, 95% CI 0.13-0.32; P<.001), again with moderate heterogeneity (I2=68.27%; τ2=0.03; Figure 2).

    Efficacy for Stress

    Stress was the most common mental health outcome assessed in this study. In 76% (57/75) of studies, a small positive effect was found overall at the postintervention time point (Hedges g=0.25, 95% CI 0.17-0.34; P<.001). However, large heterogeneity was detected (I2=76.11%; τ2=0.07). Overall, a significant but small positive effect was observed at follow-up in 40% (30/75) of studies (Hedges g=0.28, 95% CI 0.17-0.40; P<.001), with moderate heterogeneity detected (I2=73.36%; τ2=0.06; Figure 4).

    ‎
    Figure 4. Effects on stress symptoms at postintervention and follow-up time points.
    View this figure
    Moderating Factors for Outcome Efficacy
    Anxiety

    Mixed-effects meta-regression (Table 2) showed that for anxiety, the difference between approach efficacy was statistically significant (Q=21.72; P<.001; R2=0.64). Stress management interventions (Hedges g=0.79, 95% CI 0.64-0.93; P<.001) and mindfulness (Hedges g=0.42, 95% CI 0.14-0.60; P<.001) interventions were more effective than CBT (Hedges g=0.11, 95% CI 0.04-0.19; P=.004) and other interventions (eg, cognitive training; Hedges g=0.13, 95% CI 0.01-0.25; P=.04). There were no significant between-group differences between the intervention approach, level of support, or type of control used.

    Table 2. Meta-regression analysis for moderators of true effect on each mental health outcome at the postintervention time point.
    Intervention component and study design factorStudy, n (%)Meta-analysisHeterogeneityMeta-regression between-group tests


    		Hedges g (95% CI)P valueQP valueI2QP valueR2
    Anxiety

    		Type of intervention


    		CBTa14 (19)0.11 (0.04 to 0.19).00414.41.359.7821.72<.0010.64


    		Mindfulness5 (7)0.42 (0.14 to 0.6)<.00127.99<.00185.7121.72<.0010.64


    		Stress management4 (5)0.79 (0.64 to 0.93)<.0016.47.0953.621.72<.0010.64


    		Other6 (8)0.13 (0.01 to 0.25).045.45.358.221.72<.0010.64

    		Intervention approach


    		Indicated9 (12)0.44 (0.33 to 0.54)<.00160.13<.00186.703.48.320.03


    		Tailored3 (4)0.06 (−0.09 to 0.22).440.76.680.0003.48.320.03


    		Tertiary5 (7)0.14 (0.04 to 0.23).0062.85.580.0003.48.320.03


    		Universal12 (16)0.29 (0.17 to 0.4)<.00138.03<.00171.073.48.320.03

    		Person support


    		Yes14 (19)0.32 (0.24 to 0.4)<.00168.96<.00181.151.02.310.01


    		No15 (20)0.16 (0.07 to 0.24)<.00147.33<.00170.421.02.310.01

    		Type of control


    		Active4 (5)0.26 (0.07 to 0.45).0095.14.1641.640.08.780


    		WLCb25 (33)0.25 (0.19 to 0.31)<.001119.23<.00179.870.08.780
    Depression

    		Type of intervention


    		CBT23 (31)0.11 (0.06 to 0.17)<.00144.12.00350.1320.32<.0010.47


    		Mindfulness5 (7)0.46 (0.28 to 0.64)<.00116.14.00375.2220.32<.0010.47


    		Stress management4 (5)0.61 (0.47 to 0.75)<.0014.03.2625.5220.32<.0010.47


    		Other14 (19)0.15 (0.09 to 0.21)<.00121.25.0738.8320.32<.0010.47

    		Intervention approach


    		Indicated9 (12)0.32 (0.21 to 0.42)<.00145.95<.00185.594.45.220.01


    		Tailored6 (8)0.04 (−0.06 to 0.14).4112.13.0358.794.45.220.01


    		Tertiary11 (15)0.2 (0.12 to 0.28)<.00118.98.0447.314.45.220.01


    		Universal20 (27)0.19 (0.13 to 0.25)<.00144.01.00156.834.45.220.01

    		Person support


    		Yes20 (27)0.33 (0.27 to 0.4)<.00142.04.00254.87.20.0070.27


    		No26 (35)0.11 (0.06 to 0.16)<.00165.58<.00161.887.20.0070.27

    		Type of control


    		Active9 (12)0.19 (0.11 to 0.28)<.00116.78.0352.330.00.950.00


    		WLC37 (49)0.18 (0.14 to 0.23)<.001119.35<.00169.840.00.950.00
    Stress

    		Type of intervention


    		CBT19 (25)0.1 (0.04 to 0.16).00147.89<.00162.4210.12.020.09


    		Mindfulness15 (20)0.42 (0.31 to 0.53)<.00146.31<.00169.7710.12.020.09


    		Stress management14 (19)0.28 (0.2 to 0.37).00671.38<.00188.7910.12.020.09


    		Other9 (12)0.12 (0.04 to 0.21)<.00137.74<.00165.5610.12.020.09

    		Intervention approach


    		Indicated9 (12)0.38 (0.28 to 0.49)<.00156.76<.00185.913.26.350.00


    		Tailored6 (8)0.07 (−0.03 to 0.16).1627.10<.00181.553.26.350.00


    		Tertiary5 (7)0.12 (0.09 to 0.22).014.78.3116.33.26.350.00


    		Universal37 (49)0.18 (0.13 to 0.24)<.001125.06<.00171.213.26.350.00

    		Person support


    		Yes22 (29)0.33 (0.26 to 0.39)<.00180.20<.00173.826.79.0090.15


    		No35 (47)0.09 (0.04 to 0.14).001121.76<.00172.086.79.0090.15

    		Type of control


    		Active7 (9)0.17 (0.05 to 0.29).00618.38.00567.360.03.870.00


    		WLC50 (67)0.18 (0.14 to 0.23)<.001215.97<.00177.310.03.870.00

    aCBT: cognitive behavioral therapy.

    bWLC: wait-list control.

    Depression

    Similarly, significant differences were observed in the ESs for depression across the types of interventions (Q=21.72; P<.001; R2=0.64). Again, stress management interventions (Hedges g=0.61, 95% CI 0.47-0.75; P<.001) and mindfulness (Hedges g=0.46, 95% CI 0.28-0.64; P<.001) interventions were more effective than CBT (Hedges g=0.11, 95% CI 0.06-0.17; P<.001) and other interventions (Hedges g=0.15, 95% CI 0.09-0.21; P<.001). Supported interventions had a higher ES (g=0.33, 95% CI 0.27-0.40; P<.001) than unsupported (Hedges g=0.11, 95% CI 0.06-0.16; P<.001) interventions (difference Q=7.20; P=.007; R2=0.27). There were no significant differences between the intervention approach and the type of control used.

    Stress

    Differences were observed in the ESs across the type of intervention (Q=10.12; P=.02; R2=0.09). Mindfulness interventions had the largest ES (Hedges g=0.42, 95% CI 0.31-0.53; P<.001), followed by stress management (Hedges g=0.28, 95% CI 0.20-0.37; P=.006), which was more effective than CBT (Hedges g=0.10, 95% CI 0.04-0.16; P=.001) and other interventions (Hedges g=0.12, 95% CI 0.04-0.21; P<.001). Supported interventions had a higher ES (Hedges g=0.33, 95% CI 0.26-0.39; P<.001) than unsupported (Hedges g=0.09, 95% CI 0.04-0.14; P=.001). The difference between these factors was statistically significant (Q=6.79; P=.009; R2=0.15). For stress, there were no significant differences between the intervention approach and type of control used.

    Effectiveness of eHealth Interventions Over Time
    Overview

    Over 5 times more studies evaluating eHealth interventions have been published since 2013 (n=64) than between 2004 and 2013 (n=11). However, the effectiveness of eHealth interventions did not seem to improve over time. The mean within-group ES reported in each year of the eHealth intervention arms in trials published since 2004 remained unchanged, with no significant SMD observed over time for anxiety (F1,9=0.28; P=.61), depression (F1,10=0.31; P=.59), or stress (F1,11=0.75l; P=.41; Figure 5). There appears to have been a nadir in the effectiveness of studies published immediately after the financial crash and recession of 2008/2009.

    ‎
    Figure 5. Mean within-intervention group effect sizes reported in trials each year.
    View this figure
    Small Study Effect

    A funnel plot for each mental health outcome is shown in Multimedia Appendix 6. No significant asymmetry was found for anxiety outcomes (n=29; Egger intercept 0.24; P=.80). However, significant asymmetry was observed for depression (n=46; Egger intercept 1.71; P=.001). After conducting a trim and fill analysis, 3% (2/75) of studies were imputed; the observed postintervention ES was adjusted to Hedges g=0.28 (95% CI 0.18-0.39). Similarly, significant asymmetry was observed in stress (n=57; Egger intercept 1.89; P=.004). After conducting a trim and fill analysis, 23% (17/75) of studies were imputed; the observed postintervention ES was adjusted to Hedges g=0.08 (95% CI 0.00-0.19), suggesting a greater effect reported in smaller studies.


    Principal Findings

    This updated review aimed to synthesize the burgeoning literature on the efficacy of eHealth interventions for anxiety, depression, and stress outcomes in organizational settings and employee samples. We identified 52 new trial interventions published since the 23 identified in our prior review in 2017 [19], a tripling of the body of evidence. The systematic search identified 75 relevant trials for the meta-analysis, delivering eHealth interventions either on the web or via a smartphone, with a combined sample of 14,747 employees.

    eHealth interventions reduced mental health symptoms immediately after use, with small positive effects observed in anxiety (Hedges g=0.26), depression (Hedges g=0.26), and stress (Hedges g=0.25), and data from trials with longer follow-up periods showed similar effects. These results are comparable with those of previous reviews, where small overall effects were found at the postintervention and follow-up time points [19,20]. These results imply that since 2017, the efficacy of eHealth interventions compared with control conditions reported in trials has remained unchanged, suggesting that the effectiveness of eHealth interventions does not seem to be improving over time. An analysis of the within-group effect observed in the intervention arms confirmed that since 2004, there has been no apparent systematic improvement in the effectiveness of these interventions. This is a surprising finding, given the enormous literature on methods to improve engagement with such interventions, greater penetration of technology to populations that do not access health care, and the increasing number of interventions delivered as apps, which reportedly improve access. All of these are commonly cited as factors favoring eHealth as a mode of intervention delivery or improving effectiveness.

    The small but significant effect at follow-up suggests that eHealth interventions might have sustained positive effects on mental health. However, only half of the studies assessed follow-up outcomes, which may reflect a reporting bias. Supporting this, null, and in some cases, even negative, effects on mental health were observed in 27% (20/75) of the studies, and a greater proportion—60% (45/75)—did not report follow-up outcomes.

    Less than one-fifth of all the trials used active controls. However, we found no significant differences in the efficacy of interventions tested against active versus passive controls overall or for any individual mental health outcome (Table 2). These findings do not confirm the digital placebo effect, which has been found in other eHealth reviews in clinical populations, where the ESs were lower in trials that used active controls than in those that used passive controls [24]. Given that most (47/75, 63%) of these trials were not conducted in clinically unwell populations but were delivered universally to employees, this contrary finding may have implications for the types of control recommended for different settings of eHealth trials and for future framework analysis or guidelines.

    The primary moderator of efficacy appeared to be the content of the intervention, with stress management and mindfulness-based interventions being seemingly more efficacious than CBT-based interventions. This suggests that CBT, which was adapted from its intended use in a more personalized and clinical setting, may have less useful content, especially for use in organizations with universal delivery, than other approaches adapted for more universal or indicated tools such as mindfulness and stress management.

    In our previous review, stronger effects were seen in eHealth interventions that were supported [19], and other studies have found increased effects when eHealth interventions were supported by trained mental health professionals [105]. The positive impacts of supported interventions may suggest that even in generally subclinical populations, a combination of eHealth and adjunctive support is the most effective in reducing mental health symptoms in employees.

    Previous research has pointed to the importance of tailoring eHealth interventions to match individual user needs, as mental health symptoms differ from person to person [106]. However, this review did not support the notion that the tailoring of interventions provides any benefit to improving mental health. The limited evidence available suggests no greater efficacy when using tailored interventions than when using interventions delivered universally or for indicated or even unwell samples. In fact, 43% (3/7) of the tailored interventions showed potential for harm in at least one mental health outcome.

    Limitations

    This study had some limitations. Significant heterogeneity was detected; however, this is not uncommon in this field or in meta-analytic research [24]. Differences across studies should be considered when interpreting these findings. It must also be acknowledged that some of the follow-up analyses were underpowered and that the findings should be interpreted tentatively. Furthermore, this review did not include any gray literature, and some authors did not provide data. Finally, the individual study pooled data calculated using the Comprehensive Meta-Analysis by Biostat Inc are not presented in this manuscript. Instead, we report the raw data of the pre-post means, SDs, and sample sizes in Table 1.

    Future Directions

    As we were unable to detect that eHealth interventions are improving over time, to design effective eHealth interventions, a better understanding of the factors that may influence efficacy is required. This study did not consider engagement and adherence and only considered a small number of potential moderating factors for each intervention. A subsequent framework analysis model is being undertaken to establish the potential beneficial or harmful features of the different types of interventions. This is an important future direction and requires further in-depth analysis as eHealth interventions have the potential to offer a range of novel self-management tools for employees with clinical and subclinical mental health conditions.

    Another recommendation is to develop a standard framework for eHealth interventions to best understand the features that have therapeutic benefits and those that may potentially cause harm. This will guide developers to ensure that eHealth interventions are designed in the most effective manner. A framework may address the significant heterogeneity within studies; if the development standards are regulated against this framework, the differences in the features of the interventions and how they are delivered may be reduced. This research suggests that those evaluating interventions or designing protocols should carefully consider the level of support provided when interpreting reported ESs.

    Conclusions

    This review and meta-analysis confirms that eHealth interventions have a small but positive impact overall on reducing mental health symptoms in employees. There was significant heterogeneity between trials; however, overall, stress management and mindfulness interventions comprising in-person support appeared to be the most effective. Organizations should carefully consider the interventions delivered within the workplace; otherwise, they may not see long-term value in their return on investments. A substantial minority of intervention trials have demonstrated no efficacy, and a few may even be harmful. There is not enough evidence to make recommendations for the preferential use or development of therapeutically tailored interventions. Disappointingly, we found no evidence that despite the advancements in technology and the enormous resources in time, research, and finance devoted to this area for over a decade, better interventions are being produced. Hopefully, these small ESs do not represent the optimum outcome in organizational settings, and stakeholders and researchers should focus on improving effectiveness and efficacy or comparing and understanding the effects of current interventions. Despite the small effectiveness, this level of reduction could result in a large economic value in an organizational setting.

    Authors' Contributions

    ES contributed to the conceptualization, data curation, formal analysis, investigation, methodology, validation, visualization, writing of the original draft, and review and editing of the manuscript. AL and IC contributed to data curation, methodology, and review and editing of the manuscript. IC contributed to data curation, methodology, and review and editing of the manuscript. HMG contributed to the data curation and review and editing of the manuscript. MA contributed to screening and study identification. JT contributed to the methodology, analysis plan, and review and editing of the manuscript. RAC and SMH contributed to the methodology and review and editing of the manuscript. NG contributed to data curation, investigation, methodology, supervision, and review and editing of the manuscript.

    Conflicts of Interest

    NG has a patent PCT/AU2019/051186 pending. NG and ES own the IP of a web-based disclosure decision aid tool for employees.

    ‎
    Multimedia Appendix 1

    Example of search terms.

    DOCX File , 38 KB
    ‎
    Multimedia Appendix 2

    References of the included studies.

    DOCX File , 71 KB
    ‎
    Multimedia Appendix 3

    Estimated risk of bias.

    PNG File , 278 KB
    ‎
    Multimedia Appendix 4

    Mental health measures.

    DOCX File , 28 KB
    ‎
    Multimedia Appendix 5

    Detailed description of the identified studies and results.

    DOCX File , 101 KB
    ‎
    Multimedia Appendix 6

    Overall studies small study effect.

    PNG File , 51 KB
    1. Harvey SB, Modini M, Joyce S, Milligan-Saville JS, Tan L, Mykletun A, et al. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup Environ Med 2017 Mar;74(4):301-310. [CrossRef] [Medline]
    2. Glozier N. Mental ill health and fitness for work. Occup Environ Med 2002 Oct;59(10):714-720 [FREE Full text] [CrossRef] [Medline]
    3. Harvey SB, Henderson M, Lelliott P, Hotopf M. Mental health and employment: much work still to be done. Br J Psychiatry 2009 Mar;194(3):201-203. [CrossRef] [Medline]
    4. Leka S, Jain A, Iavicoli S, Vartia M, Ertel M. The role of policy for the management of psychosocial risks at the workplace in the European Union. Saf Sci 2011 Apr;49(4):558-564. [CrossRef]
    5. Stevenson D, Farmer P. Thriving at Work: a review of mental health and employers. United Kingdom Government. 2017 Oct 26.   URL: https://www.gov.uk/government/publications/thriving-at-work-a-review-of-mental-health-and-employers [accessed 2022-10-10]
    6. Knudsen AK, Harvey SB, Mykletun A, Øverland S. Common mental disorders and long-term sickness absence in a general working population. The Hordaland Health Study. Acta Psychiatr Scand 2013 Apr;127(4):287-297. [CrossRef] [Medline]
    7. Knudsen AK, Øverland S, Aakvaag HF, Harvey SB, Hotopf M, Mykletun A. Common mental disorders and disability pension award: seven year follow-up of the HUSK study. J Psychosom Res 2010 Jul;69(1):59-67. [CrossRef] [Medline]
    8. Harvey SB, Joyce S, Tan L, Johnson A, Nguyen H, Modini M, et al. Developing a mentally healthy workplace: a review of the literature – a report for the National Mental Health Commission and the Mentally Healthy Workplace Alliance. University of New South Wales. 2014 Nov.   URL: https:/​/www.​headsup.org.au/​docs/​default-source/​resources/​developing-a-mentally-healthy-workplace_final-november-2014.​pdf?sfvrsn=8 [accessed 2022-10-10]
    9. Henderson M, Harvey SB, Overland S, Mykletun A, Hotopf M. Work and common psychiatric disorders. J R Soc Med 2011 May;104(5):198-207 [FREE Full text] [CrossRef] [Medline]
    10. Ebert DD, Kählke F, Buntrock C, Berking M, Smit F, Heber E, et al. A health economic outcome evaluation of an Internet-based mobile-supported stress management intervention for employees. Scand J Work Environ Health 2018 Mar 01;44(2):171-182 [FREE Full text] [CrossRef] [Medline]
    11. Can Tech Help Manage Mental Health in the Workplace? CorporateWellnessMagazine. 2018.   URL: https://www.corporatewellnessmagazine.com/article/tech-manage-mental-health-workplace [accessed 2020-08-04]
    12. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010 Oct 13;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
    13. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, REEACT Team. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015 Nov 11;351:h5627 [FREE Full text] [CrossRef] [Medline]
    14. So M, Yamaguchi S, Hashimoto S, Sado M, Furukawa TA, McCrone P. Is computerised CBT really helpful for adult depression?-A meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity. BMC Psychiatry 2013 Apr 15;13:113 [FREE Full text] [CrossRef] [Medline]
    15. Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
    16. Spijkerman MP, Pots WT, Bohlmeijer ET. Effectiveness of online mindfulness-based interventions in improving mental health: a review and meta-analysis of randomised controlled trials. Clin Psychol Rev 2016 Apr;45:102-114 [FREE Full text] [CrossRef] [Medline]
    17. McIntyre RS, Florea I, Tonnoir B, Loft H, Lam RW, Christensen MC. Efficacy of vortioxetine on cognitive functioning in working patients with major depressive disorder. J Clin Psychiatry 2017 Jan;78(1):115-121 [FREE Full text] [CrossRef] [Medline]
    18. Donkin L, Glozier N. Motivators and motivations to persist with online psychological interventions: a qualitative study of treatment completers. J Med Internet Res 2012 Jun 22;14(3):e91 [FREE Full text] [CrossRef] [Medline]
    19. Stratton E, Lampit A, Choi I, Calvo RA, Harvey SB, Glozier N. Effectiveness of eHealth interventions for reducing mental health conditions in employees: a systematic review and meta-analysis. PLoS One 2017 Dec 21;12(12):e0189904 [FREE Full text] [CrossRef] [Medline]
    20. Carolan S, Harris PR, Cavanagh K. Improving employee well-being and effectiveness: systematic review and meta-analysis of Web-based psychological interventions delivered in the workplace. J Med Internet Res 2017 Jul 26;19(7):e271 [FREE Full text] [CrossRef] [Medline]
    21. Phillips EA, Gordeev VS, Schreyögg J. Effectiveness of occupational e-mental health interventions: a systematic review and meta-analysis of randomized controlled trials. Scand J Work Environ Health 2019 Nov 01;45(6):560-576 [FREE Full text] [CrossRef] [Medline]
    22. Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 2016 Feb;3(2):100-102. [CrossRef] [Medline]
    23. Kauer SD, Reid SC, Crooke AH, Khor A, Hearps SJ, Jorm AF, et al. Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012 Jun 25;14(3):e67 [FREE Full text] [CrossRef] [Medline]
    24. Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [CrossRef] [Medline]
    25. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    26. Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. London, UK: The Cochrane Collaboration; Jan 01, 2013.
    27. Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019 Aug 28;366:l4898. [CrossRef] [Medline]
    28. Hedges L, Olkin I. Statistical Methods for Meta-Analysis. New York, NY, USA: Academic Press; 1985.
    29. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons; 2009.
    30. Cumpston M, Li T, Page MJ, Chandler J, Welch VA, Higgins JP, et al. Updated guidance for trusted systematic reviews: a new edition of the Cochrane Handbook for Systematic Reviews of Interventions. Cochrane Database Syst Rev 2019 Oct 03;10:ED000142. [CrossRef] [Medline]
    31. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002 Jun 15;21(11):1539-1558. [CrossRef] [Medline]
    32. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 2010 Apr;1(2):97-111. [CrossRef] [Medline]
    33. Rothstein HR, Sutton AJ, Borenstein M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. Chichester, UK: John Wiley & Sons; 2005.
    34. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000 Jun;56(2):455-463. [CrossRef] [Medline]
    35. Griffiths KM, Bennett K, Walker J, Goldsmid S, Bennett A. Effectiveness of MH-Guru, a brief online mental health program for the workplace: a randomised controlled trial. Internet Interv 2016 Nov;6:29-39 [FREE Full text] [CrossRef] [Medline]
    36. Guille C, Zhao Z, Krystal J, Nichols B, Brady K, Sen S. Web-based cognitive behavioral therapy intervention for the prevention of suicidal ideation in medical interns: a randomized clinical trial. JAMA Psychiatry 2015 Dec;72(12):1192-1198 [FREE Full text] [CrossRef] [Medline]
    37. Imamura K, Furukawa TA, Matsuyama Y, Shimazu A, Kuribayashi K, Kasai K, et al. Differences in the effect of Internet-based cognitive behavioral therapy for improving nonclinical depressive symptoms among workers by time preference: randomized controlled trial. J Med Internet Res 2018 Aug 10;20(8):e10231 [FREE Full text] [CrossRef] [Medline]
    38. Ketelaar SM, Nieuwenhuijsen K, Gärtner FR, Bolier L, Smeets O, Sluiter JK. Effect of an E-mental health approach to workers' health surveillance versus control group on work functioning of hospital employees: a cluster-RCT. PLoS One 2013 Sep 12;8(9):e72546 [FREE Full text] [CrossRef] [Medline]
    39. Volker D, Zijlstra-Vlasveld MC, Anema JR, Beekman AT, Brouwers EP, Emons WH, et al. Effectiveness of a blended Web-based intervention on return to work for sick-listed employees with common mental disorders: results of a cluster randomized controlled trial. J Med Internet Res 2015 May 13;17(5):e116 [FREE Full text] [CrossRef] [Medline]
    40. Cavanagh K, Churchard A, O'Hanlon P, Mundy T, Votolato P, Jones F, et al. A randomised controlled trial of a brief online mindfulness-based intervention in a non-clinical population: replication and extension. Mindfulness (N Y) 2018;9(4):1191-1205 [FREE Full text] [CrossRef] [Medline]
    41. Villani D, Grassi A, Cognetta C, Cipresso P, Toniolo D, Riva G. The effects of a mobile stress management protocol on nurses working with cancer patients: a preliminary controlled study. Stud Health Technol Inform 2012;173:524-528. [Medline]
    42. Geraedts AS, Kleiboer AM, Twisk J, Wiezer NM, van Mechelen W, Cuijpers P. Long-term results of a Web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res 2014 Jul 09;16(7):e168 [FREE Full text] [CrossRef] [Medline]
    43. Geraedts AS, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P. Short-term effects of a Web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res 2014 May 06;16(5):e121 [FREE Full text] [CrossRef] [Medline]
    44. Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Effects of an Internet-based cognitive behavioral therapy (iCBT) program in Manga format on improving subthreshold depressive symptoms among healthy workers: a randomized controlled trial. PLoS One 2014 May 20;9(5):e97167 [FREE Full text] [CrossRef] [Medline]
    45. Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Does Internet-based cognitive behavioral therapy (iCBT) prevent major depressive episode for workers? A 12-month follow-up of a randomized controlled trial. Psychol Med 2015 Jul;45(9):1907-1917. [CrossRef] [Medline]
    46. Eisen KP, Allen GJ, Bollash M, Pescatello LS. Stress management in the workplace: a comparison of a computer-based and an in-person stress-management intervention. Comput Human Behav 2008 Mar;24(2):486-496. [CrossRef]
    47. Wolever RQ, Bobinet KJ, McCabe K, Mackenzie ER, Fekete E, Kusnick CA, et al. Effective and viable mind-body stress reduction in the workplace: a randomized controlled trial. J Occup Health Psychol 2012 Apr;17(2):246-258. [CrossRef] [Medline]
    48. Boß L, Lehr D, Schaub MP, Paz Castro R, Riper H, Berking M, et al. Efficacy of a Web-based intervention with and without guidance for employees with risky drinking: results of a three-arm randomized controlled trial. Addiction 2018 Apr;113(4):635-646 [FREE Full text] [CrossRef] [Medline]
    49. Carolan S, Harris PR, Greenwood K, Cavanagh K. Increasing engagement with an occupational digital stress management program through the use of an online facilitated discussion group: results of a pilot randomised controlled trial. Internet Interv 2017 Dec;10:1-11 [FREE Full text] [CrossRef] [Medline]
    50. Gollwitzer PM, Mayer D, Frick C, Oettingen G. Promoting the self-regulation of stress in health care providers: an Internet-based intervention. Front Psychol 2018 Jun 15;9:838 [FREE Full text] [CrossRef] [Medline]
    51. Mak WW, Chan AT, Cheung EY, Lin CL, Ngai KC. Enhancing Web-based mindfulness training for mental health promotion with the health action process approach: randomized controlled trial. J Med Internet Res 2015 Jan 19;17(1):e8 [FREE Full text] [CrossRef] [Medline]
    52. Shirotsuki K, Nonaka Y, Abe K, Adachi SI, Adachi S, Kuboki T, et al. The effect for Japanese workers of a self-help computerized cognitive behaviour therapy program with a supplement soft drink. Biopsychosoc Med 2017 Sep 19;11:23 [FREE Full text] [CrossRef] [Medline]
    53. Allexandre D, Bernstein AM, Walker E, Hunter J, Roizen MF, Morledge TJ. A Web-based mindfulness stress management program in a corporate call center: a randomized clinical trial to evaluate the added benefit of onsite group support. J Occup Environ Med 2016 Mar;58(3):254-264 [FREE Full text] [CrossRef] [Medline]
    54. Grime PR. Computerized cognitive behavioural therapy at work: a randomized controlled trial in employees with recent stress-related absenteeism. Occup Med (Lond) 2004 Aug;54(5):353-359. [CrossRef] [Medline]
    55. Hasson D, Anderberg UM, Theorell T, Arnetz BB. Psychophysiological effects of a Web-based stress management system: a prospective, randomized controlled intervention study of IT and media workers [ISRCTN54254861]. BMC Public Health 2005 Jul 25;5:78 [FREE Full text] [CrossRef] [Medline]
    56. Shimazu A, Kawakami N, Irimajiri H, Sakamoto M, Amano S. Effects of Web-based psychoeducation on self-efficacy, problem solving behavior, stress responses and job satisfaction among workers: a controlled clinical trial. J Occup Health 2005 Sep;47(5):405-413 [FREE Full text] [CrossRef] [Medline]
    57. Cook RF, Billings DW, Hersch RK, Back AS, Hendrickson A. A field test of a Web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity: randomized controlled trial. J Med Internet Res 2007 Jun 19;9(2):e17 [FREE Full text] [CrossRef] [Medline]
    58. Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B. E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther 2007;36(3):179-192. [CrossRef] [Medline]
    59. Billings DW, Cook RF, Hendrickson A, Dove DC. A Web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med 2008 Aug;50(8):960-968 [FREE Full text] [CrossRef] [Medline]
    60. Suzuki E, Tsuchiya M, Hirokawa K, Taniguchi T, Mitsuhashi T, Kawakami N. Evaluation of an Internet-based self-help program for better quality of sleep among Japanese workers: a randomized controlled trial. J Occup Health 2008 Oct 20;50(5):387-399 [FREE Full text] [CrossRef] [Medline]
    61. Yamagishi M, Kobayashi T, Nakamura Y. Effects of web-based career identity training for stress management among Japanese nurses: a randomized control trial. J Occup Health 2008;50(2):191-193 [FREE Full text] [CrossRef] [Medline]
    62. Abbott JA, Klein B, Hamilton C, Rosenthal A. The impact of online resilience training for sales managers on wellbeing and performance. E J Appl Psychol 2009 Jun 15;5(1):89-95. [CrossRef]
    63. Bennett JB, Broome KM, Schwab-Pilley A, Gilmore P. A Web-based approach to address cardiovascular risks in managers: results of a randomized trial. J Occup Environ Med 2011 Aug;53(8):911-918 [FREE Full text] [CrossRef] [Medline]
    64. Glück TM, Maercker A. A randomized controlled pilot study of a brief Web-based mindfulness training. BMC Psychiatry 2011 Nov 08;11:175 [FREE Full text] [CrossRef] [Medline]
    65. Borness C, Proudfoot J, Crawford J, Valenzuela M. Putting brain training to the test in the workplace: a randomized, blinded, multisite, active-controlled trial. PLoS One 2013;8(3):e59982 [FREE Full text] [CrossRef] [Medline]
    66. Feicht T, Wittmann M, Jose G, Mock A, von Hirschhausen E, Esch T. Evaluation of a seven-week Web-based happiness training to improve psychological well-being, reduce stress, and enhance mindfulness and flourishing: a randomized controlled occupational health study. Evid Based Complement Alternat Med 2013;2013:676953 [FREE Full text] [CrossRef] [Medline]
    67. Lappalainen P, Kaipainen K, Lappalainen R, Hoffrén H, Myllymäki T, Kinnunen ML, et al. Feasibility of a personal health technology-based psychological intervention for men with stress and mood problems: randomized controlled pilot trial. JMIR Res Protoc 2013 Jan 09;2(1):e1 [FREE Full text] [CrossRef] [Medline]
    68. Villani D, Grassi A, Cognetta C, Toniolo D, Cipresso P, Riva G. Self-help stress management training through mobile phones: an experience with oncology nurses. Psychol Serv 2013 Aug;10(3):315-322. [CrossRef] [Medline]
    69. Bolier L, Ketelaar SM, Nieuwenhuijsen K, Smeets O, Gärtner FR, Sluiter JK. Workplace mental health promotion online to enhance well-being of nurses and allied health professionals: a cluster-randomized controlled trial. Internet Interv 2014 Oct;1(4):196-204. [CrossRef]
    70. Deitz D, Cook RF, Hersch RK, Leaf S. Heart healthy online: an innovative approach to risk reduction in the workplace. J Occup Environ Med 2014 May;56(5):547-553 [FREE Full text] [CrossRef] [Medline]
    71. Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: a randomized controlled trial. Internet Interv 2014 Jul;1(3):95-101. [CrossRef]
    72. Mori M, Tajima M, Kimura R, Sasaki N, Somemura H, Ito Y, et al. A Web-based training program using cognitive behavioral therapy to alleviate psychological distress among employees: randomized controlled pilot trial. JMIR Res Protoc 2014 Dec 02;3(4):e70 [FREE Full text] [CrossRef] [Medline]
    73. Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med 2014 Mar;44(4):741-752 [FREE Full text] [CrossRef] [Medline]
    74. Umanodan R, Shimazu A, Minami M, Kawakami N. Effects of computer-based stress management training on psychological well-being and work performance in japanese employees: a cluster randomized controlled trial. Ind Health 2014;52(6):480-491 [FREE Full text] [CrossRef] [Medline]
    75. Carissoli C, Villani D, Riva G. Does a meditation protocol supported by a mobile application help people reduce stress? Suggestions from a controlled pragmatic trial. Cyberpsychol Behav Soc Netw 2015 Jan;18(1):46-53. [CrossRef] [Medline]
    76. Cook RF, Hersch RK, Schlossberg D, Leaf SL. A Web-based health promotion program for older workers: randomized controlled trial. J Med Internet Res 2015 Mar 25;17(3):e82 [FREE Full text] [CrossRef] [Medline]
    77. Ebert DD, Berking M, Thiart H, Riper H, Laferton JA, Cuijpers P, et al. Restoring depleted resources: efficacy and mechanisms of change of an Internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol 2015 Dec;34S:1240-1251. [CrossRef] [Medline]
    78. Prasek A. Randomized Controlled Trial to Evaluate a Self-Guided, Web-Based Mindfulness Program for Stress Reduction and Wellbeing Promotion. University of Minnesota. 2015 Aug.   URL: https://core.ac.uk/download/pdf/76357635.pdf [accessed 2022-08-29]
    79. Stansfeld SA, Kerry S, Chandola T, Russell J, Berney L, Hounsome N, et al. Pilot study of a cluster randomised trial of a guided e-learning health promotion intervention for managers based on management standards for the improvement of employee well-being and reduction of sickness absence: GEM Study. BMJ Open 2015 Oct 26;5(10):e007981 [FREE Full text] [CrossRef] [Medline]
    80. Yuan Q. Evaluating the Effectiveness of A Psychological Capital Development Program on Mental Health, Engagement and Work Performance. The Chinese University of Hong Kong. 2015.   URL: https://www.proquest.com/docview/1846509447?pq-origsite=gscholar&fromopenview=true [accessed 2021-10-10]
    81. Birney AJ, Gunn R, Russell JK, Ary DV. MoodHacker mobile Web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth 2016 Jan 26;4(1):e8 [FREE Full text] [CrossRef] [Medline]
    82. Bostock S, Crosswell AD, Prather AA, Steptoe A. Mindfulness on-the-go: effects of a mindfulness meditation app on work stress and well-being. J Occup Health Psychol 2019 Feb;24(1):127-138 [FREE Full text] [CrossRef] [Medline]
    83. Dyrbye LN, West CP, Richards ML, Ross HJ, Satele D, Shanafelt TD. A randomized, controlled study of an online intervention to promote job satisfaction and well-being among physicians. Burnout Res 2016 Sep;3(3):69-75. [CrossRef]
    84. Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, et al. Self-guided Internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med 2016 May;73(5):315-323. [CrossRef] [Medline]
    85. Ebert DD, Lehr D, Boß L, Riper H, Cuijpers P, Andersson G, et al. Efficacy of an Internet-based problem-solving training for teachers: results of a randomized controlled trial. Scand J Work Environ Health 2014 Nov;40(6):582-596 [FREE Full text] [CrossRef] [Medline]
    86. Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res 2016 Jan 27;18(1):e21 [FREE Full text] [CrossRef] [Medline]
    87. Hersch RK, Cook RF, Deitz DK, Kaplan S, Hughes D, Friesen MA, et al. Reducing nurses' stress: a randomized controlled trial of a web-based stress management program for nurses. Appl Nurs Res 2016 Nov;32:18-25 [FREE Full text] [CrossRef] [Medline]
    88. Jonas B, Leuschner F, Tossmann P. Efficacy of an Internet-based intervention for burnout: a randomized controlled trial in the German working population. Anxiety Stress Coping 2017 Mar;30(2):133-144. [CrossRef] [Medline]
    89. Beiwinkel T, Eißing T, Telle N, Siegmund-Schultze E, Rössler W. Effectiveness of a Web-based intervention in reducing depression and sickness absence: randomized controlled trial. J Med Internet Res 2017 Jun 15;19(6):e213 [FREE Full text] [CrossRef] [Medline]
    90. Zhang H, Jiang Y, Nguyen HD, Poo DC, Wang W. The effect of a smartphone-based coronary heart disease prevention (SBCHDP) programme on awareness and knowledge of CHD, stress, and cardiac-related lifestyle behaviours among the working population in Singapore: a pilot randomised controlled trial. Health Qual Life Outcomes 2017 Mar 14;15(1):49 [FREE Full text] [CrossRef] [Medline]
    91. Zwerenz R, Becker J, Gerzymisch K, Siepmann M, Holme M, Kiwus U, et al. Evaluation of a transdiagnostic psychodynamic online intervention to support return to work: a randomized controlled trial. PLoS One 2017 May 8;12(5):e0176513 [FREE Full text] [CrossRef] [Medline]
    92. Bostock S, Luik AI, Espie CA. Sleep and productivity benefits of digital cognitive behavioral therapy for insomnia: a randomized controlled trial conducted in the workplace environment. J Occup Environ Med 2016 Jul;58(7):683-689. [CrossRef] [Medline]
    93. Eriksson T, Germundsjö L, Åström E, Rönnlund M. Mindful self-compassion training reduces stress and burnout symptoms among practicing psychologists: a randomized controlled trial of a brief Web-based intervention. Front Psychol 2018 Nov 27;9:2340 [FREE Full text] [CrossRef] [Medline]
    94. Hamamura T, Suganuma S, Ueda M, Mearns J, Shimoyama H. Standalone effects of a cognitive behavioral intervention using a mobile phone app on psychological distress and alcohol consumption among Japanese workers: pilot nonrandomized controlled trial. JMIR Ment Health 2018 Mar 22;5(1):e24 [FREE Full text] [CrossRef] [Medline]
    95. Lilly M, Calhoun R, Painter I, Beaton R, Stangenes S, Revere D, et al. Destress 9-1-1-an online mindfulness-based intervention in reducing stress among emergency medical dispatchers: a randomised controlled trial. Occup Environ Med 2019 Oct;76(10):705-711. [CrossRef] [Medline]
    96. Mistretta EG, Davis MC, Temkit M, Lorenz C, Darby B, Stonnington CM. Resilience training for work-related stress among health care workers: results of a randomized clinical trial comparing in-person and smartphone-delivered interventions. J Occup Environ Med 2018 Jun;60(6):559-568. [CrossRef] [Medline]
    97. Oishi S, Takizawa T, Kamata N, Miyaji S, Tanaka K, Miyaoka H. Web-based training program using cognitive behavioral therapy to enhance cognitive flexibility and alleviate psychological distress among schoolteachers: pilot randomized controlled trial. JMIR Res Protoc 2018 Jan 26;7(1):e32 [FREE Full text] [CrossRef] [Medline]
    98. Persson Asplund R, Dagöö J, Fjellström I, Niemi L, Hansson K, Zeraati F, et al. Internet-based stress management for distressed managers: results from a randomised controlled trial. Occup Environ Med 2018 Feb;75(2):105-113 [FREE Full text] [CrossRef] [Medline]
    99. Querstret D, Cropley M, Fife-Schaw C. The effects of an online mindfulness intervention on perceived stress, depression and anxiety in a non-clinical sample: a randomised waitlist control trial. Mindfulness (N Y) 2018;9(6):1825-1836 [FREE Full text] [CrossRef] [Medline]
    100. Song M, Kanaoka H. Effectiveness of mobile application for menstrual management of working women in Japan: randomized controlled trial and medical economic evaluation. J Med Econ 2018 Nov;21(11):1131-1138 [FREE Full text] [CrossRef] [Medline]
    101. Coelhoso CC, Tobo PR, Lacerda SS, Lima AH, Barrichello CR, Amaro E, et al. A new mental health mobile app for well-being and stress reduction in working women: randomized controlled trial. J Med Internet Res 2019 Nov 07;21(11):e14269 [FREE Full text] [CrossRef] [Medline]
    102. Stratton E, Choi I, Calvo R, Hickie I, Henderson C, Harvey SB, et al. Web-based decision aid tool for disclosure of a mental health condition in the workplace: a randomised controlled trial. Occup Environ Med 2019 Sep;76(9):595-602. [CrossRef] [Medline]
    103. Weber S, Lorenz C, Hemmings N. Improving stress and positive mental health at work via an app-based intervention: a large-scale multi-center randomized control trial. Front Psychol 2019 Dec 6;10:2745 [FREE Full text] [CrossRef] [Medline]
    104. Deady M, Choi I, Calvo RA, Glozier N, Christensen H, Harvey SB. eHealth interventions for the prevention of depression and anxiety in the general population: a systematic review and meta-analysis. BMC Psychiatry 2017 Aug 29;17(1):310 [FREE Full text] [CrossRef] [Medline]
    105. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
    106. Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012 Mar;18(2):137-144. [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    ES: effect size
    PICOS: Patient/Population, Intervention, Comparison, and Outcomes
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    PROSPERO: International Prospective Register of Systematic Reviews
    RCT: randomized controlled trial
    SMD: standardized mean difference

    Edited by R Kukafka; submitted 07.03.22; peer-reviewed by E Phillips, L Shen, V Nagy; comments to author 18.05.22; revised version received 26.05.22; accepted 29.07.22; published 27.09.22

    Copyright

    ©Elizabeth Stratton, Amit Lampit, Isabella Choi, Hanna Malmberg Gavelin, Melissa Aji, Jennifer Taylor, Rafael A Calvo, Samuel B Harvey, Nick Glozier. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 27.09.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.