This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297) Anxiety and Stress Disorders (1053)

Published on in Vol 19, No 2 (2017): February

The Benefit of Web- and Computer-Based Interventions for Stress: A Systematic Review and Meta-Analysis

The Benefit of Web- and Computer-Based Interventions for Stress: A Systematic Review and Meta-Analysis

The Benefit of Web- and Computer-Based Interventions for Stress: A Systematic Review and Meta-Analysis

Authors of this article:

Elena Heber1, 2, 3 Author Orcid Image ;   David Daniel Ebert2, 4 Author Orcid Image ;   Dirk Lehr1, 2 Author Orcid Image ;   Pim Cuijpers2, 5 Author Orcid Image ;   Matthias Berking2, 4 Author Orcid Image ;   Stephanie Nobis2 Author Orcid Image ;   Heleen Riper2, 5, 6 Author Orcid Image
Article Authors Cited by (240) Tweetations (32) Metrics

    Original Paper

    1Department of Health Psychology and Applied Biological Psychology, Institute of Psychology, Leuphana University Lueneburg, Lueneburg, Germany

    2Division of Online Health Training, Innovation Incubator, Leuphana University Lueneburg, Lueneburg, Germany

    3Faculty of Social and Human Sciences, University of Southampton, Southampton, United Kingdom

    4Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen Nuremberg, Erlangen, Germany

    5Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

    6Telepsychiatric Centre, University of Southern Denmark, Odense, Denmark

    Corresponding Author:

    Elena Heber, PhD

    Department of Health Psychology and Applied Biological Psychology

    Institute of Psychology

    Leuphana University Lueneburg

    Scharnhorststr 1

    Lueneburg, 21335

    Germany

    Phone: 49 4131 677 ext 2720

    Fax:49 4131 677 1717

    Email: heber@leuphana.de


    Background: Stress has been identified as one of the major public health issues in this century. New technologies offer opportunities to provide effective psychological interventions on a large scale.

    Objective: The aim of this study is to investigate the efficacy of Web- and computer-based stress-management interventions in adults relative to a control group.

    Methods: A meta-analysis was performed, including 26 comparisons (n=4226). Cohen d was calculated for the primary outcome level of stress to determine the difference between the intervention and control groups at posttest. Analyses of the effect on depression, anxiety, and stress in the following subgroups were also conducted: risk of bias, theoretical basis, guidance, and length of the intervention. Available follow-up data (1-3 months, 4-6 months) were assessed for the primary outcome stress.

    Results: The overall mean effect size for stress at posttest was Cohen d=0.43 (95% CI 0.31-0.54). Significant, small effects were found for depression (Cohen d=0.34, 95% CI 0.21-0.48) and anxiety (Cohen d=0.32, 95% CI 0.17-0.47). Subgroup analyses revealed that guided interventions (Cohen d=0.64, 95% CI 0.50-0.79) were more effective than unguided interventions (Cohen d=0.33, 95% CI 0.20-0.46; P=.002). With regard to the length of the intervention, short interventions (≤4 weeks) showed a small effect size (Cohen d=0.33, 95% CI 0.22-0.44) and medium-long interventions (5-8 weeks) were moderately effective (Cohen d=0.59; 95% CI 0.45-0.74), whereas long interventions (≥9 weeks) produced a nonsignificant effect (Cohen d=0.21, 95% CI –0.05 to 0.47; P=.006). In terms of treatment type, interventions based on cognitive behavioral therapy (CBT) and third-wave CBT (TWC) showed small-to-moderate effect sizes (CBT: Cohen d=0.40, 95% CI 0.19-0.61; TWC: Cohen d=0.53, 95% CI 0.35-0.71), and alternative interventions produced a small effect size (Cohen d=0.24, 95% CI 0.12-0.36; P=.03). Early evidence on follow-up data indicates that Web- and computer-based stress-management interventions can sustain their effects in terms of stress reduction in a small-to-moderate range up to 6 months.

    Conclusions: These results provide evidence that Web- and computer-based stress-management interventions can be effective and have the potential to reduce stress-related mental health problems on a large scale.

    J Med Internet Res 2017;19(2):e32

    doi:10.2196/jmir.5774

    Keywords

    stress; mental health; internet-based interventions; review; randomized controlled trial; meta-analysis 


    Chronic stress can lead to serious psychological and physical implications, such as depression [1], sleep problems [2], neck and shoulder disorders [3], and an increased risk of coronary heart disease events [4] and related mortality [5]. Substantial economic costs of stress and stress-related psychological disorders arise as a result of absenteeism, presenteeism, productivity loss, and high staff turnover [6]. Given that stress represents a major threat to public health, effective and scalable solutions to accommodate the demand for stress-management interventions are needed.

    The evidence base on traditional face-to-face stress-management interventions is comprehensive, showing small-to-moderate overall effects in reducing adverse outcomes for mental health [7]. In their meta-analysis on interventions for work-related stress, van der Klink et al [8] reported a combined effect size of Cohen d=0.34 across all studies. The mean effect size was Cohen d=0.33 for depression and Cohen d=0.54 for anxiety for interventions that focused on individuals [8]. Richardson and Rothstein’s more recent meta-analysis [9] on occupational cognitive behavioral, relaxation, organizational, multimodal, or alternative stress-management interventions yielded somewhat larger effects, with an overall effect of Cohen d=0.53, an effect of Cohen d=0.68 for anxiety, and Cohen d=0.73 for stress. Evidence consistently shows cognitive behavioral interventions to be the most effective, with Cohen d values ranging from 0.68 [8] to 1.16 [9]. Some evidence also suggests that shorter interventions (1-4 weeks) are more effective than longer interventions; however, this effect may be confounded by treatment type [9].

    A promising medium to facilitate the dissemination of effective mental health interventions is the Internet. In recent years, Web-based interventions for the prevention and treatment of a range of psychological conditions have proliferated [10-14]. Computer- and Web-based interventions are perceived to offer several advantages that may overcome some of the limitations of face-to-face approaches, including anonymity, 24/7 availability, reduced costs in terms of traveling to courses for both participants and instructors, high scalability, and a low access threshold. Enabling participants to be reached earlier than in classical face-to-face trainings, such interventions may have the potential to prevent the onset of more severe mental health problems [15-18]. Internet-based interventions have been found to be effective in clinical applications to address, for instance, depression [10], anxiety [19], and sleep disorders [20]. However, only a few randomized controlled trials (RCTs) have investigated Web-based stress-management interventions. Research on the eff0.19

    Existing Web-based stress-management trainings differ in various aspects, such as the intervention content, length, and guidance, which may influence their efficacy. First, the theoretical basis of such interventions is diverse, including cognitive behavioral therapy (CBT) [21-26], third-wave cognitive behavioral interventions (TWC) [27-37], the use of olfactory care products [38], and physical exercise programs [39]. Second, the length and number of intervention sessions vary, ranging from short 2-week interventions [33] to interventions that allow access over several months [37]. Some interventions encourage participants to log on only as often as they like, with no specified sessions or requirements to complete the entire intervention [21], whereas others have fixed weekly appointments (eg, in an online virtual classroom) [37]. Third, the existing Web- and computer-based studies include both guided and unguided interventions and thus differ in the amount of human support given to participants during the intervention. In the guided format, individuals normally receive written feedback from a coach on the exercises that they complete within the training. In the study of Ruwaard et al [24], for example, clinical psychology students provided weekly feedback on exercises according to an instruction manual and reminded participants in cases of noncompletion. In contrast, Billings et al [21] used a less intensive, high-latitude format in which no feedback was provided.

    In recent years, the number of studies on Web- and computer-based stress management has been rising. The overall effect of Web-based stress-management interventions and the influence of specific formats of treatment delivery remain unclear. Considering the demand for effective, scalable stress-management trainings and the enormous potential of Web- and computer-based interventions, there is a need to synthesize the results of existing studies. This meta-analysis aims to integrate the effects of Web-based stress-management interventions on the level of stress of adults. Additionally, effects on depression and anxiety will be assessed. The following research questions are addressed:

    Are Web- and computer-based stress-management interventions effective in reducing stress, depression, and anxiety relative to a control group?

    Are there differences in effect sizes concerning (a) the study quality, (b) the level of guidance, (c) the theoretical basis, and (d) the length of the intervention?


    Eligibility Criteria

    We considered RCTs from 1990 to May 2016 in which adult participants (older than 18 years) experienced stress and were participating in the trial to decrease their stress levels. The search was initially conducted in August 2013 and repeated in May 2016 to ensure it was as current as possible. Studies prior to 1990 were excluded; due to the rapidly changing technology in this field, these programs cannot be compared to the current interventions that are likely to be delivered to participants seeking help. The primary intervention target of included studies needed to be a reduction of stress within the target group. We excluded trials that targeted participants with medical conditions (eg, cancer, tinnitus, headache); caregivers of people with medical conditions (eg, caregivers for dementia); and participants with psychiatric disorders (eg, depression, anxiety or posttraumatic stress disorder), posttraumatic symptoms, postpartum emotional distress, or bereavement. The studies included needed to compare an intervention to any type of comparison group. There were no restrictions with regard to dosage or intensity, delivery, duration, frequency or timing of delivery, or the type of delivery channel (eg, email, Web-based, computerized). Trials investigating stress management as merely one part of the intervention (eg, alongside depression or anxiety) were included only if the primary goal of the intervention was to reduce stress. Furthermore, trials conducted in the context of the promotion of well-being rather than stress reduction were excluded. The studies had to report at least one instrument that claimed to measure stress levels. For the main analysis, we considered postintervention data. We also included follow-up data (when available) to assess longer-term effectiveness.

    Information Sources and Search Strategy

    The search strategy for this meta-analysis was created with four categories of search terms. We defined the search terms to meet the following criteria: (1) stress reduction, (2) evaluation of an intervention program, (3) application of a RCT as an experimental design, and (4) delivery in a Web- or computer-based context. A detailed description of the search terms can be found in the Multimedia Appendix 1.

    The specified search strategy was applied in three major database search engines (PsycINFO, PubMed, and Cochrane). Additionally, manual searches in key journals (eg, International Journal of Stress Management, Journal of Medical Internet Research, Journal of Occupational Health, Scandinavian Journal of Work, Environment and Health) and in the reference lists of the included studies were conducted.

    Study Selection

    After removing duplicates of the articles identified, two researchers (EH and SN) independently screened all titles and abstracts based on the inclusion and exclusion criteria, and two researchers (EH and DL) assessed the full-text articles for eligibility. The researchers who assessed the relevance of the studies were not blind to the authors, institutions, journal of publication, or results.

    Data Extraction

    Data were extracted concerning the origin, the number of participants, age, gender, comparison group, outcomes, theoretical basis, length, guidance, the risk of bias, and follow-up data. In cases of insufficient description, the primary investigators of the respective studies were contacted to obtain missing information.

    Risk of Bias

    Two researchers (EH and DL) assessed the risk of bias of the included studies in accordance with the Cochrane Guidelines [40]. Thereby, sequence generation, allocation concealment, blinding, incomplete outcome data, and selective outcome reporting were judged.

    Power Calculation

    Because we expected only a limited number of studies with relatively small sample sizes, we conducted a power calculation to examine how many studies with how many participants would need to be included to assure sufficient statistical power to identify relevant effects. This power calculation was conducted according to the procedures described by Borenstein and colleagues [41]. We hoped to find a sufficient number of studies to be able to identify a small-to-moderate effect size of 0.35. These calculations indicated that we would need to include at least five studies with a mean sample size of 100 (50 participants per condition) to be able to detect an effect size of Cohen d=0.35 (conservatively assuming a medium level of between-study variance, T2, a statistical power of .80, and a significance level, alpha, of .05). Alternatively, we would need three studies with 150 participants each to detect an effect size of Cohen d=0.35 or nine studies with 50 participants.

    Analyses

    Analyses were conducted using the statistical software Comprehensive Meta-Analysis (version 2.2.057). The effect size of subjective level of stress was calculated as the primary outcome. Furthermore, we assessed the levels of depression and anxiety when available. If more than one measure per outcome was used and if no primary outcome was indicated, then the mean of the effect size was calculated to ensure that each study yielded only one effect size. A random-effects model was chosen because of the expectations of considerable heterogeneity between studies. We further conducted a series of subgroup analyses according to the mixed-effects model. In this model, studies within subgroups are pooled with the random-effects model, whereas tests for significant differences between subgroups are conducted with the fixed-effects model. Because of the small number of studies providing follow-up data, the subgroup analyses were performed only for posttreatment and the results from the follow-ups for the primary outcome stress were clustered into two categories (1-3 months and 4-6 months).

    In three studies, two treatments were compared with a single control group [36,38,42]. In these cases, we treated each comparison as a separate study, and we avoided the double counting of controls by dividing the control group in half.

    The effect size in the form of Cohen d was used to represent the standard mean difference between the means of the intervention group and the control group at posttest. According to Cohen [43], d=0.2 can be considered a small effect, d=0.5 is a medium effect, and d=0.8 is a large effect. Because of the difficulty of interpreting Cohen d from a clinical perspective, we also transformed these values into numbers needed to treat (NNT) according to the formula of Kraemer and Kupfer [44]. The NNT indicates the number of participants who need to be treated to generate one additional clinically significant change [45].

    Furthermore, we conducted the following subgroup analyses: the theoretical basis of the intervention (CBT, identified by including cognitive restructuring/challenging dysfunctional thoughts; TWC, identified by inclusion of more recent CBT-based techniques such as mindfulness, meditation, or acceptance of emotions; and alternative interventions [ALT]), guidance (guided with regular written feedback; unguided with no support or only technical support), length of the intervention (short: 1-4 weeks; medium: 5-8 weeks; long: ≥9 weeks), and the risk of bias (low risk=4; some risk<4).

    Moreover, a test of homogeneity of the observed effect sizes was calculated using the I2 statistic as an indicator of heterogeneity in percentages. Thereby, a value of 0% indicates no heterogeneity, 25% is considered low, 50% is considered moderate, and 75% is considered a high level of heterogeneity [46]. We calculated 95% confidence intervals around I2 [47] using the noncentral approach based on chi-square within the heterogi module for Stata [48]. Although we calculated the Q-statistics, we only report whether the result was significant.

    Publication bias was investigated by conducting a visual inspection of the funnel plot for the primary outcome measure. An asymmetric as opposed to a symmetric inverted funnel shape indicates potential publication bias that could compromise the conclusions drawn from the meta-analysis. Egger’s test [49] was used to test for the significance of the likely presence of publication bias. Additionally, we performed Duval and Tweedie’s trim-and-fill analysis [50] to verify an unbiased estimate of the pooled effect size. This analysis calculates an estimation of the number of missing studies and the potential effect of these studies on the outcome.


    Study Selection

    The systematic literature research resulted in 2137 abstracts. An additional nine potentially relevant articles were identified through other searches. After removing the duplicates, we screened the titles and abstracts of 1781 articles and excluded 1687 articles because of their apparent irrelevance. With regard to eligibility, 94 full-text articles were retrieved and assessed by two independent raters (EH and DL); Cohen kappa for agreement between the independent raters was very good (Cohen κ=.83). Any discrepancies were resolved by discussion. We included 27 studies according to the inclusion and exclusion criteria. However, the results of one study [51] were accounted for in one of the other included articles [24] and it was not possible to calculate effect sizes for three studies due to insufficient data [52-54]. In three studies [42,55,56], a small proportion of the participants were younger than 18 years, which was the prespecified criterion of being classified as adults. Because the studies fit all the other inclusion criteria and our sensitivity analyses indicated no difference in the overall results, we decided to include those studies.

    Thus, we included 23 studies in the analysis. These 23 studies included 26 comparisons from baseline to posttest. Follow-up data were available for four studies (six comparisons) at 1 to 3 months and for six studies at 4 to 6 months. The PRISMA flowchart of the study selection is presented in Figure 1.

    ‎
    Figure 1. PRISMA flowchart.
    View this figure

    Study Characteristics

    Selected characteristics of the 23 included studies [21-39,42,55-57] are presented in Table 1 (trial characteristics) and Table 2 (intervention characteristics). A more detailed description of the interventions is available in the Multimedia Appendix 2.

    The total number of participants was 4226 (intervention groups: n=2312, control groups: n=1914). The included interventions varied according to the intervention content (see Table 2 for details). Most studies evaluated interventions based on TWC (13 comparisons), followed by ALT interventions (7 comparisons), and CBT interventions (6 comparisons). Seventeen studies used a waitlist control comparison, three studies a no-treatment control group, and three studies used an attention control group. The included studies were predominantly conducted in the United States (11 studies), followed by Germany (3 studies), Sweden (2 studies), Austria/Switzerland (2 studies), Japan (2 studies), Norway (1 study), the United Kingdom (1 study), and the Netherlands (1 study).

    Table 1. Selected trial characteristics of the included studies (N=23).
    StudyOriginaNbAge (years), mean/mean rangeWomen (%)ConditionscOutcomesdRisk of biaseFollow-upf
    Aikens et al (2014) [27]US89N/AN/ATWC vs WCStress (PSS-14)+? ± ??6 m ext
    Allexandre et al (2016) [28]US5540.083.2TWC vs WCStress (PSS-10)+? ± +?4 m; 1 y ext
    Billings et al (2008) [21]US23630-3970.6CBT vs WCStress (SODS); depression (CES-D); anxiety (BAI)+? ± – ?N/A
    Cavanagh et al (2013) [29]GB10424.788.5TWC vs WCStress (PSS-10); depression (PHQ-4); anxiety (PHQ-4)+? ± +?N/A
    Drozd et al (2013) [30]NO25932.676.0TWC vs WCStress (DASS-S)+ + ± +?2 m; 6 m
    Ebert et al (2016) [31]DE26342.071.5TWC vs WCStress (PSS-10); depression (CES-D); anxiety (HADS-A)+? ± + +6 m
    Ebert et al (2016) [32]DE26342.985.9TWC vs WCStress (PSS-10); depression (CES-D) anxiety (HADS-A)+? ± + +6 m
    Frazier et al (2015) [56]US19418-2175.0ALT vs ACStress (PSS-10, DASS-S); depression (DASS-D) anxiety (DASS-A)+? ± +?5 w
    Glück & Maercker (2011) [33]AT, CH4735.273.5TWC vs WCStress (PSQ)+ – ± +?3 m ext
    Hänggi (2006) [22]CH12525-4585.0CBT vs WCStress (LOS)+ + ± – ?1 m ext; 6 m ext
    Heber et al (2016) [34]DE26443.373.1TWC vs WCStress (PSS-10); depression (CES-D); anxiety (HADS-A)+? ± + +6 m; 1 y ext
    Hinman et al (1997) [39]US5037.7100.0ALT vs NTStress (PNQ (combined)+? ± ? ?N/A
    Hintz et al (2015) [42]US20418-2170.0ALT vs AC; ALT vs AC (II)Stress (PSS-10, DASS-S); depression (DASS-D); anxiety (DASS-A)+? ± – ?5 w
    Ly et al (2014) [35]SE7341.542.5TWC vs WCStress (PSS-14)+ + ± + ?N/A
    Morledge/ Allexandre et al (2013) [36]US27940-5988.9TWC vs TWC+OMB vs WCStress (PSS-10)+ + ± – ?1 m
    Nguyen-Feng et al (2015) [55]US50018-2162.0ALT vs WCStress (PSS-10, DASS-S); depression (DASS-D); anxiety (DASS-A)+ ? ± + ?N/A
    Rose et al (2013) [23]US5927.350.0CBT vs ACStress (PSS-10)+ ? ± – ?N/A
    Ruwaard et al (2007) [24]NL23944.060.0CBT vs WCStress (DASS-S); depression (DASS-D); anxiety (DASS-A)+ ? ± + ?3 y ext
    Umanodan et al (2014) [25]JP26338.97.2CBT vs WCStress (BJSQ)+ ? ± + ?19 w
    Wiegand et al (2010) [38]US56235.8100.0ALT vs TWC vs NTStress (PSS-10); anxiety (STAI)+ ? ± – ?N/A
    Wolever et al (2012) [37]US10542.976.6TWC vs NTStress (PSS-10); depression (CES-D)+ + ± + ?N/A
    Yamagishi et al (2008) [57]JP3633.0N/AALT vs WCStress (JSBQ combined); depression (BJSQ-D); anxiety (BJSQ-A)+ ? ± – ?1 m ext
    Zetterqvist et al (2003) [26]

    		SE6339.261.9CBT vs WCStress (PSS-14); depression (HADS-D); anxiety (HADS-A)+ ? ± – ?N/A

    aAT: Austria; CH: Switzerland; DE: Germany; GB: United Kingdom; JP: Japan; NL: the Netherlands; NO: Norway; SE: Sweden; US: United States of America.

    bIndicates only the number of participants included in this analysis.

    cAC: attention control group; ALT: alternative; CBT: cognitive behavioral therapy; NT: no treatment; OMB: online message board; TWC: third-wave cognitive behavioral therapy; WC: waitlist control.

    dBAI: Beck Anxiety Inventory; BJSQ-A: Brief Job Stress Questionnaire-Anxiety Subscale; BJSQ-D: Brief Job Stress Questionnaire-Depression Subscale; CES-D: Center for Epidemiologic Studies Depression Scale; DASS-A: Depression Anxiety Stress Scales-Anxiety Subscale; DASS-D: Depression Anxiety Stress Scales-Depression Subscale; DASS-S: Depression Anxiety Stress Scales-Stress Subscale; HADS-A: Hospital Anxiety and Depression Scales-Anxiety Subscale; HADS-D: Hospital Anxiety and Depression Scales-Depression Subscale; JSBQ: Job Stress Brief Questionnaire; LOS: Level of Stress (self-created instrument); PHQ-4: Patient Health Questionnaire for Depression and Anxiety; PNQ: Personal Strain Questionnaire; PSQ: Perceived Stress Questionnaire; PSS-10: Cohen’s Perceived Stress Questionnaire (10-item version); PSS-14: Cohen’s Perceived Stress Questionnaire (14-item version); SODS: Symptoms of Distress Scale; STAI: State Trait Anxiety Inventory.

    eRisk of bias was judged according to the following criteria: (1) adequate sequence generation, (2) allocation concealment, (3) blinding (± indicates that only self-reported data were used), (4) adequate consideration of incomplete data, and (5) prevention of selective outcome. +: no bias; –: bias; ?: information was insufficient to make judgments.

    fExt: extended follow-up; m: months; w: weeks; y: years.

    Table 2. Selected intervention characteristics of included studies.
    StudyLabel and contentTypeaGuidancebDeliveryLength (weeks)
    Aikens et al (2014) [27]Mindfulness goes to work: Mindfulness program combined live, instructor-led, weekly hour-long virtual meetings (webinar) with online applied training.TWCGWeb7
    Allexandre et al (2016) [28]Stress Free Now / Online Mindfulness Program for Stress Management: Interactive, educational program based on mindfulness meditation. Includes exercises, email reminders, and downloads.TWCUG (R)Web8
    Billings et al (2008) [21]Stress and Mood Management Intervention: Array of CBT techniques. Entire program is audio-narrated with the use of videos and graphics.CBTUGWeb12
    Cavanagh et al (2013) [29]Learning Mindfulness Online: Daily, 10-min guided mindfulness meditation audio tracks delivered via a virtual learning facility (Moodle). Four reminder emails.TWCUG (R)Web2
    Drozd et al (2013) [30]Less Stress intervention: Eclectic approach that included mindfulness and metacognitive exercises with 13 short modules. Hyperlinks sent via email to the participants.TWCUGWeb4
    Ebert et al (2016) [31]GET.ON Stress Self-Guided: Theory-based intervention focusing on problem solving and emotion regulation. Tailored to employees; optional text message coaching.TWCUGWeb7
    Ebert et al (2016) [32]GET.ON Stress Adherence-Focused Guided: Theory-based intervention focusing on problem solving and emotion regulation. Tailored to employees; optional text message coaching; written feedback on request; reminders.TWCAFGWeb7
    Frazier et al (2015) [56]Present Control Intervention: Theory-based intervention focused on perceived control; 4 modules over 2-week period. Included stress logs and reminder emails.ALTUG (R)Web2
    Glück & Maercker (2011) [33]Brief Web-based mindfulness training: 2 modules. 20-minute units per day, audio files, a flash animated exercise, and written text.TWCUG (R)Web2
    Hänggi (2006) [22]Online parental training on coping with family stress: 4 modules (eg, cognitive restructuring, time management, muscle and breathing relaxation, problem solving).CBTUGWeb4
    Heber et al (2016) [34]GET.ON Stress Guided: Theory-based intervention focusing on problem solving and emotion regulation. Tailored to employees; optional text message coaching; written feedback; reminders.TWCGWeb7
    Hinman et al (1997) [39]Exercise Break: 2 × 15 minutes per day. Stretching, circulatory and relaxation exercises. Accessed via local computer network at the workplace.ALTUGPC8
    Hintz et al (2015) [42]Present Control Intervention: Theory-based intervention focused on perceived control; 4 modules over 2-week period. Included stress logs and reminder emails. Group I: with personalized feedback, Group II: unguided.ALTUG (R) &
    G    		Web2
    Ly et al (2014) [35]Acceptance- and commitment-based mobile phone app: step-by-step behavior program including 6 basic principles to handle stress. 15 min daily. Short writing reflection. One-way therapist-client support through text messages every other day.TWCGWeb (smartphone)6
    Morledge/Allexandre et al (2013) [36]Online Mindfulness Program for Stress Management: Eight mindfulness modules consisting of introduction, meditations, articles, and tips and exercises. Group II: program plus online message board.TWCUG (R)Web8
    Nguyen-Feng et al (2015) [55]Present Control Intervention: Theory-based intervention focused on perceived control; 3 modules: psycho-educational video of a professor, animated video (Prezi), and a written exercise. Includes stress logs and reminders.ALTUG (R)Web5
    Rose et al (2013) [23]Self-guided, multimedia stress management and resilience training program, SMART-OP: consists of at least one exercise in each of 3 domains: feelings, thoughts, and actions. Includes game-like activities.CBTUG (R)PC6
    Ruwaard et al (2007) [24]Emailed Standardized CBT of Work-Related Stress: 7 modules (eg, relaxation, challenging dysfunctional thoughts, time management). 10 feedbacks/5 hours of therapist time.CBTGWeb7
    Umanodan et al (2014) [25]SMT program in employees: Self-paced program. (1) behavioral techniques, (2) communication techniques, and (3) cognitive techniques; skill acquisition and practice phase; weekly emails.CBTUG (R)PC7
    Wiegand et al (2010) [38]Comprehensive program for reducing stress: Group I: Daily use of olfactory care products plus an Internet-based program focusing on stress reduction, prevention and behavioral modification. Periodic feedback reports are provided. Group II: Internet-based program only.ALT, TWCUGWeb12
    Wolever et al (2012) [37]Mindfulness at Work Intervention: Virtual classroom with real-time bidirectional communication with an experienced mindfulness trainer (12 modules, 14 hours). Includes brief exercises designed to be used at work.TWCGWeb12
    Yamagishi et al (2008) [57]Web-based career identity training for stress management: 4 modules. (1) definition of career identity, (2): cognition of own career identity, (3): characteristics of nurses’ career identity, (4): career goal management and planning.ALTUGWeb3
    Zetterqvist et al (2003) [26]Internet-based self-help stress-management program: Each module consists of 3 sections: relaxation, additional exercises (eg, problem solving), and information (eg stress at work). Exercises were sent in and participants received feedback as a prompt to continue; includes reminders.CBTGWeb6

    aALT: alternative; CBT: cognitive behavioral therapy; TWC: third-wave cognitive behavioral therapy.

    bAFG: Adherence-focused guidance; G: guided; UG: unguided; UG (R): unguided with reminders via mail or telephone.

    The interventions of the included studies were mainly Web-based interventions (n=20). For Web-based interventions, an active Internet connection is necessary (eg, to access a website, use a mobile phone app, or visit a virtual classroom). A total of three computer-based interventions [23,25,39] were found in which interventions were installed, for example, on a desktop computer in a separate room at work.

    The Perceived Stress Scale (PSS-10, PSS-14) was predominantly used to assess the level of stress (15 of 23 studies). Follow-up assessments have been reported for nine studies (12 comparisons); whereas four studies (6 comparisons) reported data for up to 3 months and six studies (6 comparisons) for 4 to 6 months (see Table 1). Extended follow-ups (no comparison with the respective control groups) were conducted at 1 month (2 studies), 3 months (1 study), 6 months (2 studies), one year (2 studies), and 3 years (1 study). For nine studies, only posttest assessments were available. The participants received guidance in seven studies [24,26,27,34,35,37,42], one study [32] assessed a less intensive guidance format (adherence-focused guidance: combination of reminders and written feedback only on request of the participants), and 16 studies (18 comparisons) investigated unguided interventions. Nine unguided studies reported the use of automated or telephone and mail reminders for completion of the intervention [23,25,28,29,33,36,42,55,56] (Table 2).

    Risk of Bias Within Studies

    A risk of bias was present in all studies. Only six studies fulfilled four of the five criteria used. Nine studies met three criteria, and eight studies fulfilled two criteria. In most cases, the concealment of allocation was insufficiently described, and only 13 of 23 studies reported adequate handling of missing data. In particular, the risk for selective outcome reporting was unclear because the study registration prior to the trials could only be retrieved from three studies [31,32,34]. Although another four studies registered their trial [28,30,33,36], this step occurred retrospectively. The mean interrater reliability between independent raters was κ=.84 and ranged from .60 (selective outcome reporting) to 1.0 (blinding). Disagreements were handled by discussion.

    Publication Bias

    Neither the inspection of the funnel plot nor the Egger’s test [49] indicated a possible publication bias. Moreover, the Duval and Tweedie trim-and-fill analysis [50] indicated no missing studies.

    Effects on Levels of Stress, Depression, and Anxiety

    Table 3 presents the effect sizes, confidence intervals, level of significance and heterogeneity for the overall effects on stress, depression, and anxiety as well as for the subgroups. The overall analysis of effect sizes yielded a significant effect size of Cohen d=0.43 for the primary outcome stress across all studies (95% CI 0.31-0.54; n=26). Heterogeneity was moderate (I2=68.01, 95% CI 52.08-78.72). Significant small effect sizes were observed for the secondary outcomes depression (Cohen d=0.34, 95% CI 0.21-0.48; n=13) and anxiety (Cohen d=0.32, 95% CI 0.17-0.47; n=14). Figure 2 displays a forest plot of the effect sizes and the confidence intervals of the included studies.

    Table 3. Effects of computer- and Web-based stress-management interventions for healthy adults compared to control groups.
    StudyComparisons, nCohen d (95% CI)PZI2 (95% CI)PaNNTPb
    Overall effect








    		Primary outcome









    		Stress at posttest260.43 (0.31, 0.54)<.0017.1268.01 (52.08, 78.72)<.0014.20


    		1-3 m follow-up60.33 (0.19, 0.46)<.0014.600.00 (0.00, 74.62).555.43


    		4-6 m follow-up60.56 (0.25, 0.87)<.0013.5585.93 (71.44, 93.07)<.0013.25

    		Further outcomes (posttest)









    		Depressive symptoms130.34 (0.21, 0.48)<.0014.9358.25 (22.81, 77.41).0045.26


    		Anxiety symptoms140.32 (0.17, 0.47)<.0014.1671.13 (50.34, 83.22)<.0015.56
    Risk of bias scorec








    		Low risk60.74 (0.59, 0.89)<.0019.8235.44 (0.00, 74.20).172.50<.001

    		Some risk200.30 (0.21, 0.40)<.0016.2631.43 (0.00, 60.17).095.95
    Theoretical basisc








    		CBT60.40 (0.19, 0.61)<.0013.7552.68 (0.00, 81.12).064.50.03

    		TWC130.53 (0.35, 0.71)<.0015.6774.50 (55.98, 85.23)<.0013.42

    		ALT70.24 (0.12, 0.36)<.0014.030.00 (0.00,70.81).857.46
    Guidancec,d








    		Yes70.64 (0.50, 0.79)<.0018.5311.81 (0.00, 74.24).342.86.002

    		No180.33 (0.20,0.46)<.0015.0262.72 (38.03, 77.57)<.0015.43
    Lengthc








    		Short90.33 (0.22, 0.44)<.0015.940.00 (0.00, 64.80).565.43.006

    		Medium130.59 (0.45, 0.74)<.0017.8954.92 (15.83, 75.86).0083.09

    		Long40.21 (-0.05, 0.47).111.6171.00 (17.25, 89.85).028.47

    aThe P values indicate whether the Q-statistic is significant (the I2 statistics do not include a test of significance).

    bThis P value indicates whether the differences between subgroups were significant.

    cSubgroup calculations for the primary outcome stress.

    dOne study [32] was excluded because it could not be classified.

    ‎
    Figure 2. All effect sizes included in the meta-analysis from the studies comparing computer- and Web-based stress-management interventions to a control group. BJSQ: Brief Job Stress Questionnaire; DASS-S: Depression Anxiety Stress Scales-Stress Subscale; LOS: Level of Stress (self-created instrument); PSQ: Perceived Stress Questionnaire; PSS-10: Cohen’s Perceived Stress Questionnaire (10 item version); PSS-14: Cohen’s Perceived Stress Questionnaire (14 item version); SODS: Symptoms of Distress Scale.
    View this figure

    Subgroup Analyses

    Risk of Bias

    Accounting for the risk of bias level was associated with a considerable reduction of heterogeneity: studies with a lower risk of bias showed low heterogeneity (I2=35.44, 95% CI 0.00-74.20). Subgroup analyses revealed that studies at low risk produced significantly larger effect sizes (Cohen d=0.74, 95% CI 0.59-0.89; n=6) than did studies with some risk (Cohen d=0.30, 95% CI 0.21-0.40; n=20; P<.001).

    Theoretical Basis

    The subgroup analysis of the theoretical basis of the interventions was significant (P=.03) and showed that TWC interventions produced a highly significant medium effect size of Cohen d=0.53 (95% CI 0.35-0.71; n=13). CBT interventions led to reductions in stress levels with a highly significant effect size of Cohen d=0.40 (95% CI 0.19-0.61; n=6). In contrast, ALT interventions produced a small effect size (Cohen d=0.24; 95% CI 0.12-0.36; n=7).

    Guidance

    With regard to the subgroup guidance, the results show that guided interventions (Cohen d=0.64, 95% CI 0.50-0.79; n=7) were significantly more effective than unguided interventions (Cohen d=0.33, 95% CI 0.20-0.46; n=18; P=.002).

    Length

    A significant difference was also found for the length of the interventions (P=.006). We found significant small-to-medium effect sizes for short (Cohen d=0.33, 95% CI 0.22-0.44; n=9) and medium interventions (Cohen d=0.59, 95% CI 0.45-0.74; n=13), whereas long interventions led to a nonsignificant effect size of Cohen d=0.21 (95% CI –0.05 to 0.47; n=4; P=.11).

    Follow-up

    Results of the 1 to 3 month follow-ups for the primary outcome stress showed a small effect size of Cohen d=0.33 (95% CI 0.19-0.46; n=6) and, for the 4 to 6 month follow-ups, a medium effect size of Cohen d=0.56 (95% CI 0.25-0.87; n=6).


    The aim of this paper was to conduct a meta-analysis of Web- and computer-based interventions for stress management in adults. Our analyses yielded four main findings. First, Web- and computer-based stress-management interventions can be effective in reducing stress, depression, and anxiety and maintain those effects for up to 6 months. Second, interventions using TWC and CBT interventions as a theoretical basis proved moderately effective in reducing stress. Third, short and medium interventions (up to 8 weeks) are more effective than long interventions (9 weeks and longer). Fourth, guided interventions yielded a greater effect size for reduction of stress than unguided interventions.

    For the primary outcome stress, an effect size of Cohen d=0.43 was found across the 26 comparisons. Small effect sizes were found for depression (Cohen d=0.34) and anxiety (Cohen d=0.32). A recent synthesis of five meta-analyses of traditional stress-management interventions found a between-group overall mean effect size of Cohen d=0.45 (95% CI 0.41-0.48) [58], which is comparable to the effect of stress, but somewhat higher for the effects of depression and anxiety found in this meta-analysis. To date, no trials have been conducted that aim to assess the equivalence of face-to-face and Web-based stress-management interventions in a methodologically robust design. One trial comparing an online versus face-to-face version of stress management in a single trial indicates that there is no difference in reductions of stress or depression levels [37]. Nevertheless, higher effect sizes were also found in Richardson and Rothstein’s work [9] concerning individual outcome measures in traditional stress-management interventions, particularly for stress (Cohen d=0.73) and anxiety (Cohen d=0.68). Therefore, it may be possible that traditional interventions yield slightly higher effect sizes. One possible explanation for this is that face-to-face interventions are superior in reducing these outcomes. An alternative explanation is that Web-based interventions may reach affected individuals at an earlier stage, with lower baseline levels and thus less room for improvement. In fact, most studies included in the present work did not use a cut-off on a relevant stress scale, and most of those that did (eg, [23,37,38]) used a relatively low cut-off threshold. It may be the case that participants who are more severely impaired might generally prefer face-to-face over Web-based interventions and that this effect is reflected in greater improvements stemming from their higher baseline stress levels. This is in line with the fact that the highest effect sizes in this meta-analysis were produced from trials targeting highly stressed individuals [31,32,34]. Evidence on a Web-based depression intervention also showed that a higher severity of baseline scores significantly predicted better treatment outcomes [59]. Although the effect sizes found here are somewhat smaller than those found in traditional face-to-face interventions, Web- and computer-based interventions can have greater reach. At the population level, even small-to-moderate effects can have a substantial influence. More research is needed to clarify the effects of face-to-face and Web-based stress-management interventions in direct comparisons.

    Based on the assumption that the treatment effect varied as a function of other factors, we conducted a number of subgroup analyses. First, we investigated the effect of guidance. The results showed that guided interventions (Cohen d=0.64) are significantly more effective than unguided interventions (Cohen d=0.33), with effect sizes for guided interventions comparable to traditional face-to-face interventions. The finding of guided interventions being superior to unguided interventions is consistent with results on Web-based interventions for other conditions, such as depression and anxiety [60,61]. Providing support to clients in terms of weekly feedback may enhance adherence to the intervention and thus improve treatment efficacy [62]. Such an assumption is in line with a study on pooled data from three RCTs showing that guidance was associated with greater adherence rates in Internet-based stress-management compared to unguided interventions [63]. The finding that guided interventions are more effective than unguided interventions is consistent with a systematic review that found guided Web-based mental health interventions to be significantly superior to unguided interventions [61]. The results in this analysis indicate an advantage for guided interventions, although it is unclear how much guidance and in what manner it produces the largest effect sizes. One trial in this meta-analysis used an alternative, more economic format of guidance (adherence-focused guidance; ie, feedback only on request plus weekly reminders) and produced large effect sizes at posttest and 6-month follow-up [32], and the adherence rates were comparable to a more intensive guidance format using the same intervention [63]. More research on the relative level and type of human involvement needed in these interventions would be useful [64].

    Consistent with existing evidence on face-to-face interventions [8,9], the CBT interventions included in this study were efficacious. The effect sizes of CBT (Cohen d=0.40) and TWC interventions (Cohen d=0.53) were smaller than the average effect size found in two meta-analyses for traditional CBT interventions (Cohen d=0.68 [8]; Cohen d=1.16 [9]), although, on average, the effect sizes fall within the confidence interval of the latter work [9]. Direct comparisons examining the relative efficacy of the two training formats would be needed to draw firm conclusions. Due to their proliferation in the last few years, TWC interventions were introduced as a new category in this meta-analysis as they extend the traditional CBT techniques with newer “third-wave” components, such as acceptance of emotions or mindfulness. These interventions have been found to be effective in alleviating symptoms that are associated with a wide range of physical, psychosomatic, and psychiatric disorders [65,66], including stress [67]. Comparable to our findings, early evidence on face-to-face interventions suggests that TWC and traditional CBT approaches are equally effective and acceptable in the treatment of acute depression [68]; nevertheless, more high-quality studies are needed to support this assumption. As opposed to (third-wave) CBT interventions, alternative approaches (eg, career identity training, combination with olfactory care products) only produced a small effect size.

    A significant between-group effect was also found for the length of the intervention. In contrast to short-to-medium interventions, long interventions (9 weeks and longer) were not found to be effective. One possible explanation is that it may be more difficult for participants to remain engaged in longer interventions compared with shorter interventions. These results correspond to findings from the area of depression, in which shorter interventions have been found to be more effective than longer interventions [61]. Research on the relevance of treatment intensity suggests that the number of therapy sessions is not related to the outcome and keeping the number of sessions equal, but providing the sessions over a shorter period of time, may be associated with better treatment outcomes [69]. Nevertheless, because there were only four comparisons available for long interventions, this conclusion should be interpreted with care. Future research should examine the optimal intensity and length of interventions.

    Limitations

    This meta-analysis has a number of limitations. First, because the risk of bias in the included studies was high, these results must be interpreted with caution. Second, the overall number of studies for the follow-up points and for some subgroups is small, limiting the strength of conclusions that can be drawn from these results. Third, we found a moderate degree of heterogeneity for the primary outcome that was reduced when analyzing the level of risk of bias and guidance subgroups; nevertheless, the number of comparisons in some subgroups was small and did occasionally overlap concerning individual studies. Fourth, we are mindful of the possibility that despite our thorough literature search, we might have missed a relevant study. Finally, because of the lack and inconsistency of information provided by the included studies, we were unable to analyze the effect of potentially relevant moderators of the treatment effect, such as the effect of adherence to the intervention on the overall outcome.

    Future Recommendations

    In future studies, preregistration of trials, an adequate calculation of the sample size, a more detailed description of allocation concealment, and appropriate methods to account for missing values are strongly recommended. We observed that a growing number of studies adhered to the intention-to-treat principle (ie, by employing mixed-effects models); nevertheless, especially for those studies, we recommend to ensure that the reported descriptive statistics are based on adequate methods to handle missing values and do not solely rely on complete cases. This will ensure adherence to the intention-to-treat principle throughout all statistics.

    Although in most studies the therapeutic approach that was used in the intervention (eg, CBT) is often well described, it would be desirable for future studies to also specify the theoretical model that was used to develop the intervention (eg, conservation of resources theory, transactional model of stress).

    Overall, more methodologically rigorous studies with a low risk of bias are needed to assess the effect of, for example, particular characteristics of interventions, such as treatment latitude, different levels of guidance, and different types of interventions in a direct comparison. Moreover, information on the number of participants who adhered to the intervention and details on co-interventions alongside training would be insightful. Future research should also test Web- and computer-based stress management interventions against the highest standard in this field (ie, classical face-to-face stress management interventions) and should more frequently include longer follow-up periods (eg, up to 6 months). Data on the cost-effectiveness of such interventions would also be highly relevant.

    Conclusions

    Despite the limitations discussed, it appears safe to conclude that Web- and computer-based interventions can be effective. In particular, interventions that include guidance from an online coach, are of medium length, or that are based on (third-wave) CBT lead to moderate improvements in stress levels. Initial evidence also suggests that the effects can be maintained up to 6 months. Whereas research and practice on traditional face-to-face interventions have been prolific [8,9], research on the efficacy and dissemination of Web-based stress-management interventions is still at the beginning despite the high potential and reach of such interventions. This work draws attention to the need for further studies on the efficacy, cost-effectiveness, and mechanisms of change of such interventions. In summary, the integration of evidence-based Web-based stress management interventions into health care systems has the potential to make a valuable contribution to reducing stress-related mental health problems on a large scale.

    Acknowledgments

    We would like to acknowledge the European Union (EFRE) for funding this project within the Lueneburg Innovation Incubator, TM 1.1 (project number: CCI 2007DE161PR001). Furthermore, we would like to thank Stefanie Christmann for assisting in the coding of the studies. We would also like to thank Douglas W Billings, Filip Drozd, Katherine Eisen, Yves Hänggi, Jeroen Ruwaard, Benjamin Wiegand, Viann Nguyen-Feng, and Ruth Q Wolever for providing further information on their studies.

    Authors' Contributions

    EH, DE, DL, MB, PC, and HR conceptualized the study. EH, DE, DL, and PC developed the search string. EH, SN, and DE performed the search, selected the studies, and collected the data. EH, DE, and DL analyzed and interpreted the data. EH drafted the manuscript; DE supervised the writing process; and DL, PC, MB, SN, and HR critically revised it. All authors agreed on the final version of the paper.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Search strategy.

    PDF File (Adobe PDF File), 10KB
    ‎

    Multimedia Appendix 2

    Selected intervention characteristics of included studies.

    PDF File (Adobe PDF File), 119KB
    1. Hammen C. Stress and depression. Annu Rev Clin Psychol 2005 Apr;1:293-319. [CrossRef] [Medline]
    2. Akerstedt T. Psychosocial stress and impaired sleep. Scand J Work Environ Health 2006 Dec;32(6):493-501 [FREE Full text] [Medline]
    3. Kraatz S, Lang J, Kraus T, Münster E, Ochsmann E. The incremental effect of psychosocial workplace factors on the development of neck and shoulder disorders: a systematic review of longitudinal studies. Int Arch Occup Environ Health 2013 May;86(4):375-395. [CrossRef] [Medline]
    4. Richardson S, Shaffer JA, Falzon L, Krupka D, Davidson KW, Edmondson D. Meta-analysis of perceived stress and its association with incident coronary heart disease. Am J Cardiol 2012 Dec 15;110(12):1711-1716 [FREE Full text] [CrossRef] [Medline]
    5. Steptoe A, Kivimäki M. Stress and cardiovascular disease. Nat Rev Cardiol 2012 Jun;9(6):360-370. [CrossRef] [Medline]
    6. Hassard J, Teoh K, Cox T, Dewe P, Cosmar M, Grundler R, et al. Calculating the Cost of Work-Related Stress and Psychosocial Risks. Luxembourg: Publications Office of the European Union; 2014.
    7. Bhui KS, Dinos S, Stansfeld SA, White PD. A synthesis of the evidence for managing stress at work: a review of the reviews reporting on anxiety, depression, and absenteeism. J Environ Public Health 2012;2012:515874 [FREE Full text] [CrossRef] [Medline]
    8. van der Klink JJ, Blonk RW, Schene AH, van Dijk FJ. The benefits of interventions for work-related stress. Am J Public Health 2001 Feb;91(2):270-276. [Medline]
    9. Richardson KM, Rothstein HR. Effects of occupational stress management intervention programs: a meta-analysis. J Occup Health Psychol 2008 Jan;13(1):69-93. [CrossRef] [Medline]
    10. Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother 2012 Jul;12(7):861-869; quiz 870. [CrossRef] [Medline]
    11. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res 2012 Dec;12(6):745-764. [CrossRef] [Medline]
    12. Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, et al. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One 2014;9(6):e99912 [FREE Full text] [CrossRef] [Medline]
    13. Lin J, Ebert DD, Lehr D, Berking M, Baumeister H. [Internet based cognitive behavioral interventions: state of the art and implementation possibilities in rehabilitation]. Rehabilitation (Stuttg) 2013 Jun;52(3):155-163. [CrossRef] [Medline]
    14. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One 2015;10(3):e0119895 [FREE Full text] [CrossRef] [Medline]
    15. van Zoonen K, Buntrock C, Ebert DD, Smit F, Reynolds CFIII, Beekman AT, et al. Preventing the onset of major depressive disorder: a meta-analytic review of psychological interventions. Int J Epidemiol 2014 Apr;43(2):318-329 [FREE Full text] [CrossRef] [Medline]
    16. Buntrock C, Ebert DD, Lehr D, Cuijpers P, Riper H, Smit F, et al. Evaluating the efficacy and cost-effectiveness of web-based indicated prevention of major depression: design of a randomised controlled trial. BMC Psychiatry 2014;14:25 [FREE Full text] [CrossRef] [Medline]
    17. Buntrock C, Ebert DD, Lehr D, Smit F, Riper H, Berking M, et al. Effect of a Web-based guided self-help intervention for prevention of major depression in adults with subthreshold depression: a randomized clinical trial. JAMA 2016 May 3;315(17):1854-1863. [CrossRef] [Medline]
    18. Ebert DD, Buntrock C, Cuijpers P. Online intervention for prevention of major depression--reply. JAMA 2016;316(8):881-882. [CrossRef] [Medline]
    19. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther 2009;38(2):66-82. [CrossRef] [Medline]
    20. Cheng SK, Dizon J. Computerised cognitive behavioural therapy for insomnia: a systematic review and meta-analysis. Psychother Psychosom 2012;81(4):206-216. [CrossRef] [Medline]
    21. Billings DW, Cook RF, Hendrickson A, Dove DC. A web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med 2008 Aug;50(8):960-968. [CrossRef] [Medline]
    22. Hänggi Y. Kann web-basierte Stressprävention erfolgreich sein? [Can web-based parental training on coping with familiy stress be successful?]. Zeitschrift für Klinische Psychologie und Psychotherapie 2006 Jul 1;35(3):169-177. [CrossRef]
    23. Rose RD, Buckey JC, Zbozinek TD, Motivala SJ, Glenn DE, Cartreine JA, et al. A randomized controlled trial of a self-guided, multimedia, stress management and resilience training program. Behav Res Ther 2013 Feb;51(2):106-112. [CrossRef] [Medline]
    24. Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B. E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther 2007;36(3):179-192. [CrossRef] [Medline]
    25. Umanodan R, Shimazu A, Minami M, Kawakami N. Effects of computer-based stress management training on psychological well-being and work performance in japanese employees: a cluster randomized controlled trial. Ind Health 2014;52(6):480-491 [FREE Full text] [Medline]
    26. Zetterqvist K, Maanmies J, Ström L, Andersson G. Randomized controlled trial of internet-based stress management. Cogn Behav Ther 2003;32(3):151-160. [CrossRef] [Medline]
    27. Aikens KA, Astin J, Pelletier KR, Levanovich K, Baase CM, Park YY, et al. Mindfulness goes to work: impact of an online workplace intervention. J Occup Environ Med 2014 Jul;56(7):721-731. [CrossRef] [Medline]
    28. Allexandre D, Bernstein AM, Walker E, Hunter J, Roizen MF, Morledge TJ. A Web-based mindfulness stress management program in a corporate call center: a randomized clinical trial to evaluate the added benefit of onsite group support. J Occup Environ Med 2016 Mar;58(3):254-264 [FREE Full text] [CrossRef] [Medline]
    29. Cavanagh K, Strauss C, Cicconi F, Griffiths N, Wyper A, Jones F. A randomised controlled trial of a brief online mindfulness-based intervention. Behav Res Ther 2013 Sep;51(9):573-578. [CrossRef] [Medline]
    30. Drozd F, Raeder S, Kraft P, Bjørkli CA. Multilevel growth curve analyses of treatment effects of a Web-based intervention for stress reduction: randomized controlled trial. J Med Internet Res 2013;15(4):e84 [FREE Full text] [CrossRef] [Medline]
    31. Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, et al. Self-guided internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med 2016 May;73(5):315-323. [CrossRef] [Medline]
    32. Ebert DD, Lehr D, Heber E, Riper H, Cuijpers P, Berking M. Internet- and mobile-based stress management for employees with adherence-focused guidance: efficacy and mechanism of change. Scand J Work Environ Health 2016 Sep 01;42(5):382-394. [CrossRef] [Medline]
    33. Glück TM, Maercker A. A randomized controlled pilot study of a brief web-based mindfulness training. BMC Psychiatry 2011;11:175 [FREE Full text] [CrossRef] [Medline]
    34. Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res 2016;18(1):e21 [FREE Full text] [CrossRef] [Medline]
    35. Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: A randomized controlled trial. Internet Interventions 2014 Jul;1(3):95-101. [CrossRef]
    36. Morledge TJ, Allexandre D, Fox E, Fu AZ, Higashi MK, Kruzikas DT, et al. Feasibility of an online mindfulness program for stress management-a randomized, controlled trial. Ann Behav Med 2013 Oct;46(2):137-148 [FREE Full text] [CrossRef] [Medline]
    37. Wolever RQ, Bobinet KJ, McCabe K, Mackenzie ER, Fekete E, Kusnick CA, et al. Effective and viable mind-body stress reduction in the workplace: a randomized controlled trial. J Occup Health Psychol 2012 Apr;17(2):246-258. [CrossRef] [Medline]
    38. Wiegand B, Luedtke K, Friscia D, Nair M, Aleles M, McCloskey R. Efficacy of a comprehensive program for reducing stress in women: a prospective, randomized trial. Curr Med Res Opin 2010 Apr;26(4):991-1002. [CrossRef] [Medline]
    39. Hinman M, Ezzo L, Hunt D, Mays A. Computerized exercise program does not affect stress levels of asymptomatic VDT users. J Occup Rehab 1997 Mar;7(1):45-51. [CrossRef]
    40. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928 [FREE Full text] [Medline]
    41. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons; 2009.
    42. Hintz S, Frazier PA, Meredith L. Evaluating an online stress management intervention for college students. J Couns Psychol 2015 Apr;62(2):137-147. [CrossRef] [Medline]
    43. Cohen J. A power primer. Psychol Bull 1992 Jul;112(1):155-159. [Medline]
    44. Kraemer HC, Kupfer DJ. Size of treatment effects and their importance to clinical research and practice. Biol Psychiatry 2006 Jun 1;59(11):990-996. [CrossRef] [Medline]
    45. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988 Jun 30;318(26):1728-1733. [CrossRef] [Medline]
    46. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003 Sep 6;327(7414):557-560 [FREE Full text] [CrossRef] [Medline]
    47. Ioannidis JP, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ 2007 Nov 3;335(7626):914-916 [FREE Full text] [CrossRef] [Medline]
    48. Orsini N, Higgins J, Bottai M, Buchan I. EconPapers. 2005 Feb 16. Heterogi: Stata module to quantify heterogeneity in a meta-analysis   URL: http://econpapers.repec.org/RePEc:boc:bocode:s449201 [accessed 2017-01-25] [WebCite Cache]
    49. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997 Sep 13;315(7109):629-634 [FREE Full text] [Medline]
    50. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000 Jun;56(2):455-463. [Medline]
    51. Lange A, van de Ven J, Schrieken B, Smit M. ‘Interapy’ Burn-out: Prävention und Behandlung von Burn-out über das Internet [‘Interapy’ burnout: prevention and therapy of burnout via the Internet]. Verhaltenstherapie 2004 Oct 4;14(3):190-199. [CrossRef]
    52. Chiauzzi E, Brevard J, Thum C, Thurn C, Decembrele S, Lord S. MyStudentBody-Stress: an online stress management intervention for college students. J Health Commun 2008 Sep;13(6):555-572. [CrossRef] [Medline]
    53. Eisen KP, Allen GJ, Bollash M, Pescatello LS. Stress management in the workplace: a comparison of a computer-based and an in-person stress-management intervention. Comput Hum Behav 2008;24:486-496. [CrossRef]
    54. Hasson H, Anderberg UM, Theorell T, Arnetz BB. Psychophysiological effects of a web-based stress management system: a prospective, randomized controlled intervention study of IT and media workers [ISRCTN54254861]. BMC Public Health 2005 Jul 25;5:78 [FREE Full text] [CrossRef] [Medline]
    55. Nguyen-Feng VN, Frazier PA, Greer CS, Howard KG, Paulsen JA, Meredith L, et al. A randomized controlled trial of a web-based intervention to reduce distress among students with a history of interpersonal violence. Psychol Violen 2015;5(4):444-454. [CrossRef]
    56. Frazier P, Meredith L, Greer C, Paulsen JA, Howard K, Dietz LR, et al. Randomized controlled trial evaluating the effectiveness of a web-based stress management program among community college students. Anxiety Stress Copin 2015;28(5):576-586. [CrossRef] [Medline]
    57. Yamagishi M, Kobayashi T, Nakamura Y. Effects of web-based career identity training for stress management among Japanese nurses: a randomized control trial. J Occup Health 2008;50(2):191-193 [FREE Full text] [Medline]
    58. Johnson BT, Scott-Sheldon LA, Carey MP. Meta-synthesis of health behavior change meta-analyses. Am J Public Health 2010 Nov;100(11):2193-2198 [FREE Full text] [CrossRef] [Medline]
    59. Junge MN, Lehr D, Bockting CL, Berking M, Riper H, Cuijpers P, et al. For whom are internet-based occupational mental health interventions effective? Moderators of internet-based problem-solving training outcome. Internet Interventions 2015 Mar;2(1):39-47. [CrossRef]
    60. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
    61. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    62. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    63. Zarski AC, Lehr D, Berking M, Riper H, Cuijpers P, Ebert DD. Adherence to Internet-based mobile-supported stress management: a pooled analysis of individual participant data from three randomized controlled trials. J Med Internet Res 2016;18(6):e146 [FREE Full text] [CrossRef] [Medline]
    64. Ebert DD, Lehr D, Smit F, Zarski AC, Riper H, Heber E, et al. Efficacy and cost-effectiveness of minimal guided and unguided internet-based mobile supported stress-management in employees with occupational stress: a three-armed randomised controlled trial. BMC Public Health 2014;14:807 [FREE Full text] [CrossRef] [Medline]
    65. Grossman P, Niemann L, Schmidt S, Walach H. Mindfulness-based stress reduction and health benefits. A meta-analysis. J Psychosom Res 2004 Jul;57(1):35-43. [CrossRef] [Medline]
    66. Vøllestad J, Nielsen MB, Nielsen GH. Mindfulness- and acceptance-based interventions for anxiety disorders: a systematic review and meta-analysis. Br J Clin Psychol 2012 Sep;51(3):239-260. [CrossRef] [Medline]
    67. Chiesa A, Serretti A. Mindfulness-based stress reduction for stress management in healthy people: a review and meta-analysis. J Altern Complement Med 2009 May;15(5):593-600. [CrossRef] [Medline]
    68. Hunot V, Moore TH, Caldwell DM, Furukawa TA, Davies P, Jones H, et al. 'Third wave' cognitive and behavioural therapies versus other psychological therapies for depression. Cochrane Database Syst Rev 2013;10:CD008704. [CrossRef] [Medline]
    69. Cuijpers P, Huibers M, Ebert DD, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? A metaregression analysis. J Affect Disord 2013 Jul;149(1-3):1-13. [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    NNT: number needed to treat
    RCT: randomized controlled trial
    TWC: third-wave cognitive behavioral therapy

    Edited by G Eysenbach; submitted 23.03.16; peer-reviewed by J García Campayo, C Buntrock; comments to author 20.04.16; revised version received 15.11.16; accepted 23.11.16; published 17.02.17

    Copyright

    ©Elena Heber, David Daniel Ebert, Dirk Lehr, Pim Cuijpers, Matthias Berking, Stephanie Nobis, Heleen Riper. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 17.02.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.