This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3092) Anxiety and Stress Disorders (999)

Published on in Vol 18, No 1 (2016): January

Web-Based and Mobile Stress Management Intervention for Employees: A Randomized Controlled Trial

Web-Based and Mobile Stress Management Intervention for Employees: A Randomized Controlled Trial

Web-Based and Mobile Stress Management Intervention for Employees: A Randomized Controlled Trial

Authors of this article:

Elena Heber1 Author Orcid Image ;   Dirk Lehr1, 2 Author Orcid Image ;   David Daniel Ebert1, 3 Author Orcid Image ;   Matthias Berking1, 3 Author Orcid Image ;   Heleen Riper1, 4, 5 Author Orcid Image
Article Authors Cited by (139) Tweetations (18) Metrics

    Original Paper

    1Leuphana University Lueneburg, Division of Online Health Training, Innovation Incubator, Lueneburg, Germany

    2Leuphana University Lueneburg, Institute of Psychology, Lueneburg, Germany

    3Friedrich-Alexander-University Erlangen-Nuremberg, Department of Clinical Psychology and Psychotherapy, Erlangen, Germany

    4VU University Amsterdam, Department of Clinical Psychology, Amsterdam, Netherlands

    5Telepsychiatric Centre, University of Southern Denmark, Odense, Denmark

    Corresponding Author:

    Elena Heber, MSc

    Leuphana University Lueneburg

    Division of Online Health Training, Innovation Incubator

    Scharnhorststrasse 1

    Lueneburg, 21335

    Germany

    Phone: 49 4131 677 1708

    Fax:49 4131 677 1717

    Email: heber@leuphana.de


    Background: Work-related stress is highly prevalent among employees and is associated with adverse mental health consequences. Web-based interventions offer the opportunity to deliver effective solutions on a large scale; however, the evidence is limited and the results conflicting.

    Objective: This randomized controlled trial evaluated the efficacy of guided Web- and mobile-based stress management training for employees.

    Methods: A total of 264 employees with elevated symptoms of stress (Perceived Stress Scale-10, PSS-10≥22) were recruited from the general working population and randomly assigned to an Internet-based stress management intervention (iSMI) or waitlist control group. The intervention (GET.ON Stress) was based on Lazarus’s transactional model of stress, consisted of seven sessions, and applied both well-established problem solving and more recently developed emotion regulation strategies. Participants also had the opportunity to request automatic text messages on their mobile phone along with the iSMI. Participants received written feedback on every completed session from an e-coach. The primary outcome was perceived stress (PSS-10). Web-based self-report assessments for both groups were scheduled at baseline, 7 weeks, and 6 months. At 12 months, an extended follow-up was carried out for the iSMI group only.

    Results: An intention-to-treat analysis of covariance revealed significantly large effect differences between iSMI and waitlist control groups for perceived stress at posttest (F1,261=58.08, P<.001; Cohen’s d=0.83) and at the 6-month follow-up (F1,261=80.17, P<.001; Cohen’s d=1.02). The effects in the iSMI group were maintained at 12-month follow-up.

    Conclusions: This Web- and mobile-based intervention has proven effective in reducing stress in employees in the long term. Internet-based stress management interventions should be further pursued as a valuable alternative to face-to-face interventions.

    Trial Registration: German Clinical Trials Register (DRKS): 00004749; http://drks-neu.uniklinik-freiburg.de/ drks_web/setLocale_EN.do (Archived by WebCite at http://www.webcitation.org/6e8rl98nl)

    J Med Internet Res 2016;18(1):e21

    doi:10.2196/jmir.5112

    Keywords

    Internet; randomized controlled trial; work; stress; stress management; mental health 


    Introduction

    Stress and related adverse outcomes for physical and mental health are highly prevalent and pose a major threat to public health. Individuals with high stress levels face various negative consequences of stress including sleeping problems [1], burnout [2], an increased risk of depression, anxiety [3], and coronary heart disease [4,5]. According to a recent survey [6], 31% of US employees feel tense or stressed on a daily basis. Meanwhile, 64% report receiving insufficient stress management resources from their employers.

    In recent years, Web-based and mobile-based interventions for coping with work-related stress have emerged. The advantages attributed to Web-based interventions include the potential for large-scale delivery, 24/7 availability, low costs and a low-access threshold [7]. Moreover, a recent meta-analysis reveals an equivalence between face-to-face and Internet-based guided cognitive behavioral therapy [8]. For populations experiencing high levels of work-related stress, Web-based interventions can be an appealing method for flexibly integrating stress management exercises into daily life. In particular, mobile behavioral intervention technologies for mental health offer the potential to deliver training components in real time and the real world [9]. Internet-based interventions may also reach those who are unwilling to participate in traditional face-to-face interventions [10].

    Face-to-face training on stress management has been proven to be effective [11-13]. However, the evidence base for Internet-based stress management interventions (iSMIs) remains inconclusive, as only a limited number of randomized controlled trials (RCTs) have been conducted. Some of these studies showed a significant moderate reduction of stress for Web-based interventions compared with a waitlist group [14-17], a no-treatment group [18], and an attention control group [19]. Other studies did not find beneficial between-group effects for stress at posttest [20-22]. For instance, Wolever et al [18] found an effect size of d=0.74 for reduced stress for a guided mindfulness at work intervention, whereas Wiegand et al [21] did not find significant between-group effects for an unguided comprehensive Web program that included olfactory care products for women. This lack of conclusiveness of iSMIs also applies to other mental health indicators, such as depression. The differences in effectiveness may result from variations in the type and length of interventions studied, the usage of guidance, the outcomes, the measurements, or the setting. Likewise, little is known about the long-term efficacy of iSMIs. Two RCTs have investigated the efficacy of iSMIs at the 6-month follow-up relative to a control group finding a non-significant effect for stress reduction in students [23] and a small to moderate effect for the general population [14]. An extended follow-up conducted by Ruwaard et al [16] over a 3-year period yielded beneficial results for reducing stress. However, no RCTs investigating an intervention combining Web-based and mobile components with a focus on stress reduction have addressed employees as a target group relative to a control group in long-term follow-ups (eg, 6 months). With regard to the content of such interventions, currently available iSMIs do not base their theoretical foundation on a specific stress model such as the effort-reward imbalance model [24] or the job-demand control model [25]. Likewise, more generic, established models of stress, such as Lazarus’s transactional model of stress [26] are not applied. Lazarus’s transactional model of stress specifies two coping strategies. Problem-focused coping is used to actively influence a stress situation in a positive way through the use of cognitive or behavioral efforts. Emotion-focused coping primarily serves the function of managing difficult emotions such as anger, disappointment, and sadness in relation to the specific situation. On the one hand, employees are often faced with problems that theoretically can be solved. Problem solving [27] is an established therapeutic method in dealing with such problems and has been proven to be successful in reducing mental and physical health problems [28]. This method has also been effectively used in Web-based interventions to manage depression, anxiety, and stress [29], although mixed results have been observed in studies targeting employees with depressive symptoms [30,31]. On the other hand, the working context also frequently requires dealing with problems that are unsolvable; such situations are commonly accompanied by strong negative emotions and require effective strategies on how to regulate these emotions. Emotion regulation skills have been shown to be relevant and successful in a broad range of mental disorders including depression and anxiety [32]; nevertheless, they remain largely untargeted in research on stress management interventions. From a theoretical perspective, promoting problem- and emotion-focused coping skills according to Lazarus’s model as two major intervention components within the same intervention appears promising; however, this approach has not yet been introduced. This study aimed to fill this gap in the research by investigating an iSMI based on the combination of problem solving and emotion regulation.

    This paper presents the results of a waitlist-controlled randomized trial to investigate the efficacy of a newly developed iSMI that includes mobile components for reducing stress in employees with elevated stress levels. We assessed whether the participants in the intervention group (iSMI) reported significantly lower scores on the primary outcome of perceived stress on the Perceived Stress Scale-10 (PSS-10) at posttest and at 6-month follow-up as compared to those in a waitlist control (WLC) group. Among the secondary outcomes, selected mental- and work-related health indicators often perceived to arise due to chronic stress, such as depression, anxiety, and emotional exhaustion, were also considered.


    Trial Design

    Using a 2-arm randomized controlled design, 264 participants were randomly allocated (at a ratio of 1:1 and a block size of 2) to the iSMI or to a WLC group. Both groups had full access to treatment as usual.

    Participants

    Participants 18 years and older were included if they were currently employed and scored 22 or above on the PSS-10. Due to the fact that the PSS is not a diagnostic measurement and there is no official cut-off available, we decided to use one standard deviation (SD 6.2) above the mean (PSS-10=15.3) in a large working population [33] as a cut-off value to choose participants with an elevated level of stress. We excluded any applicants who were at risk of suicide (Beck Suicide Item >1; [34]) or self-reported to have been previously diagnosed with dissociative or psychotic symptoms. Participants were recruited in Germany from January to October 2013 in the general working population through newspaper articles and announcements by the ministry of education. Primarily, they were recruited through a large German health insurance company. The intervention addressed employees who were frequently stressed or exhausted, who felt that problems were increasingly difficult to handle, and who struggled to cope with difficult emotions. The intervention was advertised in the print membership magazine of a large German health insurance company. It was stated that, in cooperation with a university, the health insurance company offered this online training to employees, provided they were suitable for study inclusion. Moreover, the advantages of the intervention such as 24/7 availability, personal e-coaching, and participation free of charge were delineated. Those interested in participating had to provide an email address and a first and last name that could be pseudonyms if desired. Individuals received a link to the online screening questionnaire via email. Provided they were eligible, applicants had to submit their signed informed consent via regular post or scanned via email. Upon receipt of the informed consent, participants had to complete all baseline questionnaires. Subsequently, they were randomized into either the intervention or the waitlist control group.

    Intervention

    The iSMI GET.ON Stress is based on Lazarus’s transactional model of stress [26]. This intervention applied both well-established problem solving and more recently developed emotion regulation strategies. Important principles for health behavior change such as goal setting, action planning, and coping planning were followed. The iSMI consisted of seven sessions and a booster session provided 4 weeks after training completion. Following psycho-education (Session 1), the participants learned a 6-step procedure to systematically solve problems (Sessions 2-3). In Sessions 4-6, the participants were introduced to emotion regulation techniques (muscle- and breathing relaxation, acceptance of negative emotions, and self-support in difficult situations). Session 7 included a plan for the future. The iSMI was specifically tailored to employees; this was reflected in the wording of the intervention, the example characters provided throughout the training, as well as in optional short informational material related to typical stress-related topics (eg, psychological detachment from work, time management, sleep hygiene, worrying, and organization of breaks during work) provided alongside the intervention. The application of exercises was strongly recommended. The participants were advised to complete 1-2 sessions per week. The program included interactive exercises, audio/video files, and downloadable material and was presented on a secured Web-based platform. Upon activation of the account through the research team, participants used their email address and a self-chosen password to log on. Within 48 hours after session completion, an e-coach provided approximately three quarters of a page of written, non-therapeutic feedback intended to increase adherence and motivation. The e-coaches reported that the average time spent per feedback was 30 minutes. In the event of non-completion of a session, they also sent reminders. Each e-coach had a degree in psychology and followed a standardized manual on feedback writing. Fidelity and adherence to the feedback manual was ensured by providing extensive coaching on feedback writing and by employing a psychotherapist who provided supervision for the e-coaches. The participants could receive automatic text messages on their mobile phone along with the iSMI (eg, short relaxation exercises: “Relax your muscles in your hands and arms for 3 seconds now. Follow your breathing and each time you breathe out, relax a little more.”) and were given the choice of either light (1 text message every other day) or intensive support (2-3 text messages per day) according to personal preferences. The text message coach was part of the intervention, aimed at reminding participants to practice and increasing the adherence to the intervention [9]. A more detailed description of the iSMI can be found in the protocol of the trial [35]. Screenshots of the intervention are available in the Multimedia Appendix 1.

    Measurements

    All questionnaires were self-assessed online at baseline (T1), 7 weeks (T2, post-treatment), 6 months (T3), and 12 months (T4, iSMI group only) after randomization. The WLC group received access to the intervention following T3.

    Primary Outcome Measure

    The primary outcome was the level of perceived stress as measured by the PSS-10 [36]. As this scale is based on Lazarus’s transactional model of stress, it fits well with the theoretical basis of the intervention. We further decided to employ a general stress scale as previous research in a similar intervention for employees showed that work-related and non-work-related problems are equally often indicated and addressed [30]. The items were answered using a 5-point Likert scale (0=never; 1=almost never; 2=sometimes; 3=fairly often; 4=very often; range 0-40) referring to the past week. Cronbach alphas for this scale have been reported to range from .78 to .91 [37] and was .70 at T1, .90 at T2, .90 at T3, and .91 at T4 in this study.

    Secondary Outcome Measures

    Mental Health

    Among the secondary outcomes concerning mental health, the following outcomes were measured using the specified scales: depression, using the Center for Epidemiological Studies’ Depression Scale (CES-D) [38] (20 items; range 0-60; α=.91); insomnia severity, using the Insomnia Severity Index (ISI) [39] (7 items; range 0-28; α=.90); anxiety, using the subscale of the Hospital Anxiety and Depression Scales (HADS-A) [40] (7 items; range 0-21; α=.83); worrying, using the Penn State Worry Questionnaire, Ultra Brief Version-past week (PSWQ-PW) [41] (3 items; range 0-18; α=.87); and quality of life, using the Short Form 12 (SF-12) PH (physical health) and MH (mental health) [42].

    Work-Related Health

    Within the area of work-related health, we assessed emotional exhaustion, using the subscale of the Maslach Burnout Inventory (MBI-EE) [43] (5 items; range 1-6; α=.89); work engagement, using the Utrecht Work Engagement Scale (UWES) [44] (9 items; range 0-6; α=.94); and psychological detachment, using the subscale of the Recovery Experience Questionnaire (REQ-PD) [45] (4 items; range 1-5; α=.93). Moreover, mean days of absenteeism and presenteeism within the previous 3 months were assessed using the respective items of the German Version of the Trimbos and Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry (TiC-P-G) [46].

    Skills/Competencies

    Emotion regulation in terms of comprehension (-C), acceptance (-A), and self-support (-SS) of the Emotion Regulation Skills Questionnaire (ERSQ), using the ERSQ-27 [47] (9 items; range 0-4; α=.87, .86, .85), and general distress, using the Emotion Specific Version, ERSQ-ES-GD [48] (12 items; range 0-4; α=.88) were assessed as measures of skills/competencies.

    Other Measures

    Client satisfaction, using the Client Satisfaction Questionnaire (CSQ-8) [49], demographic variables, and reasons for dropout will also be reported.

    Sample Size

    As the current evidence base for Web-based stress-management is limited and the effect sizes vary considerably between trials (from non-significant to moderate-large), we decided to use a conservative approach in order to also detect small effect sizes. We relied on a meta-analysis on traditional stress management interventions [13] and expected an effect size of d=0.35. Therefore, based on an alpha of .05 (two-tailed test), and a power of 80%, a sample size of 132 participants per group was necessary.

    Randomization

    The applied random integer list was created by an independent researcher using a Web-based randomization program (Randlist). The participants were informed about the randomization outcome via email.

    Statistical Analyses

    All analyses are reported according to the Consolidated Standards of Reporting Trials (CONSORT) statement regarding eHealth [50] using intention-to-treat (ITT) procedures (see Multimedia Appendix 2). Additionally, per-protocol and study completers-only analyses are reported. A significance level of .05 (two-sided) was used for all analyses. Analyses were performed using IBM SPSS version 22.

    Missing Data

    Multiple imputation was used to handle missing data [51]. Ten single imputations of the missing values were calculated based on the valid data for all outcome measures at all assessment points (T1, T2, T3, and T4) as well as age and gender and were aggregated into a single overall estimate of the effects of the intervention.

    Intervention Effect

    The iSMI and WLC groups were compared at 7 weeks (T2) and 6 months (T3) using analysis of covariance (ANCOVA) with baseline levels as covariates. Cohen’s d with 95% confidence intervals (CIs) was calculated based on the imputed dataset by comparing the means and SDs of the iSMI and WLC groups at the respective time points (eg, post-means and post-SDs). According to Cohen [52], d=0.2 can be considered a small effect, d=0.5 a medium effect, and d=0.8 a large effect.

    Reliable Change

    The clinical significance in terms of reliable change was calculated according to the method of Jacobson and Truax [53] using the following formula: 1.96 × SD1 × sqrt(2) × sqrt(1-rel). Thereby, we used the standard deviation of the norm population (SD 6.2) and the reliability of the PSS-10 scale (α=.91) according to Cohen’s and Janicki-Deverts’ samples in 2006 and 2009 [37]. The participants were defined as having reliably changed if their PSS-10 score differed more than (+/-) 5.16 points from T1-T2 and T1-T3.

    Symptom-Free Status

    According to Jacobson and Truax [53], a cut-off point indicating symptom-free status was calculated and defined as scoring more than 2 SDs below the mean (T1) of the stressed population.

    Number Needed to Treat

    The number needed to treat (NNT) indicates the number of participants who must be treated to generate one additional clinically significant change. NNTs and their 95% confidence intervals [54,55] were calculated for reliable change and symptom-free status.

    All procedures involved in the study were consistent with the generally accepted standards of ethical practice and were approved by the ethical committee of the University of Marburg (reference number AZ 2012-43K).


    Participants

    A total of 450 individuals were screened for eligibility, and 186 were excluded primarily because they scored below the cut-off (136/450), because of a lack of informed consent/baseline (30/450), or other reasons (20/450). The study flow is illustrated in Figure 1.

    ‎
    Figure 1. Flow of participants.
    View this figure

    Baseline Characteristics

    The demographic variables for all study participants are displayed in Table 1.

    The average age of the participants was 43.3 years (SD 10.2). The sample was primarily female (193/264, 73.1%), married or in a relationship (160/264, 60.6%), and highly educated (203/264, 76.9%). Most participants were employed full-time (204/264, 77.3%); their average working experience was 18.1 (SD 11.1) years; and they were working in various sectors, most frequently in the social sector (97/264, 36.7%). Only a small percentage of participants had previously taken part in any health training (34/264, 12.9%). Having received psychotherapy was indicated by 95 (36.0%) of the 264 participants and currently being in psychotherapy by 16/264 (6.1%). Table 2 summarizes all means and SDs for the iSMI and WLC groups.

    Table 1. Baseline characteristics.
    CharacteristicsAll participants (n=264)iSMI (n=132)WLC (n=132)
    Sociodemographic characteristics

    		Age, mean (SD)43.3 (10.2)42.4 (10.7)44.2 (9.6)

    		Gender, female, n (%)193 (73.1)97 (73.5)96 (72.7)

    		Married or in a relationship, n (%)160 (60.6)80 (60.6)80 (60.6)
    Ethnicity, n (%)

    		Caucasian/white220 (83.3)110 (83.3)110 (83.3)

    		Prefer not to say44 (16.7)22 (16.7)22 (16.7)
    Educational level, n (%)

    		Low5 (1.9)3 (2.3)2 (1.5)

    		Middle56 (21.2)25 (18.9)31 (23.5)

    		High203 (76.9)104 (78.8)99 (75.0)
    Work characteristics

    		Full-time employed, n (%)204 (77.3)105 (79.5)99 (75.0)

    		Part-time employed, n (%)57 (21.6)25 (18.9)32 (24.2)

    		On sick leave, n (%)3 (1.1)2 (1.5)1 (0.8)

    		Work experience in years, mean (SD)18.1 (11.1)17.2 (10.8)18.9 (11.2)
    Work sectors, n (%)

    		Social97 (36.7)48 (36.4)49 (37.1)

    		Service43 (16.3)21 (15.9)22 (16.7)

    		Health36 (13.6)22 (16.7)14 (10.6)

    		Economy31 (11.7)14 (10.6)17 (12.9)

    		IT15 (5.7)8 (6.1)7 (5.3)

    		Others42 (16.0)19 (14.3)23 (17.4)
    Income in Euro, per year, n (%)

    		<10,00010 (3.8)7 (5.3)3 (2.3)

    		10,000-30,00048 (18.2)28 (21.2)20 (15.2)

    		30,000-40,00063 (23.9)26 (19.7)37 (28.0)

    		40,000-50,00047 (17.8)26 (19.7)21 (15.9)

    		50,000-60,00029 (11.0)15 (11.4)14 (10.6)

    		60,000-100,0032 (12.1)17 (12.9)15 (11.4)

    		>100,0004 (1.5)1 (0.8)3 (2.3)

    		Prefer not to say31 (11.7)12 (9.1)19 (14.4)
    Experience, n (%)

    		Previous health training34 (12.9)17 (12.9)17 (12.9)

    		Previous psychotherapy95 (36.0)52 (39.4)43 (32.6)

    		Current psychotherapy16 (6.1)5 (3.8)11 (8.3)
    Table 2. Means and standard deviations for the iSMI group and the WLC group (ITT sample).

    		T1T2aT3aT4a,b
    iSMIWLCiSMIWLCiSMIWLCiSMI
    OutcomeMSDMSDMSDMSDMSDMSDMSD
    Primary outcome

    		PSS-1025.893.8525.153.9617.886.1722.966.0716.086.0322.105.8116.256.35
    Mental health

    		CES-D23.348.4723.777.5915.619.0921.358.8213.837.7121.498.4814.568.71

    		ISI15.536.6214.826.488.795.8111.206.158.185.5811.005.357.824.95

    		HADS-A11.233.3410.673.367.793.9810.323.466.733.389.653.586.513.46

    		PSWQ-PW10.173.9810.233.746.823.998.914.216.083.788.654.365.833.82

    		SF-12 MHc32.298.4432.558.08N/AN/AN/AN/A43.3810.5636.549.5044.169.69

    		SF-12 PHc48.2410.0548.449.43N/AN/AN/AN/A49.238.1747.578.7549.276.39
    Work-related health

    		MBI-EE4.730.684.770.673.951.034.640.803.701.014.540.953.670.95

    		UWESc3.181.263.311.153.381.293.101.133.461.173.161.143.461.13

    		REQ-PDc2.110.812.160.852.750.912.260.923.060.952.380.872.970.88

    		Absenteeismd4.938.704.409.62N/AN/AN/AN/A3.646.705.2312.105.8810.57

    		Presenteeismd15.9814.2717.2916.51N/AN/AN/AN/A11.3212.8811.4711.938.229.59
    Skills/Competencies

    		ERSQ-Cc2.480.882.470.863.010.702.570.903.150.642.780.803.160.70

    		ERSQ-Ac1.950.931.980.792.640.762.160.852.840.782.310.882.820.78

    		ERSQ-SSc2.020.912.150.822.700.732.320.882.790.822.380.862.730.86

    		ERSQ-ES-GDc1.790.541.810.522.430.562.040.592.530.582.040.552.580.52

    aMissing data imputed by multiple imputation.

    bExtended follow-up for intervention group only.

    cHigher scores indicate better outcomes.

    dIn relation to the previous 3 months.

    Dropout Attrition and Handling of Missing Data

    Overall, 8.0% (21/264) of participants at T2, 10.6% (28/264) of participants at T3, and 30.3% (40/132; iSMI only) of participants at T4 did not provide follow-up data for the primary outcome. A somewhat higher dropout rate was observed for the iSMI group (T2: 16/132, T3: 17/132) compared with the WLC (T2: 5/132, T3: 11/132). Thereby, groups significantly differed at T2 (χ21=6.26; P<.05), but not at T3. Participants who did not provide follow-up data did not differ in a meaningful way from those who provided data, neither on baseline stress scores or any other baseline outcomes, with the exception of worrying (P<.05). Little’s overall test of randomness indicated that data were missing completely at random. Thus, multiple imputations to estimate missing values could be performed [56].

    Non-Usage Attrition

    Intervention

    Of the 132 individuals participating in the iSMI, Session 1 was completed by 122 of the participants (92.4%), Session 2 by 117 (88.6%), Session 3 by 112 (84.8%), Session 4 by 108 (81.8%), Session 5 by 103 (78.0%), Session 6 by 97 (73.5 %), Session 7 by 93 (70.5%), and the booster session by 72 (54.5%) of the participants. Because of a lack of time and changes in personal circumstances, 10 participants (7.6%) did not start the iSMI. Nine participants (6.8%) reported reasons for discontinuing the iSMI; these included lack of time (4/9), lack of motivation (3/9), technical problems (1/9), and dissatisfaction with the intervention (1/9). On average, the participants in the iSMI group completed 5.70 (SD 2.32) of the 7 sessions (81.4% of the intervention) and used the iSMI for 8.27 weeks (SD 8.54, range 0-56).

    Text Message Support

    Among the iSMI group, three-quarters (101/132, 76.5%) requested text message support via mobile phone. Of those, 43.6% (44/101) preferred light coaching and 56.4% (57/101) preferred intensive coaching at the start of the intervention.

    Other Treatment During the Trial

    In the WLC condition, 37 participants (28.0%) indicated at T2 that they had received other help within the previous 7 weeks (eg, psychotherapy, health training other than the iSMI) as opposed to 24 participants (18.2%) in the iSMI condition. No significant differences were found in stress levels between those participants who had received help and those who had not.

    Primary Outcome Analyses

    Intervention Effect

    As shown in Table 3, a significant group effect in the ANCOVA indicated that lower scores on the PSS-10 (relative to the WLC) were present for the iSMI group at T2 (F1,261=58.08, P<.001) and T3 (F1,261=80.17, P<.001). Large between-group effect sizes were observed at T2 (Cohen’s d=0.83; 95% CI 0.58-1.08) and T3 (d=1.02; 95% CI 0.76-1.27). In the intervention group, the within-group effect sizes were d=1.54 (95% CI 1.22-1.86) from pretest to post-test, d=1.92 (95% CI 1.55-2.29) from pre-test to 6-month follow-up, and d=1.83 (95% CI 1.45-2.21) from pre-test to 12-month follow-up. In the control group, within-group effect sizes of d=0.41 (95% CI 0.23-0.60) were observed from pre-test to post-test and d=0.60 (95% CI 0.39-0.81) from pre-test to 6-month follow-up. Figure 2 shows the PSS-10 scores for both groups at all assessment points.

    ‎
    Figure 2. Levels of perceived stress (means and SDs) according to the PSS-10 for the iSMI and WLC groups at all assessment points for the ITT sample at pre-test (T1), post-test (T2), 6 months (T3), and 12 months (T4, iSMI only) (asterisks indicate P<.001).
    View this figure
    Reliable Change

    At T2, more participants in the iSMI group (81/132, 61.4%) showed reliable improvement on the PSS-10 compared with the WLC (33/132, 25.0%). A reliable deterioration was present in 1.5% (2/132) of the iSMI and 8.3% (11/132) of the WLC, whereas 37.1% (49/132; iSMI) and 66.7% (88/132; WLC) were reliably unchanged. At T3, those showing reliable improvement numbered over three-quarters (102/132, 77.3%) in the iSMI and nearly half (44/132, 33.3%) in the WLC group. Those showing reliable deterioration numbered 0.8% (1/132) in the iSMI and 6.1% (8/132) in the WLC group. No reliable change was present in 22.0% (29/132; iSMI) and 60.6% (80/132; WLC). The NNTs for reliable improvement were 2.75 (95% CI 2.11-3.96) at T2 and 2.28 (95% CI 1.83-3.01) at T3. The groups significantly differed from T1-T2 (χ21=37.54, P<.001) and from T1-T3 (Fisher’s exact=53.53, P<.001).

    Table 3. Results of the ANCOVAs and Cohen’s d for the primary and secondary outcome measures (ITT sample) at posttest (T2) and at 6-month follow-up (T3).
    OutcomeT2a Between-groups effectT3a Between-groups effect
    d (95% CI)ANCOVAbd (95% CI)ANCOVAb
    F1,261PF1,261P
    Primary outcome

    		PSS-100.83 (0.58-1.08)58.08<.0011.02 (0.76-1.27)80.17<.001
    Mental health

    		CES-D0.64 (0.39-0.89)34.92<.0010.95 (0.69-1.20)68.29<.001

    		ISI0.40 (0.16-0.65)20.43<.0010.52 (0.27-0.76)28.82<.001

    		HADS-A0.68 (0.43-0.93)49.57<.0010.84 (0.59-1.09)78.94<.001

    		PSWQ-PW0.51 (0.26-0.75)19.74<.0010.63 (0.38-0.88)31.00<.001

    		SF-12 MHN/AN/A
    		0.68 (0.43-0.93)34.28<.001

    		SF-12 PHN/AN/A
    		0.20 (-0.05 to 0.44)4.27 .04
    Work-related health

    		MBI-EE0.75 (0.50-1.00)48.55<.0010.86 (0.60-1.11)63.84<.001

    		UWES0.23 (-0.01 to 0.47)13.28<.0010.26 (0.02-0.50)10.09.002

    		REQ-PD0.54 (0.29-0.78)27.45<.0010.75 (0.50-1.00)52.82<.001

    		AbsenteeismN/AN/A
    		0.16 (-0.08 to 0.40)2.94

    		PresenteeismN/AN/A
    		0.01 (-0.23 to 0.25)0.02
    Skills/Competencies

    		ERSQ-27-C0.55 (0.30-0.79)31.56<.0010.51 (0.27-0.76)22.03<.001

    		ERSQ-27-A0.60 (0.35-0.84)30.94<.0010.64 (0.39-0.88)34.96<.001

    		ERSQ-27-SS0.47 (0.23-0.71)29.32<.0010.49 (0.24-0.73)24.56<.001

    		ERSQ-ES-GD0.68 (0.43-0.93)36.39<.0010.87 (0.61-1.12)54.71<.001

    aMissing data imputed by multiple imputation.

    bControlling for pre-treatment scores (T1).

    Symptom-Free Status

    In this study, the cut-off score was 17.70 and below indicating a value of 2 SDs below the mean of the stressed population at T1 (mean 25.52, SD 3.91). More participants in the iSMI group met the criterion for full remission of stress symptoms compared with the WLC group at T2 (iSMI: 68/132, 51.5%; WLC: 26/132, 19.7%; χ21=29.14, P<.001; NNT=3.14, 95% CI 2.34-4.78) and T3 (iSMI: 79/132, 59.8%; WLC: 31/132, 23.5%; χ21=35.91, P<.001; NNT=2.75, 95% CI 2.11-3.95).

    Completers-Only Analysis

    Completers-only analyses on participants who completed all questionnaires revealed similar large effect sizes for the primary outcome at T2 (243/264, 92.0%, d=0.85; CI 0.59-1.11) and T3 (236/264, 89.4%, d=1.01; 95% CI 0.74-1.28).

    Secondary Outcome Analyses

    Table 3 also shows the results of the ITT analyses for secondary outcomes for mental health, work-related health, and skills/competencies. The ANCOVAs showed highly significant between-group effects for almost all outcomes at both assessment points; all significance levels were P<.001 apart from work engagement at T3 (P=.002) and the physical health component of quality of life at T3 (P=.04). Between-group effects were not significant for absenteeism and presenteeism.

    At T2, the majority of effect sizes were in the range of moderate (eg, d=0.40 for insomnia) to large (eg, d=0.75 for emotional exhaustion) apart from work engagement for which a small effect size was obtained (d=0.23). At T3, almost all effect sizes became more pronounced apart from the comprehension subscale of the ERSQ-27, which only slightly decreased (from d=0.55 at T2 to d=0.51 at T3). Thereby, large effect sizes were found at T3 for depression (d=0.95), anxiety (d=0.84), emotional exhaustion (d=0.86), and emotion regulation skills regarding general distress (d=0.87). The additional measurements taken at T3 yielded effect sizes of d=0.68 for the mental health component and d=0.20 for the physical health component of quality of life, as well as d=0.16 for absenteeism and d=0.01 for presenteeism.

    Extended Follow-Up at 12 Months

    The within-group effect size (from T1-T4) for the primary outcome PSS-10 was d=1.83 (95% CI 1.45-2.21). At 12-month-follow-up, improvements in all other outcome measures were also maintained at the 6-month level in the iSMI group apart from absenteeism (P<.05). On a descriptive level, the gains in mean days for absenteeism in the iSMI group from T1 (mean 4.93, SD 8.70) to T3 (mean 3.64, SD 6.70) could not be maintained at T4 (mean 5.88, SD 10.57). In contrast, the mean days for presenteeism in the iSMI group were almost reduced by half from T1 (mean 15.98, SD 14.27) to T4 (mean 8.22, SD 9.59).

    Client Satisfaction

    Client satisfaction with the training was high, with 92.2% (107/116) being “satisfied in an overall, general sense” (“very satisfied” or “mostly satisfied”). The majority of the participants indicated that they have received the kind of training they wanted (92.2%, 107/116; “yes, definitely” or “yes, generally”), that the training met their needs (88.8%, 103/116; “almost all […]” or “most […]”), that they are satisfied with the amount of training they received (87.9%, 102/116; “very satisfied” or “mostly satisfied”), that the training has helped them to deal more effectively with their problems (92.2%, 107/116; “yes it helped a great deal” or “yes, it helped”), and that they would use the training again if they needed to (92.2%, 107/116; “yes, definitely” or “yes, I think so”). Moreover, 90.5% (105/116) stated that they would recommend the iSMI to a friend (“yes, definitely” or “yes, I think so”).

    Explorative Analyses

    Intervention Completion

    A separate per-protocol analysis was conducted for participants who completed the intervention (≥6 sessions), which was defined as working through all of the theoretical intervention content presented up to Session 6. The ANCOVA showed significant differences between the subsample of intervention completers (97/132) and the WLC (132) with regard to perceived stress in favor of the experimental condition at T2 (F1,226=66.85, P<.001) and T3 (F1,226=74.70, P<.001) with slightly higher effect sizes at T2 (d=0.95; 95% CI 0.69-1.20) and T3 (d=1.05; 95% CI 0.79-1.31) as compared to the total iSMI sample. Within the iSMI group, we further compared intervention completers to non-completers. The ANCOVA showed a significant difference for reduction of perceived stress at T2 (F1,129=7.76, P=.006), but not at T3 or T4.

    Text Message Support

    There were no significant differences in the primary outcome between participants who received text messages and those who did not, nor was there any significant difference depending on the level of intensity of the individually chosen text message support.


    Principal Findings

    The primary aim of this study was to evaluate the efficacy of a guided iSMI for employees. For this purpose, a two-arm, waitlist-controlled randomized trial was conducted. The results indicate that the training is highly effective in reducing employee stress levels in the short term (d=0.83) and long term (d=1.02) compared with the levels observed in a waitlist control group. Reduced stress levels in the iSMI group could be maintained up to 12 months. Significant medium to large between-group effects were also found for relevant secondary outcomes concerning mental health (eg, depression), work-related health (eg, emotional exhaustion), and stress-related skills (eg, emotion regulation competencies). High levels of client satisfaction and adherence were observed, and the study dropout rate was low.

    The posttest effect size in stress reduction found in this study is larger than what has been found in other iSMI trials. Available RCTs on iSMI with employees show mixed between-group effects for stress at posttest, ranging from non-significant [22] to moderate effect sizes (eg, d=0.74; [18]). Several reasons could explain the large effect sizes found in this study. First, the intervention used a guided format. Guided iSMIs [16-18] appear to be more effective than unguided interventions [21,22,57,58], a result that is known from Web-based interventions for other mental health problems [59]. Second, the theoretical basis of the intervention was confined to two evidence-based components. Research on face-to-face interventions suggests that interventions with fewer treatment components are superior to those using more components [12]. Problem-solving training that has already been successfully introduced in other Web-based interventions to manage depressive symptoms [30] was combined here with evidence-based emotion regulation techniques including the acceptance of emotions and compassionate self-support based on the Affect Regulation Training (ART) [32]. Third, the use of mobile components to flexibly introduce training components into daily life in real time may have reinforced a regular application of the intervention exercises and therefore the efficacy of the training. The text messages were particularly popular among the participants, and given the choice of receiving the text messages, the vast majority requested this mobile component. However, we did not assess the actual engagement and future studies should compare the efficacy of the intervention with and without the mobile phone component. Fourth, the level of intervention adherence, which is regarded as leading to better treatment outcomes [60], was relatively high. Unfortunately, comparisons with adherence levels of other iSMIs are difficult as few studies have reported this information so far. Compared with the available intervention completion rates (eg, 38.5%, [14]; 44.0%, [61]; 88.2% [19]), the percentage of participants completing the intervention in this study was higher range (70.5%). Considerable efforts were undertaken to increase adherence through methods that are generally considered to be effective, including human support [62], interactive exercises [63], tailoring of the intervention [64], and reminders [65] via mobile phone. Finally, the effects may have further stabilized at T3 through the booster session, as booster sessions can be successful in maintaining treatment outcomes [66].

    With regard to long-term follow-up, this work is the first study of an intervention combining Web-based and mobile components that focuses on stress reduction in employees to assess the effects compared to a control group over a longer time period (ie, 6 months). The results show that this type of intervention can have large long-term effects. Moreover, the results of this study also compare favorably to the limited existing evidence on the long-term effectiveness of iSMIs in populations other than employees, including studies finding non-significant [23] and small to moderate effect sizes (d=0.37 [14]) at the 6-month follow-up point. In addition, the results found for stress show similar effect sizes to those found for face-to-face interventions (d=0.73 [12]). Thus, iSMIs may be a useful alternative to traditional interventions.

    With regard to the relationship of treatment intensity and outcome, no clear conclusions can be drawn from this study. Although intervention completers showed significantly lower stress levels at posttest, this effect was not maintained at the later follow-up points. Participants receiving text message support were also not doing significantly better as compared to those who did not receive any messages. Future research may benefit from further information on the amount of time spent on the intervention exercises in between sessions and the actual engagement with the text messages.

    Limitations

    The following limitations of this trial must be acknowledged. First, for feasibility reasons, only self-report measures were assessed. Although the replacement of self-report measures with physiological measures is not recommended in occupational stress research [67], a combination of both could produce further valuable insights. Second, because this study was in the setting of indicated prevention, these results only account for participants showing relatively high baseline scores. The current sample was severely distressed and showed high baseline scores on all measures. Thus, no conclusions can be drawn regarding participants with lower stress levels (eg, in a universal prevention setting). Third, with regard to the generalizability of results, the fact that participants self-selected into the trial, the majority were female, and individuals working in the social sector were slightly overrepresented needs to be taken into consideration. Fourth, to determine the added value of the mobile component providing real-time support and encouragement, direct comparison studies would be needed comparing the intervention with and without mobile support. Fifth, the fact that the effects on physical health and work engagement were smaller than the effects on the other outcome measures demands an explanation. It is possible that Web-based interventions do not produce meaningful differences on these outcomes. Alternative explanations for the small effect sizes include that, for physical health, a rather global measure (SF-12) was applied and it may be more promising to use more specific, stress-related health measures. For work engagement, the original response categories were not adapted to the study period and the outcome measure may therefore not have been as sensitive to change. Finally, it is important to acknowledge that some improvements were also observed in the WLC group over time; in fact, this pattern has previously been found in other trials [17,31].

    Future Directions

    Future research should replicate the results of this trial and investigate the moderators of outcome and adherence. It is also of interest whether the coaching time spent on each individual (up to 4 hours) could be reduced without losses in treatment effects, thereby resulting in more economical versions of iSMI. Moreover, future research should test iSMIs against the gold standard in the field (ie, face-to-face interventions) and assess which training format works best for which type of participant and under what circumstances. Although both formats may be equally effective, they may work differently on participants with varying personal characteristics and Web-based interventions may be more advantageous in terms of efficiency and costs.

    Conclusion

    This trial contributes to the limited evidence base on the feasibility and efficacy of Web-based and mobile-supported stress management interventions and is among the first studies to include a longer follow-up period. The iSMI presented herein proved feasible and highly effective in improving perceived stress and other mental and work-related health indices in employees in the long term. These results indicate that this iSMI could be a valuable alternative to face-to-face trainings. Web-based interventions for coping with stress should be further evaluated as such interventions have the potential to improve the mental health of individuals on a large scale.

    Acknowledgments

    We would like to acknowledge the European Union (EFRE) for funding this project within the Lueneburg Innovation Incubator, TM 1.1 (CCI 2007DE161PR001). Additionally, we would like to thank Angelina Scheel and Stefanie Christmann for their assistance in administrative tasks and all of the e-coaches for providing guidance to the participants. This study was partially funded by the health insurance company, Barmer GEK.

    Authors' Contributions

    MB obtained the funding for the study. All authors contributed to the design of this study. EH was responsible for the intervention development, study administration, and statistical analyses of the data. DL, HR and DE supervised this process. All authors contributed to the data interpretation. EH drafted the manuscript. All authors contributed to the further writing of the manuscript and approved the final version of the manuscript.

    Conflicts of Interest

    DL, DE, and MB are stakeholders of the Institute for Online Health Training, which aims to transfer scientific knowledge related to this research into routine health care. EH is an employee at this Institute. The Institute was founded in January 2015. At the time of planning, conducting, and evaluating the study, it did not yet exist.

    ‎

    Multimedia Appendix 1

    Screenshots of the intervention.

    PDF File (Adobe PDF File), 458KB
    ‎

    Multimedia Appendix 2

    CONSORT-EHEALTH checklist V1.6.2 [67].

    PDF File (Adobe PDF File), 1MB
    1. Akerstedt T. Psychosocial stress and impaired sleep. Scand J Work Environ Health 2006 Dec;32(6):493-501 [FREE Full text] [Medline]
    2. Maslach C, Leiter M. The truth about burnout: how organizations cause personal stress and what to do about it. San Francisco, CA: Jossey-Bass; 1997.
    3. Melchior M, Caspi A, Milne BJ, Danese A, Poulton R, Moffitt TE. Work stress precipitates depression and anxiety in young, working women and men. Psychol Med 2007 Aug;37(8):1119-1129 [FREE Full text] [CrossRef] [Medline]
    4. Kivimäki M, Virtanen M, Elovainio M, Kouvonen A, Väänänen A, Vahtera J. Work stress in the etiology of coronary heart disease--a meta-analysis. Scand J Work Environ Health 2006 Dec;32(6):431-442 [FREE Full text] [Medline]
    5. Steptoe A, Kivimäki M. Stress and cardiovascular disease. Nat Rev Cardiol 2012 Jun;9(6):360-370. [CrossRef] [Medline]
    6. APA Center for Organizational Excellence. 2014 Work and well-being survey. 2014.   URL: http://www.apaexcellence.org/assets/general/2014-work-and-wellbeing-survey-results.pdf [accessed 2015-09-08] [WebCite Cache]
    7. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res 2006;8(2):e10 [FREE Full text] [CrossRef] [Medline]
    8. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
    9. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Feb;15(Pt 1):1-39 [FREE Full text] [CrossRef] [Medline]
    10. Marks I, Cavanagh K, Gega L. Hands-on help: computer-aided psychotherapy. Hove, England: Psychology Press; 2007.
    11. Bhui KS, Dinos S, Stansfeld SA, White PD. A synthesis of the evidence for managing stress at work: a review of the reviews reporting on anxiety, depression, and absenteeism. J Environ Public Health 2012;2012:515874 [FREE Full text] [CrossRef] [Medline]
    12. Richardson KM, Rothstein HR. Effects of occupational stress management intervention programs: a meta-analysis. J Occup Health Psychol 2008 Jan;13(1):69-93. [CrossRef] [Medline]
    13. van der Klink JJ, Blonk RW, Schene AH, van Dijk FJ. The benefits of interventions for work-related stress. Am J Public Health 2001 Feb;91(2):270-276. [Medline]
    14. Drozd F, Raeder S, Kraft P, Bjørkli CA. Multilevel growth curve analyses of treatment effects of a Web-based intervention for stress reduction: randomized controlled trial. J Med Internet Res 2013;15(4):e84 [FREE Full text] [CrossRef] [Medline]
    15. Hänggi Y. Kann web-basierte Stressprävention erfolgreich sein? [Can web-based parental training on coping with familiy stress be successful?]. Zeitschrift für Klinische Psychologie und Psychotherapie 2006 Jul 1;35(3):169-177. [CrossRef]
    16. Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B. E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther 2007;36(3):179-192. [CrossRef] [Medline]
    17. Zetterqvist K, Maanmies J, Ström L, Andersson G. Randomized controlled trial of internet-based stress management. Cogn Behav Ther 2003;32(3):151-160. [CrossRef] [Medline]
    18. Wolever RQ, Bobinet KJ, McCabe K, Mackenzie ER, Fekete E, Kusnick CA, et al. Effective and viable mind-body stress reduction in the workplace: a randomized controlled trial. J Occup Health Psychol 2012 Apr;17(2):246-258. [CrossRef] [Medline]
    19. Rose RD, Buckey JC, Zbozinek TD, Motivala SJ, Glenn DE, Cartreine JA, et al. A randomized controlled trial of a self-guided, multimedia, stress management and resilience training program. Behav Res Ther 2013 Feb;51(2):106-112. [CrossRef] [Medline]
    20. Glück TM, Maercker A. A randomized controlled pilot study of a brief web-based mindfulness training. BMC Psychiatry 2011;11:175 [FREE Full text] [CrossRef] [Medline]
    21. Wiegand B, Luedtke K, Friscia D, Nair M, Aleles M, McCloskey R. Efficacy of a comprehensive program for reducing stress in women: a prospective, randomized trial. Curr Med Res Opin 2010 Apr;26(4):991-1002. [CrossRef] [Medline]
    22. Yamagishi M, Kobayashi T, Nakamura Y. Effects of web-based career identity training for stress management among Japanese nurses: a randomized control trial. J Occup Health 2008;50(2):191-193 [FREE Full text] [Medline]
    23. Chiauzzi E, Brevard J, Thum C, Thurn C, Decembrele S, Lord S. MyStudentBody-Stress: an online stress management intervention for college students. J Health Commun 2008 Sep;13(6):555-572. [CrossRef] [Medline]
    24. Siegrist J. Adverse health effects of high-effort/low-reward conditions. J Occup Health Psychol 1996 Jan;1(1):27-41. [Medline]
    25. Karasek R. Job Demands, Job Decision Latitude, and Mental Strain: Implications for Job Redesign. Administrative Science Quarterly 1979 Jun;24(2):285-308. [CrossRef]
    26. Lazarus R, Folkman S. Stress, appraisal, and coping. New York: Springer Publishing Co; 1984.
    27. D'Zurilla T, Nezu A. Problem-solving therapy. In: Dobson KS, editor. Handbook of cognitive-behavioral therapies. 3rd edition. New York: Guilford Press; 2010.
    28. Malouff JM, Thorsteinsson EB, Schutte NS. The efficacy of problem solving therapy in reducing mental and physical health problems: a meta-analysis. Clin Psychol Rev 2007 Jan;27(1):46-57. [CrossRef] [Medline]
    29. van Straten A, Cuijpers P, Smits N. Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J Med Internet Res 2008;10(1):e7 [FREE Full text] [CrossRef] [Medline]
    30. Ebert DD, Lehr D, Boß L, Riper H, Cuijpers P, Andersson G, et al. Efficacy of an internet-based problem-solving training for teachers: results of a randomized controlled trial. Scand J Work Environ Health 2014 Nov;40(6):582-596. [CrossRef] [Medline]
    31. Geraedts AS, Kleiboer AM, Wiezer NM, van MW, Cuijpers P. Short-term effects of a web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res 2014;16(5):e121 [FREE Full text] [CrossRef] [Medline]
    32. Berking M, Whitley B. Affect Regulation Training. New York, NY: Springer New York; 2014.
    33. Lesage F, Berjot S, Deschamps F. Psychometric properties of the French versions of the Perceived Stress Scale. Int J Occup Med Environ Health 2012 Jun;25(2):178-184 [FREE Full text] [CrossRef] [Medline]
    34. Beck A, Steer R, Brown G. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation; 1996.
    35. Heber E, Ebert DD, Lehr D, Nobis S, Berking M, Riper H. Efficacy and cost-effectiveness of a web-based and mobile stress-management intervention for employees: design of a randomized controlled trial. BMC Public Health 2013;13:655 [FREE Full text] [CrossRef] [Medline]
    36. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983 Dec;24(4):385-396. [Medline]
    37. Cohen S, Janicki-Deverts D. Who's stressed? Distributions of psychological stress in the United States in probability samples from 1983, 2006, and 2009. J Appl Soc Psychol 2012;42(6):1320-1334. [CrossRef]
    38. Hautzinger M, Bailer M, Hofmeister D, Keller F. Allgemeine Depressionsskala (ADS) Manual [Center for Epidemiological Studies Depression Scale (CES-D; Radloff LS, 1977)]. Boston, MA: Hogrefe Publishers; 2012.
    39. Bastien CH, Vallières A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med 2001 Jul;2(4):297-307. [Medline]
    40. Hermann-Lingen C, Buss U, Snaith R. Hospital Anxiety and Depression Scale - Deutsche Version (HADS-D). 3rd edition. Bern: Hans Huber; 2011.
    41. Stöber J. PSWQ-PW: Penn State Worry Questionnaire-Past Week. In: Brähler E, Schumacher J, Strauß B, editors. Diagnostische Verfahren in der Psychotherapie. Göttingen, Germany: Hogrefe; 2002:291-294.
    42. Ware J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996 Mar;34(3):220-233. [Medline]
    43. Schaufeli W, Leiter M, Maslach C, Jackson S. Maslach Burnout Inventory - General Survey (MBI-GS). In: Maslach C, Jackson SE, Leiter MP, editors. Maslach Burnout Inventory Manual. 3rd edition. Palo Alto, CA: Consulting Psychologists Press; 1996.
    44. Schaufeli W, Bakker A. UWES - Utrecht Work Engagement Scale: Test Manual. Utrecht, The Netherlands: Department of Psychology, Utrecht University; 2003.
    45. Sonnentag S, Fritz C. The Recovery Experience Questionnaire: development and validation of a measure for assessing recuperation and unwinding from work. J Occup Health Psychol 2007 Jul;12(3):204-221. [CrossRef] [Medline]
    46. Hakkaart-van Roijen L. Manual Trimbos/iMTA Questionnaire for costs associated with psychiatric illness (in Dutch). Rotterdam: Institute for Medical Technology Assessment; 2002.
    47. Berking M, Znoj H. Entwicklung und Validierung eines Fragebogens zur standardisierten Selbsteinschätzung emotionaler Kompetenzen (SEK-27) [Development and validation of a self-report instrument for the assessment of emotion-specific regulation skills (ERSQ-27)]. Zeitschrift für Psychiatrie, Psychologie und Psychotherapie 2008 Jan;56(2):141-153. [CrossRef]
    48. Ebert DD, Christ O, Berking M. Entwicklung und Validierung eines Fragebogens zur emotionsspezifischen Selbsteinschätzung emotionaler Kompetenzen (SEK-ES) [Development and validation of a self-report instrument for the assessment of emotion-specific regulation skills (ERSQ-ES)]. Diagnostica 2013 Jan;59(1):17-32. [CrossRef]
    49. Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann 1982;5(3):233-237. [Medline]
    50. Eysenbach G, Consort-EHEALTHGroup. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
    51. Little R, Rubin D. Statistical analysis with missing data. Hoboken, NJ: Wiley; 2002.
    52. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
    53. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline]
    54. Altman DG. Confidence intervals for the number needed to treat. BMJ 1998 Nov 7;317(7168):1309-1312 [FREE Full text] [Medline]
    55. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ 1995 Feb 18;310(6977):452-454 [FREE Full text] [Medline]
    56. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods 2002 Jun;7(2):147-177. [Medline]
    57. Billings DW, Cook RF, Hendrickson A, Dove DC. A web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med 2008 Aug;50(8):960-968 [FREE Full text] [CrossRef] [Medline]
    58. Hinman M, Ezzo L, Hunt D, Mays A. Computerized exercise program does not affect stress levels of asymptomatic VDT users. J Occup Rehab 1997 Mar;7(1):45-51. [CrossRef]
    59. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    60. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
    61. Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: A randomized controlled trial. Internet Interventions 2014 Jul;1(3):95-101. [CrossRef]
    62. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    63. Hasson H, Brown C, Hasson D. Factors associated with high use of a workplace web-based stress management program in a randomized controlled intervention study. Health Educ Res 2010 Aug;25(4):596-607 [FREE Full text] [CrossRef] [Medline]
    64. Kreuter MW, Wray RJ. Tailored and targeted health communication: strategies for enhancing information relevance. Am J Health Behav 2003 Dec;27 Suppl 3:S227-S232. [Medline]
    65. Fry JP, Neff RA. Periodic prompts and reminders in health promotion and health behavior interventions: systematic review. J Med Internet Res 2009;11(2):e16 [FREE Full text] [CrossRef] [Medline]
    66. Whisman MA. The efficacy of booster maintenance sessions in behavior therapy: Review and methodological critique. Clinical Psychology Review 1990 Jan;10(2):155-170. [CrossRef]
    67. Semmer N, Grebner S, Elfering A. Beyond self-report: using observational, physiological, situation-based measures in research on occupational stress. In: Perrewe P, Ganster D, editors. Emotional and Physiological Processes and Positive Intervention Strategies (Research in Occupational Stress and Well-being, Volume 3). Bingley, UK: Emerald Group Publishing Limited; 2003:205-263.

    ‎
    ANCOVA: analysis of covariance
    CES-D: Center for Epidemiological Studies’ Depression Scale
    CONSORT: Consolidated Standards of Reporting Trials
    CSQ-8: Client Satisfaction Questionnaire
    ERSQ-27-A: Emotion Regulation Skills Questionnaire – subscale acceptance
    ERSQ-27-C: Emotion Regulation Skills Questionnaire – subscale comprehension,
    ERSQ-ES-GD: Emotion Regulation Skills Questionnaire – Emotion specific version – subscale general distress
    ERSQ-27-SS: Emotion Regulation Skills Questionnaire – subscale self-support
    HADS-A: Hospital Anxiety and Depression Scales – subscale anxiety
    ISI: Insomnia Severity Index
    iSMI: Internet-based stress management intervention
    ITT: intention-to-treat
    MBI-EE: Maslach Burnout Inventory–subscale Emotional Exhaustion
    NNT: number needed to treat
    PSS-10: Perceived Stress Scale – 10 item version
    PSWQ-PW: Penn State Worry Questionnaire-Past Week
    RCT: randomized controlled trial
    REQ-PD: Recovery Experience Questionnaire – subscale psychological detachment
    SD: standard deviation
    SF-12 MH: Short Form 12–Quality of life (mental health component)
    SF-12 PH: Short Form 12–Quality of life (physical health component)
    TIC-P-G: Trimbos and Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry
    UWES: Utrecht Work Engagement Scale
    WLC: waitlist control

    Edited by G Eysenbach; submitted 10.09.15; peer-reviewed by N Glozier, D Richards; comments to author 05.10.15; revised version received 17.10.15; accepted 19.10.15; published 27.01.16

    Copyright

    ©Elena Heber, Dirk Lehr, David Daniel Ebert, Matthias Berking, Heleen Riper. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 27.01.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.