This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297) Anxiety and Stress Disorders (1053)
Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9293, first published .
Effectiveness of an Internet- and App-Based Intervention for College Students With Elevated Stress: Randomized Controlled Trial

Effectiveness of an Internet- and App-Based Intervention for College Students With Elevated Stress: Randomized Controlled Trial

Effectiveness of an Internet- and App-Based Intervention for College Students With Elevated Stress: Randomized Controlled Trial

Authors of this article:

Mathias Harrer1 Author Orcid Image ;   Sophia Helen Adam1 Author Orcid Image ;   Rebecca Jessica Fleischmann1 Author Orcid Image ;   Harald Baumeister2 Author Orcid Image ;   Randy Auerbach3 Author Orcid Image ;   Ronny Bruffaerts4 Author Orcid Image ;   Pim Cuijpers5 Author Orcid Image ;   Ronald C Kessler6 Author Orcid Image ;   Matthias Berking1 Author Orcid Image ;   Dirk Lehr7 Author Orcid Image ;   David Daniel Ebert1 Author Orcid Image
Article Authors Cited by (134) Tweetations (14) Metrics

    Original Paper

    1Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany

    2Clinical Psychology and Psychotherapy, University of Ulm, Ulm, Germany

    3Department of Psychiatry, Columbia University, New York, NY, United States

    4Universitair Psychiatrisch Centrum, Katholieke Universiteit Leuven, Leuven, Belgium

    5Department of Clinical, Neuro, and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

    6Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

    7Division of Online Health Trainings, Innovation Incubator, Leuphana University, Lüneburg, Germany

    Corresponding Author:

    Mathias Harrer, BSc

    Clinical Psychology and Psychotherapy

    Friedrich-Alexander-University Erlangen-Nuremberg

    Nägelsbachstraße 25a

    Erlangen, 91052

    Germany

    Phone: 49 1708237654

    Email: mathias.harrer@fau.de


    Background: Mental health problems are highly prevalent among college students. Most students with poor mental health, however, do not receive professional help. Internet-based self-help formats may increase the utilization of treatment.

    Objective: The aim of this randomized controlled trial was to evaluate the efficacy of an internet-based, app-supported stress management intervention for college students.

    Methods: College students (n=150) with elevated levels of stress (Perceived Stress Scale 4-item version, PSS-4 ≥8) were randomly assigned to either an internet- and mobile-based stress intervention group with feedback on demand or a waitlist control group. Self-report data were assessed at baseline, posttreatment (7 weeks), and 3-month follow-up. The primary outcome was perceived stress posttreatment (PSS-4). Secondary outcomes included mental health outcomes, modifiable risk and protective factors, and college-related outcomes. Subgroup analyses were conducted in students with clinically relevant symptoms of depression (Center for Epidemiological Studies’ Depression Scale >17).

    Results: A total of 106 participants (76.8%) indicated that they were first-time help-seekers, and 77.3% (intervention group: 58/75; waitlist control group: 58/75) showed clinically relevant depressive symptoms at baseline. Findings indicated significant effects of the intervention compared with the waitlist control group for stress (d=0.69; 95% CI 0.36-1.02), anxiety (d=0.76; 95% CI 0.43-1.09), depression (d=0.63; 95% CI 0.30-0.96), college-related productivity (d=0.33; 95% CI 0.01-0.65), academic work impairment (d=0.34; 95% CI 0.01-0.66), and other outcomes after 7 weeks (posttreatment). Response rates for stress symptoms were significantly higher for the intervention group (69%, 52/75) compared with the waitlist control group (35%, 26/75, P<.001; number needed to treat=2.89, 95% CI 2.01-5.08) at posttest (7 weeks). Effects were sustained at 3-month follow-up, and similar findings emerged in students with symptoms of depression.

    Conclusions: Internet- and mobile-based interventions could be an effective and cost-effective approach to reduce consequences of college-related stress and might potentially attract students with clinically relevant depression who would not otherwise seek help.

    Trial Registration: German Clinical Trial Register DRKS00010212; http://www.drks.de/drks_web/navigate.do? navigationId=trial.HTML&TRIAL_ID=DRKS00010212 (Archived by WebCite at http://www.webcitation.org/6w55Ewhjd)

    J Med Internet Res 2018;20(4):e136

    doi:10.2196/jmir.9293

    Keywords

    randomized controlled trial; stress, psychological; depression; telemedicine; students; help-seeking behavior 


    Background

    Between 25% and 50% of college students meet the criteria for at least one mental health disorder in a given year [1,2]. Data suggest that mental disorders account for about half the disease burden of young adults in developed countries [3] and are associated with a range of negative consequences, including lowered academic performance [4] and college attrition [5].

    Despite the availability of effective treatment [6], only 1 in 5 students with mental disorders receives minimally adequate treatment [1]. Reasons for this treatment gap include attitudinal barriers such as stigma and a preference for self-help [7].

    Internet- and mobile-based interventions [8] might help to increase the utilization of psychological interventions, as they can be easily accessed, allow for high scalability, and can be provided at a low cost [9,10]. Internet-based interventions may also be suitable for college student populations [11], with research indicating that preference for help-seeking through the internet is higher among younger and well-educated individuals [12]. There is meta-analytic evidence suggesting the efficacy of internet interventions for a range of conditions and populations [6,13-16], including college students [17], with effect sizes of technology-delivered interventions ranging from standardized mean difference (SMD) of 0.42 to 0.43 for depression, 0.30 to 0.56 for anxiety, and 0.73 to 0.82 for stress [17,18]. However, the few results for internet and mobile-based interventions targeting stress in students are conflicting in terms of their effectiveness [19,20], warranting further research.

    A recent meta-analysis suggests that intervention effects are considerably higher in indicated compared with general student populations [18], stressing the importance of developing suitable intervention approaches for at-risk students. Internet-based interventions which are labeled to improve stress coping skills, as opposed to focusing on reducing symptoms of mental disorders, could represent a promising way to reach such burdened individuals. In an Australian investigation among severely distressed college students, 55.7% indicated that they were quite or very likely to use an internet-delivered program to seek help [21]. A significant association between heightened stress levels and positive attitudes toward internet intervention usage has also been found in a German general population sample [22].

    If proven to be effective, internet- and mobile-based approaches could provide a feasible instrument to help avert the onset of more severe stress-related mental health concerns in at-risk college students [8]. More research is therefore required to corroborate results on the effectiveness of internet- and mobile-based stress interventions and assess the potential of such interventions to reach and be effective in burdened students who already show symptoms of mental illness such as depression. Facing the deleterious effects of poor mental health on academic functioning, it is also important to assess whether such interventions may have an impact on important college-related outcomes such as academic self-efficacy and impairment [17].

    Objectives

    The aim of this study is thus to evaluate the effectiveness of an internet- and mobile-based intervention targeting university students with heightened stress levels. We hypothesized the internet intervention to be more effective in reducing symptoms of stress compared with a waitlist control group (WCG). It was furthermore assumed that more students participating in the intervention compared with the WCG would show a reliable change in perceived stress outcomes and attain close to symptom-free status. The second objective of this study was to investigate the hypothesized positive effect of the intervention on further mental health outcomes, modifiable risk and protective factors, and college-related outcomes compared with the WCG. Finally, our aim was to explore intervention participants’ adherence to, and acceptance of, the intervention.


    This study was carried out as part of the WHO World Mental Health International College Student project [23]. The WHO World Mental Health International College Student project aims to obtain accurate cross-national information on the prevalence, incidence, and correlates of mental, substance, and behavioral problems among college students worldwide, to describe patterns of service use and unmet need for treatment, to investigate the associations of these disorders with academic functioning, and to evaluate the effects of a wide range of preventive and clinical interventions on student mental health, functioning, and academic performance.

    Design

    A 2-armed randomized controlled trial was conducted with 150 participants, comparing an internet and app-based intervention with feedback on demand (StudiCare Stress) to a waitlist control group (WCG). Both conditions had full access to treatment as usual (TAU). The sample size allowed to detect effect sizes of d=0.41 with a power (1− β) of 0.80 with alpha of .05 and was based on a meta-analysis on internet-based interventions for college students, which reported an SMD of 0.73 for stress but lower effects for depression outcomes (SMD=0.43) [17]. A sample size of 150 was therefore chosen to also detect significant changes for secondary outcomes in this study such as depression.

    ‎
    Figure 1. Flow of participants (CONSORT flow chart). BDI: Beck Depression Inventory; IG: intervention group; WCG: waitlist control group; CONSORT: Consolidated Standards of Reporting Trials.
    View this figure

    Assessments took place at baseline (T1), posttreatment (T2; 7 weeks), and 3 months after baseline (T3; see Figure 1). Self-report data were collected using a Web-based assessment tool (Advanced Encryption Standard, 256-bit encryption). All procedures involved in the study were consistent with the generally accepted standards of ethical practice. The study was approved by the University of Erlangen-Nuremberg ethics committee (Erlangen, Germany; 322_15 B). The trial is registered in the German Clinical Trials Register (DRKS00010212 [24]). This study describes the main effectiveness analysis of the intervention; we also assessed moderator and mediator variables, which are listed in the trial registration (see Multimedia Appendix 1) and will be analyzed and reported in due length elsewhere.

    Participants

    Inclusion criteria were (1) elevated levels of perceived stress (Perceived Stress Scale 4-item version, PSS-4≥8 [25]; representing a level of stress one SD=2.92 above the mean of 4.49 in a large student sample [25]), (2) enrollment in a German-speaking university at the beginning of the training, (3) age ≥18 years, (4) internet access, (5) willingness to provide self-report data at all assessment points, and (6) informed consent. Exclusion criteria were (1) self-reported diagnosis of dissociative symptoms or psychosis in the past or (2) considerable risk for suicide (Beck Depression Inventory item 9 >1; “I feel I would be better off dead” or “I would kill myself if I had the chance”). Individuals showing an elevated risk for suicide were given detailed information about treatment options and were asked to see a physician or psychiatrist as soon as possible.

    Recruitment

    Participants were recruited via university press reports, student counseling services, and social media platforms. Potential participants declared interest in partaking in the study by filling out a Web-based registration form on the study website.

    Assessment of Eligibility and Randomization

    Individuals who declared interest in participating received an information letter along with an informed consent sheet and were asked to provide an email address for their intervention platform profile. Applicants were informed that withdrawal from the study was possible at any time, did not go along with any negative consequences, and all collected case data could be deleted on request during the study. Interested participants were asked to complete the written informed consent form and fill out the Web-based screening questionnaire.

    Individuals meeting all the inclusion and none of the exclusion criteria were invited to fill out the baseline assessment. After completion, individuals were randomly allocated to either the IG or the WCG. Randomization took place at a ratio of 1:1 and a block size of 2 using an automated computer-based random integer generator (Randlist, Datinf GmbH, Tübingen, Germany) and was performed by a researcher not otherwise involved in the study. Participants could not be blinded to study conditions; yet, during the randomization process, the allocation was concealed from participants, researchers involved in recruitment, and e-coaches.

    Study Conditions

    Intervention Condition

    The framework for StudiCare Stress was derived from GET.ON Stress, a Web-based stress management intervention for employees [26]. Changes in form and therapeutic content were made to tailor the intervention to university students’ needs.

    The intervention is based on cognitive-behavioral and third-wave techniques and aligns with Lazarus’ transactional model of stress [27] in differentiating between problem-focused and emotion regulation–focused coping. For problem-focused coping, cognitive-behavioral problem-solving strategies are applied to reduce and eliminate modifiable stressors. Emotion regulation refers to the processes through which individuals monitor, evaluate, modify, and thus control emotions to reach relevant needs or goals and has been shown to be influential in reducing various symptoms of mental illness [28]. Elective modules integrated at the end of session 2 to 7 could be chosen based on individual need and interest, covering student-specific topics: social support, rumination and worrying, time management, procrastination, test anxiety, sleep, motivation, nutrition and exercise, and dealing with writer’s block and concentration.

    The intervention comprised 8 main modules. Completing 1 module took 30 to 90 min, and participants were advised to complete at least one and a maximum of 2 modules per week. Thus, the intervention was intended to be completed in about 5 to 7 weeks (see Multimedia Appendix 2 for a detailed description of the modules).

    Strong emphasis was put on the transfer of acquired knowledge, strategies, and techniques into the students’ daily life through homework assignments. A personal diary app could be downloaded by participants to keep track of mood fluctuations, monitor factors contributing to their stress levels and reflect on intervention elements they could implement into their daily life. The diary app was introduced in module 1 as an adjunct to the main sessions and contained standardized free-text fields, rating scales, and gave the opportunity to add a photo to the entry (see Textbox 1). Participants were also provided with a PDF version of the diary and were instructed to monitor their mood 2 to 3 times each week, using either the app or a printout of the PDF for their entries.

    In addition, before beginning with the intervention, participants could request automatic daily messages containing short motivational prompts and ultrabrief training exercises via SMS (short message service), aimed at facilitating transfer of learned strategies into daily life routine. Messages were prescheduled to roughly mirror content and exercises provided through the progression of the intervention.

    Participants were guided by an eCoach, a trained student in a master’s program in psychology. Contact between the eCoach and intervention participants was solely established online, and there were no face-to-face meetings. An adherence-focused guidance concept in accordance with the human accountability model [29,30] was applied, which has been shown to be noninferior to intensive guidance while minimizing human resources ([31]; for a detailed description see [26]). Guidance consisted of 3 parts: (1) monitoring adherence (sending up to 3 reminders when a module was not completed during 1 week through the internal platform messaging system and via email), (2) checking the intervention platform back-end for participants who had completed a new module to unlock the next module and send standardized motivational messages through the platform, and (3) providing feedback on demand. When requesting help, participants received feedback within 48 hours.

    The feedback reflected the participants’ individual questions and problems and gave positive reinforcement. Feedback on demand was available for each participant from module 1 until completion of the booster session and was given via the internal messaging system of the training platform. Only few participants (5%, 4/75) requested individual feedback, resulting in 5 content feedbacks for the entire sample. In total, the eCoach sent 289 reminders (3.85 reminders per participant).

    Control Condition

    Students assigned to the waitlist control condition (WCG) completed the same assessments at T1, T2, and T3 as the intervention condition, but were not given access to the intervention until 3 months after randomization. Yet, they had full access to TAU offered by routine health care.

    General structure of the app-based diary entries.
    1. How do you feel today? (Emoticons: Happy–Sad–Anxious–Angry)
    2. How stressed out do you feel today? (Rating scale 1-10)
    3. Describe what happened today. (Free text)
    4. Were you able to identify any things contributing to your stress levels today? (Free text)
    5. Are there any techniques you previously learned that you may be able to apply? (Free text)
    6. Do you want to add a photo to your entry? (Upload button)
    Textbox 1. General structure of the app-based diary entries.

    Primary Outcome Measure

    The primary outcome was perceived stress as measured by the PSS-4 [25]. The PSS-4 assesses the degree to which individuals evaluate their lives as stressful, especially regarding how uncontrollable and overloading relevant aspects of life are perceived. The PSS-4 comprises 4 items (Item 1: “How often have you felt you were unable to control the important things in your life?”; Item 2: “How often have you felt confident about your ability to handle your personal problems?”; Item 3: “How often have you felt that things were going your way?”; Item 4: “How often have you felt difficulties were piling up so high that you could not overcome them?”), yielding a score between 0 and 16. Participants rated their level of perceived stress within the last 2 weeks on a 5-point Likert scale (0= never; 4= very often). A two-factor structure has been commonly found for the PSS [32-35], with positively framed items representing perceived coping self-efficacy and negative items reflecting hopelessness, the latter being a strong predictor for depression [36]. Higher scores on the PSS have shown to have good predictive validity for several adverse health outcomes [36-38]. Despite its brevity, the PSS-4 has been found to have acceptable to good psychometric properties [39,40]. The scale has a good level of internal consistency in this study as indicated by a Cronbach alpha of .83.

    Secondary Outcome Measures

    Unless otherwise specified, all outcomes were measured for a retrospective time frame of 2 weeks. All measures were administered in German. When no German translation was available, scales were translated independently by two of the researchers (MH and SHA), who then compared and discussed the translations to resolve disagreement.

    Mental Health

    To examine effects of the stress intervention on symptoms of common mental disorders, we included mental health outcomes associated with elevated distress in college students, including depression (short German form of the Center for Epidemiological Studies’ Depression Scale, CES-D [41]; 15 items, scale 0-3, range 0-45) and state anxiety (Spielberger State-Trait Anxiety Inventory [42]; 6 items, scale 1-4, range 6-24; at the moment) [43]. General well-being as an overall marker of mental health was assessed by the WHO-Five Well-Being Index (WHO-5 [44]; 5 items, scale 0-6, range 0-30), and emotional exhaustion using the Maslach Burnout Inventory-student version ([45]; 5 items, scale 1-6, range 5-30).

    Risk and Protective Factors

    Following measures for established risk and protective factors were assessed to investigate the intervention’s effect on individual resources and vulnerabilities related to the development and proliferation of mental illness: dysfunctional perfectionism [46] (Revised Almost Perfect Scale [47]; translated; 8 items, scale 1-7, range 8-56), resilience [48] (Connor-Davidson Resilience Scale short form [49]; translated; 2 items, scale 0-4, range 0-8), self-compassion [50,51] (Self-Compassion Scale [52]; 12 items, scale 1-5, range 12-60), and self-esteem [53] (Rosenberg Self-Esteem Scale [54]; 10 items, scale 1-4, range 10-40).

    College-Related Outcomes

    To evaluate presenteeism and loss of productivity, the Presenteeism Scale for Students’([55]; translated) subscale for work impairment (Work Impairment Scale; 10 items, scale 1-5, range 10-50) was administered. Productivity losses were assessed by an adaption of the Presenteeism Scale for Students’ work output scale, investigating the current percentage to which participants were able to reach their usual academic productivity. Productivity could be rated on a visual analog scale ranging from 0%= completely unproductive to 100%= full productivity. Academic self-efficacy was measured by the academic self-efficacy scale (Wirkstud [56]; 7 items, scale 1-4, range 7-28), and academic worrying using the Academic Worrying Questionnaire ([57]; translated; 10 items, scale 0-4, range 0-40).

    Additional Measures

    Additional questionnaires assessed demographic variables, prior contact with professional health providers, and satisfaction with the intervention (IG only; Client Satisfaction Questionnaire, adapted to the web context, CSQ-8 [58]; 8 items, scale 1-4). Treatment credibility and expectancies were measured at baseline by the Credibility and Expectancy Questionnaire ([59]; translated; 4 items, scale 1-5, range 4-20, 2 items, 0%-100%). Participants in the IG could give feedback on each modules’ usefulness (1= not useful at all, 5= very useful), complexity (1= very complex, 5= very easy), and duration until termination (1= less than ½ hour, 5= more than 1½ hours) on a 5-point Likert or 4-point scale, respectively.

    Statistical Analyses

    Main Effectiveness Evaluation

    All results are reported according to the Consolidated Standards of Reporting Trials statement ([60]; see Multimedia Appendix 3). Analyses based on the intention-to-treat (ITT) principle were conducted, with missing data imputed using a Markov chain Monte Carlo multivariate imputation algorithm (multiple imputation functions in IBM SPSS 23; IBM Corp, Armonk, NY, USA) with 100 estimations per missing and all variables set as predictors for imputation. Imputed datasets were then aggregated to obtain 1 imputed dataset.

    The hypothesized superiority of the internet intervention was tested with regard to (1) change in participants’ perceived stress and secondary outcomes from baseline (T1) to post intervention (T2) and 3-month follow-up (T3), (2) the number of participants with treatment response, (3) the number of students achieving close to symptom-free status, and (4) the amount of participants who experienced symptom deterioration.

    Differences in change of perceived stress between study arms were assessed using univariate analysis of covariance (ANCOVA) with scores at baseline as covariate to control for varying degrees of baseline scores. Effect sizes (Cohen's d) were calculated based on the imputed dataset for between-group differences, using the pooled IG and WCG SD [61]. To calculate 95% CIs, the formula by Rosnow and Rosenthal [62] was used. According to Cohen [63], d=0.2 can be considered a small effect, d=0.5 a medium and d=0.8 a large effect. A significance level of .05 (2-sided) was used for all analyses.

    To ascertain the number of participants attaining a reliable improvement in stress symptomatology, participants were coded as responders or nonresponders according to the Reliable Change Index [64]. Accordingly, response was attained when participants’ scores on the PSS-4 differed more than −2.17 points from baseline to T2 and T3, respectively. Furthermore, the numbers needed to treat (NNT) to achieve 1 additional treatment response were calculated. Negative effects of the intervention were evaluated by the number of participants with reliable symptom deterioration concerning perceived stress through the Reliable Change Index. Participants were defined as symptom-free when scoring more than 2 SDs below the mean at baseline for the full sample (PSS-4 ≤7.29).

    Subgroup Analysis

    To estimate the interventions’ efficacy in a clinical population, a subgroup analysis was conducted including only participants with a score of >17 on the CES-D short form at baseline and following the same procedure as the main analysis. A score of 18 has been shown to be a valid cut-off in indicating a high probability of clinical depression [65]. Participants were classified as responders if they showed reliable change in depressive symptoms according to the Reliable Change Index.

    Study Completer Analysis

    Completer analysis based on the sample of participants who provided data at all 3 assessment points was conducted additionally as a sensitivity analysis.

    Process Evaluation

    Descriptive statistics were used for process evaluation. To assess overall user satisfaction across various domains, item data provided by the CSQ-8 was examined individually. Acceptance of intervention modules was analyzed using the module feedback of the IG. Adherence was assessed by analyzing intervention completion rates tracked within the intervention platform. Finally, we analyzed the proportion of participants who accessed the diary app and requested automated short messages via SMS.


    Recruitment for the study started on May 9, 2016. The last follow-ups were completed on January 30, 2017.

    Participants

    The study flow can be found in Figure 1. Participants who were lost to follow-up at T2, T3, or both assessments did not differ significantly from participants who adhered to the protocol on any baseline characteristic (all P>.05). Table 1 summarizes detailed baseline characteristics of study participants. The majority (76.8%, 106/138) of the participants indicated that they had not consulted a physician, psychotherapist, or counselor for their health-related problems and may thus be considered first-time help-seekers. Descriptive data including all 3 assessment points for all outcomes is depicted in Table 2. Both study arms did not differ significantly (all P>.05) on any characteristic at baseline.

    Main Effectiveness Analysis

    Changes in Perceived Stress

    As hypothesized, the ANCOVA controlling for baseline scores revealed a significant effect for perceived stress at posttest (F1,147=19.70, P<.001) and at 3-month follow-up (F1,147=15.10, P<.001; see Table 3), with moderate to large effect sizes at both T2 (d=0.69; 95% CI 0.36-1.02) and T3 (d=0.57; 95% CI 0.24-0.89).

    Treatment Response for Perceived Stress

    Chi-squared tests revealed that significantly more participants in the IG (69%, 52/75) were classified as responders compared with the WCG (35%, 26/75) at posttest (χ21=18.1, P<.001), resulting in an NNT of 2.89 (95% CI 2.01-5.08). At 3-month follow-up, 55 of 75 participants in the IG (73%) and 33 of 75 in the WCG (44%) were coded as responders (χ21=13.3, P<.001) which equals an NNT of 3.41 (95% CI 2.25-7.00).

    Symptom-Free Status for Perceived Stress

    Symptom-free status was achieved by significantly (χ21=6.7, P=.01) more participants in the IG ( 44%, 33/75) compared with the WCG (24%, 18/75) at T2, and at T3 (IG: 53%, 40/75; WCG: 35%, 26/75) with χ21=5.3 (P=.02), resulting in an NNT of 5 (T2; 95% CI 2.87-19.31) and 5.36 (T3; 95% CI 2.92-32.66), respectively.

    Symptom Deterioration for Perceived Stress

    Only a small proportion of participants experienced symptom deterioration. Fewer participants’ stress symptomatology deteriorated in the IG (0%, 0/75) compared with the WCG, where 7 of 75 (9%) participants’ symptoms deteriorated (χ21=7.3, P<.001; NNT=10.58, 95% CI 6.19-44.18) at T2. Symptom deterioration did not differ at T3, with 1 case of 75 participants (1%) in the IG and 3 of 75 (4%) in the WCG (χ21=1.0, P=.31).

    Table 1. Baseline characteristics.
    CharacteristicsAll participants (N=150)Intervention (N=75)Control (N=75)
    Sociodemographics



    		Age in years, mean (SD)24.1 (4.1)24.0 (4.6)24.2 (3.6)

    		Gender, female, n (%)112 (74.7)54 (72)58 (77)

    		In a relationship, n (%)79 (52.7)39 (52)40 (53)

    		Married, n (%)6 (4.0)4 (5)2 (3)
    Major



    		Business & Economics, n (%)33 (22.0)16 (21)17 (23)

    		Computer Science & Engineering, n (%)13 (8.7)9 (12)4 (5)

    		Education, n (%)17 (11.3)7 (9)10 (13)

    		Humanities, n (%)12 (8.0)5 (8)7 (9)

    		Law, n (%)6 (3.3)2 (3)4 (5)

    		Medicine, n (%)15 (10.0)7 (9)8 (11)

    		Natural Sciences, n (%)20 (13.3)11 (15)9 (12)

    		Social Sciences, n (%)34 (22.7)18 (24)16 (21)

    		Number of semesters (previous studies included), mean (SD)6.7 (3.6)6.4 (3)7.07 (3.9)
    Type of tertiary education facility, n (%)



    		College119 (79.3)56 (74)63 (84)

    		University of Applied Sciences31 (20.6)19 (25)12 (16)
    Housing situation, n (%)



    		Alone31 (20.7)18 (24)13 (17)

    		Flat share95 (63.3)48 (64)47 (63)

    		With parents24 (16.0)9 (12)15 (20)
    Main source of funding, n (%)



    		Parents64 (42.7)33 (44)31 (41)

    		Job48 (32.02)25 (33)23 (31)

    		Loan34 (22.7)15 (20)19 (25)

    		Partner2 (1.3)1 (1)1 (1)

    		Scholarship2 (1.3)1 (1)1 (1)

    Secondary Outcome Analysis

    Table 3 summarizes the results of the ITT analyses for the secondary outcomes. ANCOVAs revealed significant effects (P<.05) in favor of the IG for the majority of outcomes at both assessment points, with effect sizes ranging from d=0.33 (95% CI 0.01-0.65) for productivity (T2) to d=0.82 (95% CI 0.49-1.15) for emotional exhaustion (T2). No statistically significant effect was found for perfectionism (F1,147=0.38, P=.53) at T2, but at T3 (P<.001). Resilience (T2: F1,147=1.69, P=.17; T3: F1,147=2.94, P=.08), self-compassion (T2: F1,147=2.97, P=.09; T3: F1,147=1.46, P=.23), and self-esteem (T2: F1,147=0.15, P=.70; T3: F1,147=1.36, P=.25) did not differ significantly between both study arms at both assessment points.

    Subgroup Analysis

    More than three-fourths of the participants (77.3%; IG: 58/75; WCG: 58/75) showed symptoms above the cut-off for clinically relevant symptoms of depression at baseline. Between-group effect sizes for depression in this subgroup were moderate to large, both for T2 (d=0.67, 95% CI 0.34-1.00) and T3 (d=0.73, 95% CI 0.40-1.06). Treatment response was achieved by 36 (62%; T2) and 33 of 58 participants (57%, T3) in the IG compared with 14 of 58 participants (24%; T2 and T3) in the WCG, resulting in an NNT to achieve one additional treatment response in the IG compared with the WCG of 2.64 (95% CI 1.83-4.70) for T2 (χ21=17.0, P<.001) and 3.05 for T3 (95% CI 2.02-6.28, χ21=12.9, P<.001).

    Table 2. Means and SDs of the intervention group (intervention) and waitlist control group (control) for the intention-to-treat-sample at baseline, posttest (7 weeks), and 3-month follow-up.
    Outcome and assessment pointIntervention (N=75)Control (N=75)


    		Mean (SD)Mean (SD)
    Primary outcome


    		Perceived stress (low to high 0-16)



    		Baseline11.13 (1.93)11.03 (1.87)


    		7 weeks7.43 (2.93)9.49 (3.06)


    		3 months6.96 (2.73)8.66 (3.26)
    Mental health


    		Depression (0-45)



    		Baseline24.31 (9.06)23.97 (8.63)


    		7 weeks15.88 (8.85)21.47 (8.96)


    		3 months16.79 (8.72)21.92 (9.53)

    		Anxiety (6-24)



    		Baseline16.05 (3.37)15.77 (4.22)


    		7 weeks13.37 (3.51)16.03 (3.48)


    		3 months13.33 (3.59)15.50 (4.10)

    		Well-being (0-30)a



    		Baseline8.01 (4.34)8.81 (3.69)


    		7 weeks11.93 (5.03)9.36 (4.35)


    		3 months12.62 (5.34)10.57 (4.81)

    		Emotional exhaustion (5-30)



    		Baseline21.63 (4.49)22.27 (4.31)


    		7 weeks18.43 (5.64)22.36 (3.77)


    		3 months20.04 (5.08)22.30 (4.45)
    Risk and protective factors



    		Dysfunctional perfectionism (8-56)



    		Baseline44.29 (7.90)43.89 (7.50)


    		7 weeks43.02 (7.22)43.45 (7.34)


    		3 months41.05 (5.94)44.33 (6.67)

    		Resilience (0-8)a



    		Baseline4.80 (1.72)4.79 (1.87)


    		7 weeks5.38 (1.85)5.05 (1.97)


    		3 months5.56 (1.36)5.17 (1.61)

    		Self-compassion (12-60)a



    		Baseline33.95 (3.47)34.54 (3.23)


    		7 weeks34.95 (5.67)34.16 (3.72)


    		3 months35.25 (3.26)34.78 (3.96)

    		Self-esteem (10-40)a



    		Baseline29.25 (2.58)29.20 (2.78)


    		7 weeks29.14 (3.61)28.93 (2.61)


    		3 months30.10 (3.03)29.54 (2.68)
    College-related outcomes



    		Academic work impairment (10-50)



    		Baseline28.29 (5.34)27.88 (5.39)


    		7 weeks25.54 (5.83)27.55 (6.13)


    		3 months24.74 (5.06)27.44 (6.22)

    		Academic productivity (percent)a



    		Baseline52.79 (27.04)54.30 (23.03)


    		7 weeks60.36 (24.12)52.36 (24.16)


    		3 months67.76 (17.27)58.21 (23.62)

    		Academic self-efficacy (7-28)a



    		Baseline17.04 (4.46)16.34 (4.04)


    		7 weeks18.35 (4.03)16.43 (4.12)


    		3 months18.60 (3.86)16.37 (4.12)

    		Academic worrying (0-40)



    		Baseline22.35 (6.63)22.01 (6.01)


    		7 weeks18.29 (6.16)21.71 (5.94)


    		3 months17.82 (6.97)21.14 (6.18)
    Treatment expectancies (0-100)


    		Baseline62.34 (13.51)62.44 (16.03)

    aHigher scores indicate better outcomes.

    Completer Analysis

    The results of the completer analyses were similar to the ITT analyses, with moderate to large between-group effect sizes for the primary outcome at T2 (IG: mean=6.72, SD 2.86; WCG: mean=9.32, SD 3.16; F1,88=18.60, P<.001; d=0.85, 95% CI 0.44-1.27) and T3 (IG: mean=6.41, SD 2.84; WCG: mean=8.65, SD 3.43; F1,88=13.41, P<.001; d=0.69, 95% CI 0.29-1.10). In contrast to the main analysis, however, resilience had increased significantly in the IG compared with the WCG at T2 (F1,88=8.56, P=.004; d=0.46, 95% CI 0.06-0.86).

    Process Evaluation

    Adherence to the Intervention

    On average, participants in the IG completed 5.05 modules (SD 2.78), which equals 72.1% of the intervention. Participants completed optional add-on modules in the majority (82.1%) of sessions in which they were available. Most participants completed rumination & worrying (59%, 44/75), whereas only 8 of the 75 participants completed social support (11%). In all, 46 of the 75 participants in the IG (61%) downloaded and logged into the diary app at least once. Activation of the automated SMS messages was requested by 4 of 75 participants in the IG (5%) during the study.

    Client Satisfaction

    Overall client satisfaction with the intervention was high (see Table 4).

    Perceived Usefulness, Difficulty, and Duration of Sessions

    Most participants described the 8 treatment modules as useful and not overly complex (see Multimedia Appendix 4). Reported session duration was high, with participants having spent the most time on module 6 (Self-compassion; 28% spending more than 1 hour 30 min, 10/36).

    Table 3. Results for the intention-to-treat sample for analyses of covariance for between-group effects, effect sizes (Cohen's d) for primary and secondary outcomes at posttest (7 weeks; T2) and 3-month follow-up (T3).
    Outcome and assessment pointEffect sizeANCOVAa


    		Cohen\'s d95% CIF1147P value
    Primary outcome





    		Perceived stress





    		7 weeks0.690.36 to 1.0219.70<.001


    		3 months0.570.24 to 0.8915.10<.001
    Mental health





    		Depression





    		7 weeks0.630.30 to 0.9622.31<.001


    		3 months0.560.24 to 0.8916.62<.001

    		Anxiety





    		7 weeks0.760.43 to 1.0928.20<.001


    		3 months0.560.24 to 0.8914.68<.001

    		Well-being





    		7 weeks0.550.22 to 0.8721.06<.001


    		3 months0.400.08 to 0.7312.14.001

    		Emotional exhaustion





    		7 weeks0.820.49 to 1.1530.67<.001


    		3 months0.590.26 to 0.928.93.003
    Risk and protective factors





    		Dysfunctional perfectionism





    		7 weeks0.06−0.26 to 0.380.38.54


    		3 months0.520.19 to 0.8415.79<.001

    		Resilience





    		7 weeks0.17−0.15 to 0.491.69.17


    		3 months0.26−0.06 to 0.582.94.08

    		Self-compassion





    		7 weeks0.17−0.16 to 0.492.97.09


    		3 months0.13−0.19 to 0.451.46.23

    		Self-esteem





    		7 weeks0.07−0.25 to 0.390.15.70


    		3 months0.19−0.13 to 0.511.36.25
    College-related outcomes





    		Academic work impairment





    		7 weeks0.340.01 to 0.666.57.01


    		3 months0.480.15 to 0.8010.57.001

    		Academic productivity





    		7 weeks0.330.01 to 0.654.29.04


    		3 months0.460.14 to 0.799.68.002

    		Academic self-efficacy





    		7 weeks0.490.16 to 0.8112.74<.001


    		3 months0.560.23 to 0.8817.98<.001

    		Academic worrying





    		7 weeks0.560.24 to 0.8927.41<.001


    		3 months0.500.18 to 0.8316.04<.001

    aANCOVA: analysis of covariance.

    Table 4. Clients’ satisfaction with the intervention (T2; Intervention Group only).
    Ratingsn (%)
    Quality of the training rated as excellent or good59 (92)
    Indication that the training was the kind of intervention they wanted to receive (generally or definitely)51 (80)
    Indication that the own needs were almost all or mostly met47 (73)
    Inclination to recommend the training to a friend in need of similar help58 (91)
    Satisfaction with the amount of help received (mostly or very satisfied)51 (80)
    Indication that the training has helped (a great deal) to deal more effectively with problems53 (83)
    Satisfaction with the training in a general, overall sense (mostly or very satisfied)55 (86)
    Inclination to use the training again if in need for help49 (77)

    Principal Findings

    Results of this study indicate moderate to large intergroup effects for the reduction of perceived stress and other relevant health- and college-related outcomes, as well as substantial effects in individuals with clinically relevant symptoms of depression, which were highly prevalent in our recruited sample. No significant effects were found for self-compassion, perfectionism (T2), resilience, and self-esteem.

    The benefits of this intervention were larger than those found in previous trials evaluating internet-based stress interventions in college students [19,20], albeit somewhat smaller than the reported overall effect of technology-delivered skill training interventions on perceived stress [18], and comparable to internet-based stress interventions in general, as reported in a recent meta-analysis, with a pooled standardized mean difference of d=0.64 (95% CI 0.50-0.79; perceived stress) in guided internet- and mobile-based interventions [66]. The study further contributes to current literature by showing that targeting perceived stress in students does not only result in better mental health–related outcomes and well-being but can also have a substantial beneficial impact on college-related outcomes which, to the best of our knowledge, have not been investigated so far.

    Participants’ adherence to the intervention was satisfying, and the intervention was well accepted among the large majority of students. Participant feedback on the length of specific modules, however, suggests that participants may have spent more time than anticipated on some of the modules. This may indicate that some modules could be shortened to attempt to further improve adherence. Whether shortening might, in fact, result in higher adherence to the intervention, however, is not fully clear. Earlier research has reported higher adherence rates for shorter interventions, albeit focusing on the number of modules used [67]. Whether this effect also holds true for the length of specific modules remains unknown. Shortening some of the modules might potentially optimize adherence but may also compromise the intervention’s overall efficacy due to less potentially beneficial information or techniques being conveyed and trained. It has been argued that various ways in which participants prioritize provided content may lead to positive outcomes, and the ability to progress through interventions at one’s own pace might represent a key asset of internet-delivered treatment [68]. Qualitative interviews conducted with participants of this intervention suggest that the elective mini-modules for various student-relevant topics were very well accepted [69]. Providing larger amounts of content in a flexible way, allowing participants to tailor the intervention to their specific needs, could, therefore, be a promising approach to optimize intervention usage patterns [70]. However, research is warranted to test whether this might further increase adherence.

    Limitations

    This study has some limitations. First, women were overrepresented in the study sample, as frequently seen in preventive interventions. Second, study dropout in the IG at 3-month follow-up was relatively high and larger in the IG. Albeit being a common limitation in clinical trial research [71], and with differential dropout rates having been reported before for internet-based stress intervention trials in tertiary education students [72,73], this restricts the generalizability of our findings on long-term effects. Attrition analysis, however, did not result in any significant baseline differences between dropout and nondropout cases, which may be an indicator that results were not overly biased due to unequal dropout [74]. Third, because of ethical reasons, participants had full access to treatment-as-usual. Thus, we cannot rule out potential cointervention effects due to utilization of health services. Finally, because of feasibility and ethical reasons, such as not denying one half of the sample access to the intervention they sought after, participants in the IG of this study were compared with a WCG to assess effects of the training. The influence of treatment and change expectancies have been discussed as an artifact in clinical evaluation trials using WCGs because they potentially discourage participants with delayed access to treatment to initiate health-related behavior changes, and thus lead to accentuate effects [75].

    Most college students with depression do not seek treatment through conventional health care channels [76], and attitudinal barriers, such as fear of stigmatization, have been shown to have a large impact on treatment utilization [77]. As our findings indicate that (1) a large number of students in this sample did not use conventional treatment options before, (2) the majority of students who were willing to use this intervention reported clinically significant symptoms of depression, and (3) among this group, the treatment response was favorable; future studies should explore whether internet-delivered stress interventions, labeled as providing “support for coping with academic stress,” might potentially attract students with symptoms of depression who would not use formal mental health treatment and whether they can reduce the incidence of depressive disorders [8]. Future studies should thus investigate the utility of internet and mobile-based interventions in affected students, that is, with symptoms of major depression, as an indicated preventative or early intervention approach to narrow the treatment gap and improve academic functioning [4].

    Conclusions

    In conclusion, internet- and mobile-based interventions could be an acceptable, effective, and potentially cost-effective approach to reduce the negative consequences associated with college-related stress.

    Acknowledgments

    The study was partly funded by BARMER, a major health care insurance company in Germany.

    Conflicts of Interest

    DDE reports to have received consultancy fees or served in the scientific advisory board from several companies such as Minddistrict, Lantern, Schön Kliniken, and German health insurance companies (BARMER, Techniker Krankenkasse). DDE and MB are also stakeholders of the Institute for health trainings online (GET.ON), which aims to implement scientific findings related to digital health interventions into routine care. HB reports to have received consultancy fees and fees for lectures or workshops from chambers of psychotherapists and training institutes for psychotherapists. In the past 3 years, RCK received support for his epidemiological studies from Sanofi Aventis, was a consultant for Johnson & Johnson Wellness and Prevention, Sage Pharmaceuticals, Shire, Takeda, and served on an advisory board for the Johnson & Johnson Services Inc, and Lake Nona Life Project. RCK is a co-owner of DataStat, Inc, a market research firm that carries out health care research.

    ‎

    Multimedia Appendix 1

    Trial registration.

    PDF File (Adobe PDF File), 75KB
    ‎

    Multimedia Appendix 2

    Intervention modules.

    PDF File (Adobe PDF File), 59KB
    ‎

    Multimedia Appendix 3

    CONSORT-EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 930KB
    ‎

    Multimedia Appendix 4

    Participants’ perceived usefulness, complexity and duration for each intervention module.

    PDF File (Adobe PDF File), 62KB
    1. Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, et al. Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychol Med 2016 Oct;46(14):2955-2970 [FREE Full text] [CrossRef] [Medline]
    2. Blanco C, Okuda M, Wright C, Hasin DS, Grant BF, Liu S, et al. Mental health of college students and their non-college-attending peers: results from the National Epidemiologic Study on Alcohol and Related Conditions. Arch Gen Psychiatry 2008 Dec;65(12):1429-1437 [FREE Full text] [CrossRef] [Medline]
    3. World Health Organization. The Global Burden of Disease: 2004 update. Geneva: WHO; 2008.
    4. Bruffaerts R, Mortier P, Kiekens G, Auerbach RP, Cuijpers P, Demyttenaere K, et al. Mental health problems in college freshmen: prevalence and academic functioning. J Affect Disord 2018 Jan 01;225:97-103. [CrossRef] [Medline]
    5. Breslau J, Lane M, Sampson N, Kessler RC. Mental disorders and subsequent educational attainment in a US national sample. J Psychiatr Res 2008 Jul;42(9):708-716 [FREE Full text] [CrossRef] [Medline]
    6. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One 2015;10(3):e0119895 [FREE Full text] [CrossRef] [Medline]
    7. Eisenberg D, Downs MF, Golberstein E, Zivin K. Stigma and help seeking for mental health among college students. Med Care Res Rev 2009 Oct;66(5):522-541. [CrossRef] [Medline]
    8. Ebert DD, Cuijpers P, Muñoz RF, Baumeister H. Prevention of mental health disorders using Internet- and mobile-based interventions: a narrative review and recommendations for future research. Front Psychiatry 2017;8:116 [FREE Full text] [CrossRef] [Medline]
    9. Ebert DD, Gollwitzer M, Riper H, Cuijpers P, Baumeister H, Berking M. For whom does it work? Moderators of outcome on the effect of a transdiagnostic internet-based maintenance treatment after inpatient psychotherapy: randomized controlled trial. J Med Internet Res 2013 Oct 10;15(10):e191 [FREE Full text] [CrossRef] [Medline]
    10. Buntrock C, Berking M, Smit F, Lehr D, Nobis S, Riper H, et al. Preventing depression in adults with subthreshold depression: health-economic evaluation alongside a pragmatic randomized controlled trial of a web-based intervention. J Med Internet Res 2017 Jan 04;19(1):e5 [FREE Full text] [CrossRef] [Medline]
    11. Chan JK, Farrer LM, Gulliver A, Bennett K, Griffiths KM. University students' views on the perceived benefits and drawbacks of seeking help for mental health problems on the Internet: a qualitative study. JMIR Hum Factors 2016;3(1):e3 [FREE Full text] [CrossRef] [Medline]
    12. Batterham PJ, Calear AL. Preferences for Internet-based mental health interventions in an adult online sample: findings from an online community survey. JMIR Ment Health 2017 Jun 30;4(2):e26 [FREE Full text] [CrossRef] [Medline]
    13. Königbauer J, Letsch J, Doebler P, Ebert D, Baumeister H. Internet- and mobile-based depression interventions for people with diagnosed depression: a systematic review and meta-analysis. J Affect Disord 2017 Dec 01;223:28-40. [CrossRef] [Medline]
    14. Sander L, Rausch L, Baumeister H. Effectiveness of Internet-based interventions for the prevention of mental disorders: a systematic review and meta-analysis. JMIR Ment Health 2016;3(3):e38 [FREE Full text] [CrossRef] [Medline]
    15. Välimäki M, Anttila K, Anttila M, Lahti M. Web-based interventions supporting adolescents and young people with depressive symptoms: systematic review and meta-analysis. JMIR Mhealth Uhealth 2017 Dec 08;5(12):e180 [FREE Full text] [CrossRef] [Medline]
    16. Seo DC, Niu J. Evaluation of Internet-based interventions on waist circumference reduction: a meta-analysis. J Med Internet Res 2015 Jul 21;17(7):e181 [FREE Full text] [CrossRef] [Medline]
    17. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res 2014;16(5):e130 [FREE Full text] [CrossRef] [Medline]
    18. Conley CS, Durlak JA, Shapiro JB, Kirsch AC, Zahniser E. A meta-analysis of the impact of universal and indicated preventive technology-delivered interventions for higher education students. Prev Sci 2016 Dec;17(6):659-678. [CrossRef] [Medline]
    19. Hintz S, Frazier PA, Meredith L. Evaluating an online stress management intervention for college students. J Couns Psychol 2015 Apr;62(2):137-147. [CrossRef] [Medline]
    20. Chiauzzi E, Brevard J, Thum C, Decembrele S, Lord S. MyStudentBody-Stress: an online stress management intervention for college students. J Health Commun 2008 Sep;13(6):555-572. [CrossRef] [Medline]
    21. Ryan ML, Shochet IM, Stallman HM. Universal online interventions might engage psychologically distressed university students who are unlikely to seek formal help. Adv Ment Health 2010;9(1):73-83. [CrossRef]
    22. Apolinário-Hagen J, Vehreschild V, Alkoudmani RM. Current views and perspectives on e-mental health: an exploratory survey study for understanding public attitudes toward Internet-based psychotherapy in Germany. JMIR Ment Health 2017 Feb 23;4(1):e8 [FREE Full text] [CrossRef] [Medline]
    23. Auerbach RP, Benjet C, Bruffaerts R, Cuijpers P, Ebert DD, Kessler RC. Harvard Medical School Department of Health Care Policy. 2017. The WHO World Mental Health International College Student Project   URL: https://www.hcp.med.harvard.edu/wmh/college_student_survey.php [WebCite Cache]
    24. Deutsches Register Klinischer Studien [German Clinical Trials Register]. DRKS. 2016 Dec 05. Online based self-help stress management program for students with feedback on demand   URL: http://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00010212 [accessed 2018-03-05] [WebCite Cache]
    25. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983 Dec;24(4):385-396. [Medline]
    26. Ebert DD, Lehr D, Smit F, Zarski AC, Riper H, Heber E, et al. Efficacy and cost-effectiveness of minimal guided and unguided internet-based mobile supported stress-management in employees with occupational stress: a three-armed randomised controlled trial. BMC Public Health 2014;14:807 [FREE Full text] [CrossRef] [Medline]
    27. Lazarus RS, Folkman S. Stress, Appraisal, and Coping. New York: Springer; 1984.
    28. Berking M, Ebert D, Cuijpers P, Hofmann SG. Emotion regulation skills training enhances the efficacy of inpatient cognitive behavioral therapy for major depressive disorder: a randomized controlled trial. Psychother Psychosom 2013;82(4):234-245. [CrossRef] [Medline]
    29. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    30. Schueller SM, Tomasino KN, Mohr DC. Integrating human support into behavioral intervention technologies: the efficiency model of support. Clin Psychol Sci Pract 2016 Nov 17;24(1):27-45. [CrossRef]
    31. Zarski AC, Lehr D, Berking M, Riper H, Cuijpers P, Ebert DD. Adherence to Internet-based mobile-supported stress management: a pooled analysis of individual participant data from three randomized controlled trials. J Med Internet Res 2016;18(6):e146 [FREE Full text] [CrossRef] [Medline]
    32. Nielsen MG, Ørnbøl E, Vestergaard M, Bech P, Larsen FB, Lasgaard M, et al. The construct validity of the Perceived Stress Scale. J Psychosom Res 2016 May;84:22-30. [CrossRef] [Medline]
    33. Denovan A, Dagnall N, Dhingra K, Grogan S. Evaluating the Perceived Stress Scale among UK university students: implications for stress measurement and management. Stud High Educ 2017 Jun 22:1-14 [FREE Full text] [CrossRef]
    34. Reis D, Lehr D, Heber E, Ebert DD. The German version of the Perceived Stress Scale (PSS-10): evaluation of dimensionality, validity, and measurement invariance with exploratory and confirmatory bifactor modeling. Assessment 2017 Jun 01:1073191117715731. [CrossRef] [Medline]
    35. Lee E. Review of the psychometric evidence of the Perceived Stress Scale. Asian Nurs Res (Korean Soc Nurs Sci) 2012 Dec;6(4):121-127 [FREE Full text] [CrossRef] [Medline]
    36. Hewitt PL, Flett GL, Mosher SW. The Perceived Stress Scale: factor structure and relation to depression symptoms in a psychiatric sample. J Psychopathol Behav Assess 1992 Sep;14(3):247-257 [FREE Full text] [CrossRef]
    37. Pbert L, Doerfler LA, DeCosimo D. An evaluation of the perceived stress scale in two clinical populations. J Psychopathol Behav Assess 1992 Dec;14(4):363-375 [FREE Full text] [CrossRef]
    38. Klein EM, Brähler E, Dreier M, Reinecke L, Müller KW, Schmutzer G, et al. The German version of the Perceived Stress Scale - psychometric characteristics in a representative German community sample. BMC Psychiatry 2016 Dec 23;16:159 [FREE Full text] [CrossRef] [Medline]
    39. Warttig SL, Forshaw MJ, South J, White AK. New, normative, English-sample data for the Short Form Perceived Stress Scale (PSS-4). J Health Psychol 2013 Dec;18(12):1617-1628. [CrossRef] [Medline]
    40. Mitchell AM, Crane PA, Kim Y. Perceived stress in survivors of suicide: psychometric properties of the Perceived Stress Scale. Res Nurs Health 2008 Dec;31(6):576-585. [CrossRef] [Medline]
    41. Hautzinger M, Bailer M, Hofmeister D, Keller F. Center for Epidemiological Studies Depression Scale (CES-D; Radloff, L. S., 1977) - German adaption. Psychiatr Prax 2012;39(6):302-304 [FREE Full text] [CrossRef]
    42. Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). Br J Clin Psychol 1992 Sep;31 ( Pt 3):301-306. [Medline]
    43. Andrews B, Wilding JM. The relation of depression and anxiety to life-stress and achievement in students. Br J Psychol 2004 Nov;95(Pt 4):509-521. [CrossRef] [Medline]
    44. Brähler E, Mühlan H, Albani C, Schmidt S. Teststatistische Prüfung und Normierung der deutschen Versionen des EUROHIS-QOL Lebensqualität-Index und des WHO-5 Wohlbefindens-Index [Test statistic testing and standardization of the German versions of the EUROHIS-QOL quality of life index and the WHO-5 well-being index]. Diagn 2007 Apr;53(2):83-96 [FREE Full text] [CrossRef]
    45. Gumz A, Erices R, Brähler E, Zenger M. [Factorial structure and psychometric criteria of the German translation of the Maslach Burnout Inventory--Student Version by Schaufeli et al. (MBI-SS)]. Psychother Psychosom Med Psychol 2013 Feb;63(2):77-84. [CrossRef] [Medline]
    46. Smith MM, Saklofske DH, Yan G, Sherry SB. Does perfectionism predict depression, anxiety, stress, and life satisfaction after controlling for neuroticism? J Individ Differ 2017 Apr;38(2):63-70 [FREE Full text] [CrossRef]
    47. Slaney RB, Rice KG, Mobley M, Trippi J, Ashby JS. The revised almost perfect scale. Meas Eval Couns Dev 2001;34(3):130-145.
    48. Galatzer-Levy IR, Burton CL, Bonanno GA. Coping flexibility, potentially traumatic life events, and resilience: a prospective study of college student adjustment. J Soc Clin Psychol 2012 Jun;31(6):542-567 [FREE Full text] [CrossRef]
    49. Connor KM, Davidson JR. Development of a new resilience scale: the Connor-Davidson Resilience Scale (CD-RISC). Depress Anxiety 2003;18(2):76-82. [CrossRef] [Medline]
    50. Neff KD, Rude SS, Kirkpatrick KL. An examination of self-compassion in relation to positive psychological functioning and personality traits. J Res Pers 2007 Aug;41(4):908-916 [FREE Full text] [CrossRef]
    51. Trompetter HR, de Kleine E, Bohlmeijer ET. Why does positive mental health buffer against psychopathology? An exploratory study on self-compassion as a resilience mechanism and adaptive emotion regulation strategy. Cognit Ther Res 2017;41(3):459-468 [FREE Full text] [CrossRef] [Medline]
    52. Hupfeld J, Ruffieux N. Validierung einer deutschen Version der Self-Compassion Scale (SCS-D) [Validation of a German version of the Self-Compassion Scale (SCS-D)]. Z Klin Psychol Psychother 2011 Apr;40(2):115-123 [FREE Full text] [CrossRef]
    53. Thoits PA. Self, identity, stress, and mental health. In: Aneshensel CS, Phelan JC, Bierman A, editors. Handbook of the Sociology of Mental Health. Dordrecht: Springer; 2013:357-377.
    54. Ferring D, Filipp SH. Messung des selbstwertgefühls: befunde zu reliabilität, validität und stabilität der Rosenberg-Skala [Measurement of self-esteem: findings on reliability, validity, and stability of the Rosenberg Scale]. Diagn 1996;42(3):284-292.
    55. Matsushita M, Adachi H, Arakida M, Namura I, Takahashi Y, Miyata M, et al. Presenteeism in college students: reliability and validity of the Presenteeism Scale for students. Qual Life Res 2011 Apr;20(3):439-446. [CrossRef] [Medline]
    56. Jerusalem M, Schwarzer R. Selbstwirksamkeit. Skalen zur Befindlichkeit und Persönlichkeit [Self-Efficacy. Scales for Mental State and Personality]. Berlin: Institute for Psychology, Free University Berlin; 1986.
    57. Wolitzky-Taylor K. University of Texas at Austin. 2009. Randomized Clinical Trial Investigating the Efficacy of Self- Administered Interventions for Reducing Pathological Academic Worry   URL: https://repositories.lib.utexas.edu/bitstream/handle/2152/7513/wolitzky-taylork81714.pdf;sequence=2 [WebCite Cache]
    58. Boß L, Lehr D, Reis D, Vis C, Riper H, Berking M, et al. Reliability and validity of assessing user satisfaction with web-based health interventions. J Med Internet Res 2016 Aug 31;18(8):e234 [FREE Full text] [CrossRef] [Medline]
    59. Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 2000 Jun;31(2):73-86. [Medline]
    60. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001 Apr 17;134(8):663-694 [FREE Full text] [CrossRef]
    61. Leonhart R. [Estimating effect sizes in clinical trials]. Rehabilitation (Stuttg) 2004 Aug;43(4):241-246. [CrossRef] [Medline]
    62. Rosnow RL, Rosenthal R. Assessing the effect size of outcome research. In: Nezu AM, Nezu CM, editors. Evidence-Based Outcome Research: A Practical Guide to Conducting Randomized Controlled Trials for Psychosocial Interventions. New York, NY, US: Oxford University Press; 2008.
    63. Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd edition. Mahwah, NJ: Lawrence Erlbaum Associates; 1988.
    64. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline]
    65. Lehr D, Hillert A, Schmitz E, Sosnowsky N. Screening depressiver Störungen mittels Allgemeiner Depressions-Skala (ADS-K) und State-Trait Depressions Scales (STDS-T) [Screening of depressive disorders using the General Depression Scale (ADS-K) and State-Trait Depression Scales (STDS-T)]. Diagn 2008 Apr;54(2):61-70 [FREE Full text] [CrossRef]
    66. Heber E, Ebert DD, Lehr D, Cuijpers P, Berking M, Nobis S, et al. The benefit of web- and computer-based interventions for stress: a systematic review and meta-analysis. J Med Internet Res 2017 Feb 17;19(2):e32 [FREE Full text] [CrossRef] [Medline]
    67. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
    68. Richards D, Timulak L, O'Brien E, Hayes C, Vigano N, Sharry J, et al. A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression. Behav Res Ther 2015 Dec;75:20-31 [FREE Full text] [CrossRef] [Medline]
    69. Fleischmann RJ, Harrer M, Zarski AC, Baumeister H, Lehr D, Ebert DD. Patients' experiences in a guided Internet- and app-based stress intervention for college students: a qualitative study. Internet Interv 2018:A (forthcoming) [FREE Full text] [CrossRef]
    70. Farrer L, Gulliver A, Chan JK, Bennett K, Griffiths KM. A virtual mental health clinic for university students: a qualitative study of end-user service needs and priorities. JMIR Ment Health 2015;2(1):e2 [FREE Full text] [CrossRef] [Medline]
    71. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    72. Cavanagh K, Strauss C, Cicconi F, Griffiths N, Wyper A, Jones F. A randomised controlled trial of a brief online mindfulness-based intervention. Behav Res Ther 2013 Sep;51(9):573-578. [CrossRef] [Medline]
    73. Frazier P, Meredith L, Greer C, Paulsen JA, Howard K, Dietz LR, et al. Randomized controlled trial evaluating the effectiveness of a web-based stress management program among community college students. Anxiety Stress Coping 2015;28(5):576-586. [CrossRef] [Medline]
    74. Bell ML, Kenward MG, Fairclough DL, Horton NJ. Differential dropout and bias in randomised controlled trials: when it matters and when it may not. Br Med J 2013 Jan 21;346:e8668 [FREE Full text] [Medline]
    75. Mohr DC, Ho J, Hart TL, Baron KG, Berendsen M, Beckner V, et al. Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression. Transl Behav Med 2014 Dec;4(4):407-423 [FREE Full text] [CrossRef] [Medline]
    76. Hunt J, Eisenberg D. Mental health problems and help-seeking behavior among college students. J Adolesc Health 2010 Jan;46(1):3-10. [CrossRef] [Medline]
    77. Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med 2014 Apr;44(6):1303-1317 [FREE Full text] [CrossRef] [Medline]

    ‎
    ANCOVA: analysis of covariance
    CES-D: Center for Epidemiological Studies’ Depression Scale
    IG: intervention group
    ITT: intention-to-treat
    NNT: numbers needed to treat
    PSS-4: Perceived Stress Scale, 4-item version
    SMD: standardized mean difference
    SMS: short message service
    T1: baseline assessment
    T2: postassessment (7 weeks)
    T3: 3-month follow-up assessment
    TAU: treatment as usual
    WCG: waitlist control group

    Edited by G Eysenbach; submitted 30.10.17; peer-reviewed by P Frazier, K Baron, T Furukawa; comments to author 14.12.17; revised version received 07.02.18; accepted 12.02.18; published 23.04.18

    Copyright

    ©Mathias Harrer, Sophia Helen Adam, Rebecca Jessica Fleischmann, Harald Baumeister, Randy Auerbach, Ronny Bruffaerts, Pim Cuijpers, Ronald C Kessler, Matthias Berking, Dirk Lehr, David Daniel Ebert. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 23.04.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.