This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297) Anxiety and Stress Disorders (1053) Cost-Effectiveness and Economics (252) Occupational Health and Ergonomics/Prevention at the Workplace (238) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514)

Published on in Vol 26 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48481, first published .
A Universal Digital Stress Management Intervention for Employees: Randomized Controlled Trial with Health-Economic Evaluation

A Universal Digital Stress Management Intervention for Employees: Randomized Controlled Trial with Health-Economic Evaluation

A Universal Digital Stress Management Intervention for Employees: Randomized Controlled Trial with Health-Economic Evaluation

Authors of this article:

Johanna Freund1, 2 Author Orcid Image ;   Filip Smit3, 4, 5 Author Orcid Image ;   Dirk Lehr6 Author Orcid Image ;   Anna-Carlotta Zarski1, 2, 7 Author Orcid Image ;   Matthias Berking1 Author Orcid Image ;   Heleen Riper5, 8, 9 Author Orcid Image ;   Burkhardt Funk10 Author Orcid Image ;   David Daniel Ebert2 Author Orcid Image ;   Claudia Buntrock1, 11 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    2School of Medicine and Health, Technical University of Munich, Munich, Germany

    3Department of Mental Health and Prevention, Trimbos Institute, Utrecht, Netherlands

    4Department of Epidemiology and Biostatistics, Amsterdam University Medical Centers, Amsterdam, Netherlands

    5Department of Clinical, Neuro and Developmental Psychology, VU University, Amsterdam, Netherlands

    6Department of Health Psychology and Applied Biological Psychology, Leuphana University of Lueneburg, Lueneburg, Germany

    7Division of eHealth in Clinical Psychology, Department of Clinical Psychology, Philipps University of Marburg, Marburg, Germany

    8Department of Psychiatry, VU University, Amsterdam, Netherlands

    9Amsterdam Public Health Research Institute, VU University, Amsterdam, Netherlands

    10Institute of Information Systems, Leuphana University of Lueneburg, Lueneburg, Germany

    11Institute of Social Medicine and Health Systems Research, Otto von Guericke University Magdeburg, Magdeburg, Germany

    Corresponding Author:

    Dirk Lehr, PhD

    Department of Health Psychology and Applied Biological Psychology

    Leuphana University of Lueneburg

    Universitätsallee 1

    Lueneburg, 21335

    Germany

    Phone: 49 4131 677 2720

    Email: lehr@leuphana.de


    Background: Stress is highly prevalent and known to be a risk factor for a wide range of physical and mental disorders. The effectiveness of digital stress management interventions has been confirmed; however, research on its economic merits is still limited.

    Objective: This study aims to assess the cost-effectiveness, cost-utility, and cost-benefit of a universal digital stress management intervention for employees compared with a waitlist control condition within a time horizon of 6 months.

    Methods: Recruitment was directed at the German working population. A sample of 396 employees was randomly assigned to the intervention group (n=198) or the waitlist control condition (WLC) group (n=198). The digital stress management intervention included 7 sessions plus 1 booster session, which was offered without therapeutic guidance. Health service use, patient and family expenditures, and productivity losses were self-assessed and used for costing from a societal and an employer’s perspective. Costs were related to symptom-free status (PSS-10 [Perceived Stress Scale] score 2 SDs below the study population baseline mean) and quality-adjusted life years (QALYs) gained. The sampling error was handled using nonparametric bootstrapping.

    Results: From a societal perspective, the digital intervention was likely to be dominant compared with WLC, with a 56% probability of being cost-effective at a willingness-to-pay (WTP) of €0 per symptom-free person gained. At the same WTP threshold, the digital intervention had a probability of 55% being cost-effective per QALY gained relative to the WLC. This probability increased to 80% at a societal WTP of €20,000 per QALY gained. Taking the employer’s perspective, the digital intervention showed a probability of a positive return on investment of 78%.

    Conclusions: Digital preventive stress management for employees appears to be cost-effective societally and provides a favorable return on investment for employers.

    Trial Registration: German Clinical Trials Register DRKS00005699; https://drks.de/search/en/trial/DRKS00005699

    J Med Internet Res 2024;26:e48481

    doi:10.2196/48481

    Keywords

    economic evaluation; cost-effectiveness; cost-utility; cost-benefit; return-on-investment; employees; universal prevention; internet-based; stress management 


    In Europe, up to 27% of the working population suffers from stress [1]. Stress is often caused by work-related factors, including high perceived work demands, little work control, and little support from coworkers and supervisors [2]. Stress is linked to numerous diseases, including mental health problems and psychiatric diseases [3]. Besides the great burden of disease for the individual, stress is linked to formidable costs for employers as well as society as a whole. Due to stress and stress-related disorders, individuals suffer from impairment at work and lower productivity, are absent for more days from work, and use health services at a higher rate [4]. Lazarus’ transactional model of stress [5] delineates 2 distinct coping strategies. On the one hand, problem-focused coping involves actively influencing a stressful situation positively by using cognitive or behavioral efforts. On the other hand, emotion-focused coping primarily serves the purpose of managing challenging emotions, such as anger, disappointment, and sadness, in response to the specific situation.

    The efficacy of stress management interventions has been confirmed in numerous meta-analyses of randomized trials in the general population [6] and in occupational settings [7]. However, high levels of psychological stress among employees are omnipresent and remain largely untreated [8]. Easily accessible and highly scalable digital interventions independent of time and place represent a promising approach to lowering the threshold for use compared with face-to-face interventions [9]. A recent meta-analysis provides evidence that digital or internet-based stress management interventions (iSMIs) are effective in terms of stress reduction in adults with small-to-medium effect sizes at posttreatment (Cohen d=0.43, 95% CI 0.31-0.51) [10]. In the workplace, in particular, a universal prevention approach is especially desirable because more employees can be reached without previous screening, which might be costly [11]. Compared with selective prevention with a focus on groups of people at increased risk or indicated prevention with a focus on individuals with elevated symptoms, universal prevention aims to reach the entire population regardless of any risk status [12]. A scalable digital intervention for the universal prevention of stress in the working population has the potential for substantial reach. Furthermore, with a focus on the entire working population, individuals who do not want to disclose symptoms due to fear of stigmatization can also be reached [13]. Concerning the efficacy of a universal iSMI for employees, the findings from a pragmatic randomized controlled trial indicate significantly reduced perceived stress with medium-to-large effect sizes both at posttreatment (Cohen d=0.71, 95% CI 0.51-0.91) and at 6-month follow-up (Cohen d=0.61, 95% CI 0.41-0.81) in the iSMI group when compared with a waitlist control condition (WLC) [14].

    Yet, although the effectiveness of iSMI has been demonstrated, research on its economic merits is still limited. Evidence suggests that an indicated iSMI to proactively prevent the onset of stress in employees represents good value for money from both a societal and an employer’s perspective [15,16]. However, no study has yet evaluated the cost-effectiveness of a universal iSMI in the working population. This study, thus, aimed to evaluate the cost-effectiveness and cost-utility of a universal iSMI for stress in employees compared with a WLC from a societal perspective and the cost-benefit from the employer’s perspective over a time horizon of 6 months. The clinical effectiveness of this iSMI has already been established [14].


    Study Design

    We carried out the health-economic evaluation alongside a 2-arm randomized controlled trial (RCT), comparing the effects of a self-guided iSMI for stress prevention with a WLC. Detailed information about the study design can be found elsewhere [14]. We carried out and reported the health economic evaluation in accordance with the guidelines of the International Society for Pharmacoeconomics and Outcomes Research [17] and the Consolidated Health Economic Evaluation Reporting Standards statement [18] (more details in Multimedia Appendix 1).

    Recruitment

    Recruitment was carried out as part of the occupational health program of a large German health insurance company (BARMER) in a way similar to the intended implementation of the intervention in routine practice in the future. The goal was to recruit individuals from the general working population, not just individuals insured by this health insurance company. This was done primarily through reports in the member magazine of the health insurance company and the insurance company’s occupational health and safety management staff, who informed the human resources departments of collaborating companies about their employees’ possibility to participate in the study. Interested individuals who completed a web-based screening questionnaire and met eligibility criteria were asked to fill out the informed consent form. To best reflect the routine conditions, the inclusion and exclusion criteria were reduced to a minimum. Individuals who were included (1) were 18 years and older, (2) were currently employed, and (3) had internet access and a valid email address. Exclusion criteria only included (1) a risk of suicide as indicated by a score of greater than 1 on the Beck Depression Inventory suicide item [14] or (2) any diagnosis with psychosis or dissociative symptoms (self-reported). If participants were excluded from the study, they were provided with information about alternative treatment options available in routine care.

    Overall, 396 participants were included in the study, with 198 randomized to either the iSMI or WLC condition. All participants completed the assessment at baseline, while 313 participants (79%) provided data at 6-month follow-up. Study dropout was not statistically significantly associated with any sociodemographic characteristics or initial perceived stress level. The average participant was 41.76 (SD 10.09) years old, female (302/396, 76%), highly educated (285/396, 72%), and employed full-time (296/396, 75%) with a working experience of 17.58 (SD 10.36) years. Almost half of the participants worked in a management position (169/396, 43%).

    Randomization and Masking

    Overview

    Study participants were randomly assigned to the invention or control group in a 1:1 ratio by an independent researcher not otherwise involved in the study. Randomization took place using a computer-based random numbers table (Randlist) to ensure equal sample sizes for both conditions. Detailed information about the randomization procedure can be found elsewhere [14]. During the randomization process, the assignment was hidden from the participants as well as the researchers involved in recruitment and study administration. After randomization, participants were not blind to the study conditions due to the nature of the intervention.

    Control Condition

    In both study conditions, the participants had full access to treatment as usual (TAU). We did not interfere in TAU. Rather, we tried to maintain a naturalistic TAU state in order to represent routine care as much as possible.

    Intervention

    Participants in the intervention group (IG) received the iSMI GET.ON Stress. This iSMI entails 7 regular modules and 1 additional booster session for reviewing the most relevant content. Each module consists of psychoeducation, strategies for problem-solving, emotion regulation techniques, and plans for the future. It was recommended to complete 1-2 modules per week. The iSMI is based on the transactional model of stress [14]. The training includes interactive education, exercises, testimonials, and audio and video files. The content of the intervention is tailored to the individual needs and interests of the participants since several choices were made available through different answer options. In order to integrate the new knowledge sustainably into everyday life, homework, behavior planning, and an online diary are parts of the intervention. There was no therapeutic guidance provided. However, the participants had the opportunity to receive automatic text messages on their mobile phones. Participants could choose between light support with 1 text message every other day or intensive support with 2 or 3 text messages per day. The text messages contained very short exercises that should be carried out in everyday life to support the transfer from training to real life. More details about the intervention can be found elsewhere [14]. In this study, IG participants completed, on average, 5.23 (SD 2.74) sessions. Out of 198 participants, 94 (47%) participants finished all 7 modules, while 66 (33%) participants completed the additional booster session.

    Outcome Measurements

    Health-Related Outcome

    The health outcome in the cost-effectiveness analysis (CEA) was symptom-free status based on the Perceived Stress Scale (PSS) and defined as a score at a 6-month follow-up of 2 SDs below the baseline mean of the study population (22.65, SD 5.63) [19].

    Quality-Adjusted Life Years

    Quality-adjusted life years (QALYs) were used as a health outcome in the cost-utility analysis (CUA). QALYs were based on the 35-item version of the Assessment of Quality of Life (AQoL-8D), which was assessed at baseline and a 6-month follow-up and is a reliable and validated instrument [20]. In total, 8 dimensions of health-related quality of life were covered (ie, independent living, relationships, mental health, coping, pain, senses, self-worth, and happiness) and preference-based valuations of health states (utilities) on a scale of 0 (death) to 1 (perfect health) were generated, using the time trade-off method [21]. Cumulative QALYs gains over the study’s follow-up time of 6 months were estimated by calculating the area under the curve (AUC) of linearly interpolated AQoL-8D utilities between measurement points to cover the whole follow-up period.

    Costs
    Resource Use and Costing

    We used the Trimbos and Institute for Medical Technology Assessment “Treatment Inventory of Costs in Patients with psychiatric disorders” (TiC-P) questionnaire to collect data on health care use, patient and family costs, and productivity losses [22]. The TiC-P is a retrospective questionnaire with a 3-month recall period and has been used in a similar study [15]. Costs were expressed in Euro and indexed from 2011 to 2013, the year the study was conducted, based on the German consumer price index (index factor 1.04) [23]. Costs were converted to pound sterling (£) using the purchasing power parities reported by the Organization for Economic Cooperation and Development [24]. For the reference year 2013, €1 was equated to £0.85 (A conversion rate of 1.33 in 2013 can be used for the conversion from € to US $).

    Intervention Costs

    At the time of conducting the study, the market price of the digital intervention provided by the GET.ON Institute, a commercial health care service provider was €99 (£84; ie, US $108.62) per participant, including costs for text messages, costs for website maintenance and hosting, technical support, and overheads.

    Health Care Costs

    We used 2 German guidelines for calculating health care costs [25,26]. Health care costs on a per-participant level were based on available lists of unit costs [26]. Unit costs were as follows: €20.92 (£17.78; ie, US $22.95) for a visit to the general practitioner, €46.55 (£39.57; ie, US $51.07) for a session with a psychiatrist, and €81.44 (£69.22; ie, US $89.36) with a psychotherapist, respectively. Costs per contact for allied health services (eg, physiotherapist) were valued at €17.08 (£14.52; ie, US $18.74). Hospital stays were computed at €335.52 (£285.19; ie, US $368.13) for an in-patient day in a psychiatric hospital and €306.41 (£260.45; ie, US $336.19) for an in-patient day in a hospital for psychosomatic medicine and psychotherapy. Costs were estimated by multiplying the units of resource use with corresponding unit costs. The costs of prescribed medication were based on Lauer-Taxe [27].

    Patient and Family Costs

    Out-of-pocket payments were directly obtained from participants. Costs for traveling were valued at €0.30 (£0.27; ie, US $0.33) per kilometer. Productivity losses from unpaid work (eg, household chores, shopping, and child care) and informal care were valued using the proxy good method (eg, price of a close market substitute: domestic help). The average gross wage of domestic help per hour was estimated at €18.33 (£15.58; ie, US $20.11) per hour.

    Productivity Costs

    Costs due to absenteeism (ie, days not worked) were valued according to the human capital method [28]. Lost working days due to absenteeism were valued at the gross average income of participants per day. Lost working days due to presenteeism (ie, reduced efficiency while at work) were computed by taking into account the number of working days for which the participant reported a reduced work performance weighted by an inefficiency score for those days (Osterhaus method) [29].

    Statistical Analysis
    Health-Related Outcome, QALYs, and Costs

    Analyses were conducted according to the intention-to-treat principle using Stata (version 16; StataCorp) [30]. Missing data were imputed using multiple imputations by chained equations (MICE) using predictive mean matching to account for the skewed distribution of cost and utility data. The imputation model was stratified by the study arm and included demographic data (eg, age, gender, marital status, and education) alongside health-related outcome variables at baseline (eg, utility values and perceived stress). The number of imputed datasets was at least equal to the percentage of incomplete cases (ie, m=30). Analyses, as described below, were performed on each dataset separately, and results were pooled using Rubin’s rules.

    Economic Evaluation: Societal Perspective

    Disaggregated and total costs from the employer’s and societal perspectives, as well as QALYs per study group, were assessed with a set of ordinary least square regression equations, the latter adjusted for baseline utility values. Group differences in symptom-free status were tested using logistic regression.

    From a societal perspective, the incremental cost-effectiveness ratio (ICER) was calculated as the extra costs per additional symptom-free participant or QALY gained, respectively. We bootstrapped seemingly unrelated regression equations (SURE) models to generate 2500 simulations of incremental cost and effect pairs while allowing for correlated residuals of the cost and effect equations and adjusting for potential confounders (eg, initial utility values in the effect equation). Based on the bootstrapped SURE models, bias-corrected and accelerated 95% CIs were obtained for incremental costs and effects. Bootstrapped cost and effect pairs were plotted on cost-effectiveness planes to graphically represent the uncertainty surrounding the ICERs. Cost-effectiveness acceptability curves (CEACs) were created to depict the probability of the intervention being cost-effective compared with the control condition for varying willingness-to-pay (WTP) thresholds.

    To determine subgroups in which the intervention was particularly cost-effective, the net-benefit regression framework (NBRF) was used [31]. In the NBRF, the treatment dummy, prognostically relevant baseline characteristics, and their interactions are regressed on net benefit. Net benefit (NB) is defined as NB = (E*λ)–C, where E denotes the effects per participant (eg, QALYs), C denotes the costs per participant, and λ is the willingness-to-pay for a unit of effect (ie, €20,000 [ie, US $21,944] per QALY gained). Analyses were conducted using gender, age, education, marital status, work experience, previous health training or psychotherapy, level of perceived stress, resilience, agreeableness, psychological strain, and self-regulation competencies as independent variables.

    Economic Evaluation: Employer’s Perspective

    From the employer’s perspective, cost-benefit analyses were performed using 2 metrics: (1) net benefits (NB = benefits – costs; amount of money gained after costs are taken into account) and (2) return-on-investment (ROI = [(Benefits–Costs)/Costs] × 100%; percentage of profit per Euro invested), where costs are defined as intervention costs and benefits as the difference in productivity costs between the iSMI group and the control condition. The metrics were estimated by bootstrapping linear regression models (N=2500). The probability of a positive financial return was assessed by the proportion of positive estimates (eg, NB>0, ROI>0%).

    Sensitivity Analyses

    To test the robustness of the base case findings, we performed 3 sensitivity analyses. First, we applied Winsorizing, where extreme values of cost outliers (eg those above the 95th percentile) were replaced by the value at the 95th percentile. Second, we varied the costs of the intervention (ie, €299; ie, US $328.06) to reflect uncertainties about the actual market price. Third, we used another instrument, the EQ-5D-3L [32], to assess health-related quality of life instead of the AQoL-8D. While the first 2 sensitivity analyses were applied to both the societal and employer’s perspectives, the latter was only done from the societal perspective.

    Ethical Considerations

    The study was approved by the ethics committee of the University of Marburg (Germany [Aktenzeichen 2014-30k (date: 08/14/2014]) and registered in the German clinical trials register (DRKS00005699) on December 12, 2014.


    Health-Related Outcomes and QALYs

    Participants in the iSMI condition had a statistically significantly higher probability for symptom-free status at 6-month follow-up with an odds ratio of 5.14 (95% CI 3.23-8.18) compared with WLC based on bootstrapped data. The mean of the cumulative QALYs was higher in the iSMI condition (0.321 QALYs, 95% CI 0.32-0.33) compared with the WLC (0.306 QALYs, 95% CI 0.30-0.31). Adjusted incremental differences in QALYs between the iSMI condition and the WLC were statistically significant (Δ(e)=0.015 QALYs, 95% CI 0.01-0.02).

    Costs

    Baseline costs were slightly higher in the iSMI condition (€2283, 95% CI 1916-2650 [US $2504.91, 95% CI 2102.24-2907.58]) but comparable to the WLC (€1936, 95% CI 1569-2303 [US $21.24.18, 95% CI 1721.51-2526.85]). The imputed mean 6-month cumulative per-participant costs (in €) separately for various cost categories by study condition are presented in Table 1. Direct medical costs, as well as patient and family costs, were similar in both groups. In the iSMI group, costs for absenteeism were higher than costs due to presenteeism. The opposite was seen in the WLC. Employer’s costs were slightly higher in the WLC compared with the iSMI (incremental difference of €76, 95% CI −667 to 515 [US $83.39, 95% CI –731.83 to 565.06]). Average total costs were comparable in both groups, with €3195 (US $3505.55; 95% CI 2661-3729 [95% CI 2919.65-4091.46]) in the iSMI condition and €3233 (US $3547.25; 95% CI 2722-3744 [95% CI 2986.58-4107.92]) in the WLC, resulting in an incremental difference of €38 (US $41.69; 95% CI −771 to 695 [95% CI –845.94 to 762.55]) in favor of the iSMI group.

    Table 1. Imputed mean cumulative per-participant costs (in €) by condition over a 6-month follow-up period.

    		Intervention group (n=198)Control group (n=198)Incremental difference

    		Mean, €a95% CIMean, €95% CIMean, €95% CI
    Direct medical costs

    		Intervention costs99—b——99—

    		General practitioner5242 to 625041 to 582−11 to 16

    		Mental health care12667 to 185227168 to 286−101−184 to −18

    		Antidepressants4−5 to 14155 to 25−10−24 to 4

    		Allied health servicesc4123 to 603013 to 4611−13 to 36
    Patient and family costs

    		Over-the-counter drugs2013 to 282619 to 34−6−16 to 5

    		Out of pocket expensesd10656 to 15610461 to 1482−63 to 66

    		Travel116 to 161510 to 20−4−11 to 4

    		Unpaid work15990 to 229187122 to 252−28−125 to 70

    		Informal care330195 to 46619492 to 297136−35 to 307

    		Domestic help16889 to 24813344 to 22335−88 to 159
    Productivity costs

    		Absenteeism1217897 to 1538884584 to 1184334−102 to 770

    		Presenteeism859616 to 110313681128 to 1609−509−852 to −166
    Employer’s perspective

    		Intervention costs + productivity costs21761747 to 260522521841 to 2663−76−667 to 515
    Societal perspective

    		Total societal costse31952661 to 372932332722 to 3744−38−771 to 695

    a€1=US $1.10.

    bNot available.

    cFor example, massage, physiotherapist, and occupational therapist.

    dFor example, allied health services without prescription.

    eIncludes all cost categories based on a bootstrapped (n=5000) linear regression model. Columns may not add up correctly due to rounding.

    Economic Evaluation

    Societal Perspective
    Cost-Effectiveness

    Table 2 shows the incremental costs, effects, mean ICERs, and the distribution of cost and effect pairs on the cost-effectiveness plane based on 2500 bootstrap simulations. Cost-effectiveness analysis revealed that the iSMI generated more symptom-free individuals (Δ[E]=0.39; 95% CI 0.29-0.48) at lower costs (Δ[C]=–€38 [£953; ie, US $1246.51]; 95% CI €–705 to €692 [ie, 95% CI US $–773.32 to $759.06]) relative to the WLC. With regard to the cost-effectiveness plane, 56% of the bootstrapped ICERs fell in the southeast quadrant, indicating a 56% probability that the intervention dominates WLC (Figure 1). The remaining 44% of ICERs fell in the northeast quadrant, demonstrating a 44% probability that the intervention leads to greater health gains but at higher costs than WLC. At a willingness-to-pay of €1500 (US $1645.37) per additional symptom-free person, the iSMI showed a probability of 94% being cost-effective compared with the WLC (Figure 2).

    Table 2. Results from the societal perspective (main and sensitivity analysis) based on 2500 bootstrap simulations.
    OutcomeIncremental costs, € (95% CI)Incremental effects, points (95% CI)Incremental cost-effectiveness ratio, € or points (95% CI)aDistribution over the cost-effectiveness plane (%)




    		North-east quadrantbSouth-east quadrantcSouth-west quadrantdNorth-west quadrante
    Main analysis

    		Symptom-free status (0/1)−38 (−705 to 692)0.39 (0.29 to 0.48)Dominant445600

    		QALYsf (range: 0-1)−38 (−735 to 687)0.015 (0.01 to 0.02)Dominant455500
    Sensitivity analysis 1g

    		Symptom-free status (0/1)−110 (−652 to 477)0.39 (0.29 to 0.48)Dominant346600

    		QALYsf (range: 0-1)−110 (−652 to 477)0.025 (0.01 to 0.04)Dominant346600
    Sensitivity analysis 2h

    		Symptom-free status (0/1)162 (−505 to 892)0.39 (0.29 to 0.48)€418891100

    		QALYsf (range: 0-1)162 (−505 to 892)0.025 (0.01 to 0.04)€6,515891100
    Sensitivity analysis 3i

    		QALYsf (range: 0-1)−38 (−735 to 687)0.011 (0.00 to 0.02)Dominant455500

    aIn accordance with ISPOR best practice guidelines on “Model Parameter Estimation and Uncertainty,” we do not report any negative incremental cost effectiveness. Ratios (ICERs) since they are meaningless. Instead, we use the term “dominant,” which means that the intervention has a higher impact and comparatively lower cost with the WLC.

    bThe northeast quadrant of the CE plane, indicating that intervention is more effective and more costly.

    cThe southeast quadrant of the CE plane, indicating that intervention is more effective and less costly.

    dThe southwest quadrant of the CE plane, indicating that intervention is less effective and less costly.

    eThe northwest quadrant of the CE plane, indicating that intervention is less effective and more costly.

    fQALYs: quality-adjusted life years.

    gSensitivity analysis 1 analyses for winzorizing cost outlier to 95% percentiles.

    hSensitivity analysis 2 analyses adding intervention costs of €299 (US $327.98; instead of €99 [US $108.59]).

    iSensitivity analysis 3 analyses for EuroQol for quality-adjusted life years.

    ‎
    Figure 1. Scatterplot of 5000 replicates of the incremental cost-effectiveness ratio (mean differences in costs and symptom-free status) on the cost-effectiveness plane from the societal perspective: iSMI versus WLC.
    ‎
    Figure 2. Cost-effectiveness acceptability curves (symptom-free status) from the societal perspective.
    Cost-Utility

    Cost-utility analysis revealed similar results (Table 2). Again, the iSMI was likely to be dominant relative to the WLC with more QALY gains at lower costs, resulting in a 55% probability that the iSMI is cost-effective at a societal willingness-to-pay threshold of €0 (Figure 3). This probability increased to 80% at a societal willingness-to-pay per QALY gained of €20,000 (US $21,938.20; Figure 4). Using NBRF, no subgroups were identified for which iSMI was significantly more (or less) cost-effective (all P-values ≥.05 for both WTPs of 0€ and €20,000 [US $21,938.20] per QALY gained, respectively).

    ‎
    Figure 3. Scatterplot of 5000 replicates of the incremental cost-effectiveness ratio (mean differences in costs and QALYs) on the cost-effectiveness plane from the societal perspective: iSMI versus WLC. QALYs: quality-adjusted life years.
    ‎
    Figure 4. Cost-effectiveness acceptability curves (QALYs) from the societal perspective. QALYs: quality-adjusted life years.
    Employer’s Perspective

    The iSMI condition showed a net benefit per participant of €76 (£65; ie, US $83.37; 95% CI €−498 to 665 [ie, 95% CI US $–546.26 to $729.45]) and a benefit-to-cost ratio of 1.77 (95% CI €−4.03 to 7.72 [ie, 95% CI US $–4.42 to $8.47]). The ROI was 77% (95% CI −503% to 672%), respectively. The probability of a positive ROI was 78% for the iSMI condition (Table 3).

    Table 3. Results from the employer’s perspective (main and sensitivity analyses) of adjusted cost-benefit analyses based on 5000 bootstrapped linear regression models.

    		CostsaBenefitsbFinancial returns

    		Total95% CITotal95% CINBc95% CIROId (%)95% CIPe (%)
    Main analysis

    		Unguided intervention99—f175−399 to 76476−498 to 66577−50 to 67278
    Sensitivity analysis 1g

    		Unguided intervention99—217−226 to 687118−325 to 588120−329 to 59496
    Sensitivity analysis 2h

    		Unguided intervention299—175−399 to 764−124−698 to 465−41−233 to 15614

    aIncludes intervention costs.

    bBenefits are the difference in productivity costs between the intervention group and the control condition.

    cNB: Net benefit linear regression models adjusted for baseline costs due to absenteeism and presenteeism.

    dROI: Return on investment linear regression models adjusted for baseline costs due to absenteeism and presenteeism.

    eProbability of positive return on investment.

    fNot available.

    gSensitivity analysis 1 analyses for winzorizing cost outlier to 95% percentiles.

    hSensitivity analysis 2 analyses adding intervention costs of €299 (ie, US $ 327.98; instead of €99 [US $108.59]).

    Sensitivity Analyses

    The results of the sensitivity analyses are summarized in Table 2 and Table 3. Winsorizing cost outliers led to a slightly higher probability that the intervention produces higher health gains at lower costs than WLC with regard to symptom-free status (66%) and QALYs (65%) at a societal WTP threshold of €0. The net benefit increased when cost outliers were winzorized, and the probability of a positive return on investment was 96% for the iSMI. Increasing intervention costs up to 3 times (€299 [US $327.98] instead of €99 [108.59]), this probability decreased to 33% regarding symptom-free status and QALYs gained at a WTP of €0. However, at a WTP of €20,000 [US $21,938.20] per QALY gained, the probability of being cost-effective was comparable to the main analysis (eg, 78%). Return on investment became negative when intervention costs were increased up to 3 times, and the probability of a positive financial return decreased to 14% for the iSMI. Using the EQ-5D-3L resulted in a probability of 55% regarding QALYs gained at a WTP of €0, identical to the main analysis.


    Principal Findings

    This study evaluated the cost-effectiveness and cost-utility of a universal unguided digital stress management intervention for employees from a societal perspective and the cost-benefit from the employer’s perspective compared with a waitlist control condition over a 6-month time horizon. From a societal perspective, the iSMI had a high probability of being cost-effective (eg, 80% at a WTP of €20,000 [US $21,938.20] per QALY gained). From an employer’s perspective, the iSMI had a high probability of a positive return on investment with 78%.

    Comparison With Previous Work

    Evidence for economic evaluations of universal digital prevention of mental disorders is scarce. To our knowledge, this is the first economic evaluation of a universal iSMI to reduce stress in employees using a societal and an employer’s perspective. The results of our study are in line with other economic evaluations of the same iSMI for stressed employees in the field of indicated prevention (PSS≥22) in which e-coaches provided personalized feedback throughout the intervention [15]. Kählke et al [15] demonstrated that more QALYs were generated for lower costs in the guided iSMI compared with a waitlist control condition, indicating a similar probability of 76% compared with our study that the intervention was cost-effective compared with WLC at a societal WTP of €20,000 [US $21,938.20] per QALY gained. A similar probability of 71% at a societal WTP of US $25,000 was also found by Lindsäter et al [33], who evaluated therapist-guided internet-based Cognitive Behavioral Therapy for stress-related disorders compared to WLC.

    Results from the employer’s perspective are comparable to the study by Ebert et al [16], who examined the cost-benefit of the same iSMI as Kählke et al [15]. The analysis of Ebert et al [16] yielded a similar benefit-to-cost ratio of 1.6 (95% CI €−1.2-4.5 [US $–1.32 to $4.94]) compared with a benefit-to-cost ratio of 1.77 (95% CI €−4.0 to 7.7 [US $–4.39 to $8.45]) in this study. Our results from the employer’s perspective are also in line with findings from a recent systematic review indicating that addressing mental health in employees improves both their well-being and productivity [34]. Regarding the ROI analyses, our findings compare favorably to a systematic review of face-to-face health promotion interventions at the workplace (12 RCTs) that showed, on average, a negative ROI (ROI=–0.22, 95% CI 0.27-0.16; min=−4.3, max=5) [35]. van Dongen et al [36] also showed a negative return on investment of a combined social and physical environmental intervention in office employees.

    Limitations

    The following limitations have to be considered. First, contrary to the pharmaco-economic guidelines recommending TAU as a comparator [37], the iSMI was compared with a wait-listed control condition, albeit with unrestricted access to usual care. Second, due to the limited time horizon of 6 months, longer-term costs and effects caused by chronic stress (eg, the onset of a new health disorder or staff turnover) could not be analyzed. Future studies should examine whether treatment effects and costs are sustained over a longer period of time or decline. Third, preference-based utility values were only evaluated at baseline and after 6 months, and thus, an immediate treatment effect was not assessed. Fourth, costs due to presenteeism were only assessed using the Osterhaus method. However, this method tends to overestimate costs because it assumes a 1:1 relationship between work hours lost and productivity losses, but that relationship might be better described as an elastic relationship, for example, 100% to 90%. Future studies should also include alternative methods (eg, Health and Labor Questionnaire method) to calculate costs due to presenteeism. Fifth, self-report questionnaires were used, which may have led to effects related to social desirability and response bias [38]. Future studies could consider claims data from health insurance companies. Sixth, the sample is characterized by a high proportion of female (302/396, 76.3%) as well as highly educated (285/396, 72%) participants, limiting the generalizability of the study findings. However, women with higher education are often the typical target group who take part in preventive internet-based interventions [39].

    Clinical Implications

    Evidence-based recommendations on the cost-effectiveness of interventions can help inform decision-makers from the societal and employer’s perspective when choosing preventive stress interventions. The findings of this study support the hypothesis that a universal prevention approach could be a cost-effective strategy to reduce the adverse consequences of work-related stress besides its shown effectiveness [14]. However, it must be taken into account that the participants in the study already had a rather high initial stress level on average. That is, we cannot say whether the cost-effectiveness results will hold up if increased numbers of participants with low levels of stress participate in the intervention. An unguided universal iSMI might be appropriate from an economic perspective since there are no costs for screening compared with an iSMI for indicated prevention and additionally no costs for therapeutic guidance compared with guided interventions. Since health care professionals and available resources are often limited, unguided interventions have the potential to be implemented in routine occupational care on a large scale. Meta-analytic evidence shows that unguided interventions generate similar effects compared with guided interventions in individuals with low symptom severity [40]. However, the evidence on this is heterogeneous [41]. Therefore, future studies should directly compare the cost-effectiveness of guided compared with unguided iSMIs. From a health economic point of view, no moderators were revealed in NBRF analyses. However, this could also have been due to the small sample size and should be further researched in an individual participant data meta-analysis in the future.

    Conclusions

    A universal unguided digital stress management intervention appears to be cost-effective (ie, the health effects achieved represent good value for the invested money) as well as offering a favorable cost-to-benefit ratio (ie, the financial gains outweigh the intervention costs, so the return on investment is positive). Further research is needed to examine the long-term effects on cost-effectiveness and to compare digital stress management intervention with standard care. Unguided digital stress prevention carries the promise of being scalable, which would leverage the cost-effectiveness and positive return on investment of this type of universal stress prevention in the working population.

    Acknowledgments

    This publication was funded by the German Research Foundation (DFG). The funders did not have a role in the study design, data collection, analysis, and interpretation of results or the decision to publish the study results.

    Data Availability

    The datasets generated during and/or analyzed during this study are not publicly available but are available from the corresponding author on reasonable request by qualified scientific and medical researchers for legitimate research purposes. The data will be shared under a data-sharing agreement once the results of the health-economic evaluation are published. Requests should be sent to the last author.

    Authors' Contributions

    BF and MB obtained funding for the study. DDE and DL designed the randomized controlled trial. DDE, DL, HR, and MB developed the intervention content. ACZ was responsible for the recruitment of participants, data collection, and data curation, supervised by DL and DDE. CB developed the analysis plan for the health-economic evaluation, which was supported by FS. CB and JF accessed and verified the data. MB provided the analysis tool. CB conducted the analyses. CB, JF, and FS interpreted the data. JF drafted the manuscript and was supervised by CB. All authors participated in the critical review of the manuscript and approved the final manuscript.

    Conflicts of Interest

    BF and DDE are stakeholders of the GET.ON Institute (HelloBetter), which aims to implement scientific findings related to digital health interventions into routine care. MB is a cofounder and stakeholder of mentalis GmbH, a provider of digital mental health care products and services. DDE has served as a consultant and on the scientific advisory boards of Sanofi, Novartis, Minddistrict, Lantern, Schoen Kliniken, Ideamed, and German health insurance companies (BARMER, Techniker Krankenkasse) and a number of federal chambers for psychotherapy. ACZ receives license fees for a digital health application (DiGA) for sexual dysfunction implemented in routine care in Germany. She reports receiving fees for delivering presentations at scientific conferences and for producing expert videos. JF, FS, DL, HR, and CB report no conflicts of interest.

    Multimedia Appendix 1

    CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklist.

    DOCX File , 23 KB
    1. Eurofound. In: Developments in Working Life in Europe: EurWORK Annual Review 2016. Luxembourg. Publications Office of the European Union; 2017.
    2. Nieuwenhuijsen K, Bruinvels D, Frings-Dresen M. Psychosocial work environment and stress-related disorders, a systematic review. Occup Med (Lond). 2010;60(4):277-286. [CrossRef] [Medline]
    3. Duric V, Clayton S, Leong ML, Yuan L. Comorbidity factors and brain mechanisms linking chronic stress and systemic illness. Neural Plast. 2016;2016:5460732. [FREE Full text] [CrossRef] [Medline]
    4. Jensen I, Arapovic-Johansson Z, Aboagye E. The cost-effectiveness analysis of the productivity measurement and enhancement system intervention to reduce employee work-related stress and enhance work performance. Int J Environ Res Public Health. 2022;19(4):2431. [FREE Full text] [CrossRef] [Medline]
    5. Lazarus RS, Folkman S. Stress, Appraisal, and Coping. New York. Springer publishing company; 1984.
    6. Khoury B, Sharma M, Rush SE, Fournier C. Mindfulness-based stress reduction for healthy individuals: a meta-analysis. J Psychosom Res. 2015;78(6):519-528. [CrossRef] [Medline]
    7. Heckenberg RA, Eddy P, Kent S, Wright BJ. Do workplace-based mindfulness meditation programs improve physiological indices of stress? A systematic review and meta-analysis. J Psychosom Res. 2018;114:62-71. [CrossRef] [Medline]
    8. Hilton MF, Whiteford HA, Sheridan JS, Cleary CM, Chant D, Wang P, et al. The prevalence of psychological distress in employees and associated occupational risk factors. J Occup Environ Med. 2008;50(7):746-757. [CrossRef] [Medline]
    9. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1-18. [CrossRef] [Medline]
    10. Heber E, Ebert DD, Lehr D, Cuijpers P, Berking M, Nobis S, et al. The benefit of web- and computer-based interventions for stress: a systematic review and meta-analysis. J Med Internet Res. 2017;19(2):e32. [FREE Full text] [CrossRef] [Medline]
    11. McLaughlin KA. The public health impact of major depression: a call for interdisciplinary prevention efforts. Prev Sci. 2011;12(4):361-371. [FREE Full text] [CrossRef] [Medline]
    12. Muñoz RF, Cuijpers P, Smit F, Barrera AZ, Leykin Y. Prevention of major depression. Annu Rev Clin Psychol. 2010;6:181-212. [CrossRef] [Medline]
    13. Couser GP. Challenges and opportunities for preventing depression in the workplace: a review of the evidence supporting workplace factors and interventions. J Occup Environ Med. 2008;50(4):411-427. [CrossRef] [Medline]
    14. Ebert DD, Franke M, Zarski AC, Berking M, Riper H, Cuijpers P, et al. Effectiveness and moderators of an internet-based mobile-supported stress management intervention as a universal prevention approach: randomized controlled trial. J Med Internet Res. 2021;23(12):e22107. [FREE Full text] [CrossRef] [Medline]
    15. Kählke F, Buntrock C, Smit F, Berking M, Lehr D, Heber E, et al. Economic evaluation of an Internet-based stress management intervention alongside a randomized controlled trial. JMIR Ment Health. 2019;6(5):e10866. [FREE Full text] [CrossRef] [Medline]
    16. Ebert DD, Kählke F, Buntrock C, Berking M, Smit F, Heber E, et al. A health economic outcome evaluation of an internet-based mobile-supported stress management intervention for employees. Scand J Work Environ Health. 2018;44(2):171-182. [FREE Full text] [CrossRef] [Medline]
    17. Ramsey SD, Willke RJ, Glick H, Reed SD, Augustovski F, Jonsson B, et al. Cost-effectiveness analysis alongside clinical trials II-An ISPOR good research practices task force report. Value Health. 2015;18(2):161-172. [FREE Full text] [CrossRef] [Medline]
    18. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. ISPOR Health Economic Evaluation Publication Guidelines-CHEERS Good Reporting Practices Task Force. Consolidated health economic evaluation reporting standards (CHEERS)--explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. 2013;16(2):231-250. [FREE Full text] [CrossRef] [Medline]
    19. Reis D, Lehr D, Heber E, Ebert DD. The German version of the perceived stress scale (PSS-10): evaluation of dimensionality, validity, and measurement invariance with exploratory and confirmatory bifactor modeling. Assessment. 2019;26(7):1246-1259. [CrossRef] [Medline]
    20. Richardson J, Iezzi A, Khan MA, Maxwell A. Validity and reliability of the assessment of quality of life (AQoL)-8D multi-attribute utility instrument. Patient. 2014;7(1):85-96. [FREE Full text] [CrossRef] [Medline]
    21. Richardson J, Sinha K, Iezzi A, Khan MA. Modelling utility weights for the assessment of quality of life (AQoL)-8D. Qual Life Res. 2014;23(8):2395-2404. [CrossRef] [Medline]
    22. Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis C, Tan Swan S, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res. 2013;13:217. [FREE Full text] [CrossRef] [Medline]
    23. German Consumer Price Index. DESTATIS. URL: https://www.destatis.de/EN/Themes/Economy/Prices/Consumer-Price-Index/_node.html [accessed 2021-03-15]
    24. Exchange rates (indicator). OECD. 2015. URL: https://tinyurl.com/76er8745 [accessed 2022-09-14]
    25. Krauth C, Hessel F, Hansmeier T, Wasem J, Seitz R, Schweikert B. Empirische bewertungssätze in der gesundheitsökonomischen evaluation - ein Vorschlag der AG methoden der gesundheitsökonomischen evaluation (AG MEG). Gesundheitswesen. 2005;67(10):736-746. [CrossRef] [Medline]
    26. Bock JO, Brettschneider C, Seidl H, Bowles D, Holle R, Greiner W, et al. Calculation of standardised unit costs from a societal perspective for health economic evaluation. Gesundheitswesen. 2015;77(1):53-61. [CrossRef] [Medline]
    27. Lauer Taxe. WEBAPO. 2009. URL: https://de.wikipedia.org/wiki/Lauer-Taxe [accessed 2014-06-24]
    28. Drummond M, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Oxford. Oxford University Press; 2015.
    29. Osterhaus JT, Gutterman DL, Plachetka JR. Healthcare resource and lost labour costs of migraine headache in the US. Pharmacoeconomics. 1992;2(1):67-76. [CrossRef] [Medline]
    30. Stata Version 16. College Station, TX. Stata Corp URL: https://www.stata.com/stata-news/news35-1/updates-to-stata16/ [accessed 2022-09-14]
    31. Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ. 2002;11(5):415-430. [CrossRef] [Medline]
    32. EuroQol Group. EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199-208. [CrossRef] [Medline]
    33. Lindsäter E, Axelsson E, Salomonsson S, Santoft F, Ljótsson B, Åkerstedt T, et al. Cost-effectiveness of therapist-guided internet-based cognitive behavioral therapy for stress-related disorders: secondary analysis of a randomized controlled trial. J Med Internet Res. 2019;21(9):e14675. [FREE Full text] [CrossRef] [Medline]
    34. de Oliveira C, Cho E, Kavelaars R, Jamieson M, Bao B, Rehm J. Economic analyses of mental health and substance use interventions in the workplace: a systematic literature review and narrative synthesis. Lancet Psychiatry. 2020;7(10):893-910. [CrossRef] [Medline]
    35. Baxter S, Sanderson K, Venn AJ, Blizzard CL, Palmer AJ. The relationship between return on investment and quality of study methodology in workplace health promotion programs. Am J Health Promot. 2014;28(6):347-363. [CrossRef] [Medline]
    36. van Dongen JM, Coffeng JK, van Wier MF, Boot CRL, Hendriksen IJM, van Mechelen W, et al. The cost-effectiveness and return-on-investment of a combined social and physical environmental intervention in office employees. Health Educ Res. 2017;32(5):384-398. [CrossRef] [Medline]
    37. Sacristán JA, Abellán-Perpiñán JM, Dilla T, Soto J, Oliva J. Some reflections on the use of inappropriate comparators in CEA. Cost Eff Resour Alloc. 2020;18:29. [FREE Full text] [CrossRef] [Medline]
    38. van DMT. Faking it: social desirability response bias in self-report research. Aust J Adv Nurs. 2008;25(4):40-48.
    39. Kelders SM, Bohlmeijer ET, Van Gemert-Pijnen JE. Participants, usage, and use patterns of a web-based intervention for the prevention of depression within a randomized controlled trial. J Med Internet Res. 2013;15(8):e172. [FREE Full text] [CrossRef] [Medline]
    40. Karyotaki E, Efthimiou O, Miguel C, Bermpohl FMG, Furukawa TA, Cuijpers P, Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis. JAMA Psychiatry. 2021;78(4):361-371. [FREE Full text] [CrossRef] [Medline]
    41. Domhardt M, Geßlein H, von Rezori RE, Baumeister H. Internet- and mobile-based interventions for anxiety disorders: a meta-analytic review of intervention components. Depress Anxiety. 2019;36(3):213-224. [CrossRef] [Medline]

    ‎
    AQoL-8D: Assessment of Quality of Life
    AUC: area under the curve
    CEA: cost-effectiveness analysis
    CEAC: cost-effectiveness acceptability curve
    CUA: cost-utility analysis
    ICER: incremental cost-effectiveness ratio
    IG: intervention group
    iSMI: internet-based stress management intervention
    MICE: multiple imputations by chained equations
    NB: net benefit
    NBRF: net-benefit regression framework
    PSS: Perceived Stress Scale
    QALY: quality-adjusted life year
    RCT: randomized controlled trial
    ROI: return-on-investment
    SURE: seemingly unrelated regression equations
    TAU: treatment as usual
    TiC-P: Treatment Inventory of Costs in Patients with Psychiatric Disorders
    WLC: waitlist control condition
    WTP: willingness-to-pay

    Edited by S Ma; submitted 25.04.23; peer-reviewed by C de Miquel, F Lopez Segui, M Gasmi , S Kaur; comments to author 09.01.24; revised version received 29.01.24; accepted 28.06.24; published 22.10.24.

    Copyright

    ©Johanna Freund, Filip Smit, Dirk Lehr, Anna-Carlotta Zarski, Matthias Berking, Heleen Riper, Burkhardt Funk, David Daniel Ebert, Claudia Buntrock. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 22.10.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.