The Efficacy of a Smartphone-Based App on Stress Reduction: Randomized Controlled Trial

Introduction

Stress management is undoubtedly crucial to the mental health as well as physical health of individuals. Chronic and high stress have been linked to depression [1], anxiety [2], coronary heart disease [3], and increased mortality [4]. Stress can be acquired from all aspects of life. Nevertheless, work-related stress has been of interest to researchers, especially in South Korea, which has one of the longest working hours within the Organization for Economic Cooperation and Development countries [5]. High stress within the workplace has also been associated with calculable economic burdens [6] and various long-term complications [7]. Therefore, stress management within the workplace is vital for both employees and employers.

Of the known methods for stress management, cognitive behavioral therapy (CBT) has been identified as effective [8,9]. Traditional CBT is performed by trained professionals such as psychiatrists or psychologists. It is usually conducted face to face or in a group setting. However, advancements in technology have allowed CBT to be computerized and administered via the internet, allowing it to be administered automatically with minimal guidance [8,10,11]. Such online CBTs have demonstrated efficacy in several areas, including the reduction of stress [12], depression, anxiety [13], and insomnia [14].

Although online CBT has advantages over traditional CBT in terms of anonymity and accessibility [15], in traditional CBT, the therapist can tailor the structured CBT to suit the client. This contrasts with most online CBTs, in which participants usually receive the same program that cannot be changed. Tailored online CBT has been implemented in treating depression and is effective, as shown in a meta-analysis [16]. However, Johansson et al [17] argued that tailored treatment is essential in addressing comorbidity, showing that tailored CBT may be more effective than ordinary internet CBT in treating depression. Stress is also associated with various disorders [1,2]. Thus, we developed an individualized CBT for stress management and performed a randomized controlled trial (RCT) to demonstrate its efficacy.

We hypothesized that using an online CBT-based app designed to help manage work stress for 10 weeks would result in a statistically significant improvement in stress-related scales compared to a waiting group.

Methods

Sample Size

The sample size was calculated using G*Power (Heinrich-Heine-Universität Düsseldorf), based on a statistical test on repeated measure analysis of variance (ANOVA), since the covariance number was not determined. A type I error of .05 and statistical power of 0.8 were used. The correlation among repeated measures was set conservatively at 0, and based on previous similar studies [8], a mild to moderate effect size was estimated (effect size f=0.20). Based on these calculations, 51 participants were needed in each group, and after calculating a 20% dropout rate, the required total sample size was set at 128 participants.

Participants

Participants were recruited through billboard advertisements between November 2019 and January 2020. The inclusion criteria for the trial were as follows: (1) elevated perceived stress defined by a score of 14 or higher on the Perceived Stress Scale–10 (PSS); (2) indication that the reason for stress was mainly work-related (the reason for stress was considered to be work-related if the participant could identify one or more stress-related factors that they experienced in the work environment and stress in other areas such as family were minor in comparison to work-related stress); (3) employment of at least 20 hours per week and not self-employed; (4) aged 18 to 60 years; and (5) ability to provide informed consent to participate after being given information about the trial and other information that the participant must know to participate. The exclusion criteria were as follows: (1) inability to provide informed consent; (2) education level below the 9th grade; (3) history of congenital brain disorder, cerebral palsy, or other acquired brain injuries; (4) history of neurological disorders; (5) severe anxiety, depression, or psychotic disorder as assessed by the Korean Symptom Checklist–95; or (6) history of drug or alcohol abuse.

In total, 131 individuals who had work-related stress expressed interest in the trial. Of these, 3 withdrew participation after being informed of the detailed trial protocol; written consent was received from the remaining 128 participants. Two patients were excluded because of clinically relevant scores in the Korean Symptom Checklist–95. A total of 126 participants were included in the trial and randomized into 2 parallel groups with equal allocation ratios (Figure 1). Randomization was performed by an external researcher from a separate institute, and simple randomization via a random number generator using Excel (Microsoft Corp) was used. Each participant was assigned a random trial number at enrollment, and the external researcher was only provided with the individual trial number to ensure blinded randomization. The researcher performing data analysis (DHH) was also provided only with the individual trial number to blind the outcome assessment.

‎View this figure

The intervention group was given a random individual application ID to access the app, a link for downloading the app, and a manual of the program. Further, they were instructed to use the BetterLife program for 50 minutes per week for 10 weeks. They had the option to contact a designated person to ask for help in operating the app. Participants were sent a text message as a prompt to notify them of their app use every week during the trial. Members of the control group were placed on a waiting list for 10 weeks after which they were given access to the BetterLife program for the next 10 weeks. This was primarily done for ethical reasons, and data collected from the control group after using the BetterLife program were not used in this analysis. These data are being considered for use in secondary analyses, which can strengthen our research. This trial was approved by the institutional review board of Chung-Ang University Hospital (1712-008-304) and registered in the Clinical Research Information Service of Republic of Korea (KCT0003231), a member of the WHO International Clinical Trials Registry Platform. All participants enrolled in the trial provided written informed consent.

Of the 126 participants, 11 dropped out of the trial: 9 from the intervention group (14.3%) and 2 from the control group (3.2%). Of the 9 participants who were excluded from the intervention group, 2 left the trial due to personal reasons and 7 wished to leave due to lack of interest. Of the 2 participants who were excluded from the control group, 1 wished to leave the trial due to lack of interest and 1 was excluded because they were starting psychiatric treatment. Finally, 115 participants—54 in the intervention group and 61 in the control group—completed the 10-week trial and post-10-week data were collected. The 11 excluded participants’ post-10-week data were produced by the baseline observation carried forward method, and data were analyzed using an intention-to-treat method. The average use time of the program was 56.40 (SD 8.42) minutes per week; this was calculated by using the program time of the 54 participants who finished the 10-week program only. No substantial harm or unintended effects were observed.

Data Collection

Participant demographic data were collected including age, sex, education level, marital status, alcohol consumption, and smoking habits. Information on employment was also collected, including company size, work field, type of employment, job grade, whether they handle direct customer complaints, income, workdays per week, work hours per day, work experience in the current work field, and job. Information regarding the number of late days, early leave days, and absent days of the past month was also collected. The participants were asked to complete the PSS, Korean Utrecht Work Engagement Scale–9 (UWES), abbreviated version of the World Health Organization Quality of Life Assessment (WHOQOL), Beck Depression Inventory–II (BDI), and Beck Anxiety Inventory (BAI) to assess psychological and work stress-related status at baseline and after the 10-week intervention. PSS and the overall quality of life score of the WHOQOL at 10 weeks were set as the primary outcomes. Data were collected in a face-to-face setting by the researchers of Chung-Ang University Hospital at a separate location preferred by the participant. Participants received 100,000 Won (US $83) as compensation for their travel fees.

Perceived Stress Scale–10

The PSS was developed by Cohen et al [18] and validated in Korea by Park et al [19]. It is a widely used scale consisting of 10 items on a 5-point scale and is designed to measure the degree of stress experienced by individuals, with a high score indicating high perceived stress. Although the scale authors did not develop the scale with a cutoff value, the scores are often divided into 3 parts for clinical use: 0 to 13, low stress; 14 to 26, moderate stress; and >27, severe stress [20].

Korean Utrecht Work Engagement Scale–9

UWES is one of the most frequently used scales related to work engagement. It calculates work engagement by measuring vigor, dedication, and work absorption through a 7-point frequency rating scale ranging from 0 (never) to 6 (always). It was developed by Schaufeli et al [21] and has been validated in Korea by Kim et al [22].

World Health Organization Quality of Life Assessment, Abbreviated

WHOQOL was developed by the WHO as a measure to assess the quality of life (QoL) in individuals across different cultures [23]. It has been validated in Korea by Min et al [24]. The WHOQOL consists of 26 questions and uses a 5-point scale from 1 to 5, with a higher score indicating better QoL. One question pertains to overall perception of QoL (overall QoL); another pertains to overall perception of health (overall health). The remaining 24 questions were calculated to measure an individual’s QoL in each of 4 domains: physical health, psychological health, social relationships, and environment. The raw scores were converted into transformed scores (0 to 100) using the developers’ suggested method for easier comparison.

Beck Depression Inventory–II

The BDI was initially developed by Beck in 1961 and revised in 1979 to BDI-IA and finally to BDI-II in 1996 to accommodate the changes in the Diagnostic and Statistical Manual of Mental Disorders, fourth edition diagnosis for depressive disorders [25]. It is a self-report questionnaire with 21 items, and each item is rated on a 4-point scale from 0 to 3. It has been validated in many countries across the world, including Korea [26]. The BDI was designed to measure the depressive symptoms of an individual during the past 2 weeks.

Beck Anxiety Inventory

The BAI was developed by Beck in 1988 [27]. It is a self-report questionnaire with 21 items. Similar to the previous BDI, it rates each item from 0 to 3 on a 4-point scale. The BAI is designed to measure anxiety independent of depressive symptoms and was validated in Korea by Yook et al [28] in 1997.

Intervention

BetterLife is a smartphone-based program for the treatment of stress, depression, anxiety, and sleep disorders used for guided self-help therapy; it can be used for both prevention and treatment. It is designed for anonymous treatment, targeting individuals with mild or moderate symptoms. BetterLife uses recognized manuals for CBT and problem-solving therapy and consists of approximately 600 modules of 7 types: test modules, psychoeducational modules, cognitive exercises, practical exercises, diary modules, notification modules, and comment modules. The test modules included the World Health Organization Well-Being Index (WHO-5) [29], the Major Depression Inventory (MDI) [30,31], the General Anxiety Test–7 (GAD-7) [32], and a short version of the Copenhagen Psychosocial Questionnaire [33]; the latter is used for assessment of work-related stressors. Psychoeducational modules are e-learning modules that provide knowledge about illnesses, treatment methods, and background information. They have talking-head videos with an overlay of animated graphical visuals. Cognitive exercises are interactive exercises programmed in HTML5, providing opportunities for users to work with their thoughts and feelings. The interactivity makes it possible for the user to work with their data registered in a database on a server. Practical exercises include physical exercises, relaxation exercises such as meditation, and exercises away from home. Diary modules are used to map activities and the development of mental conditions over time. Data from the registrations in the diary modules are saved so the user can see them. Notification modules ensure the user’s adherence to the treatment flows, and comment modules provide feedback to the user’s treatment progression. Comment modules follow tests and exercises.

All treatment components in the program are based on evidence-based methods documented in scientific studies [11,34]. BetterLife is powered by an advanced treatment flow engine that enables individualized transdiagnostic treatment and continuous follow-up of treatment results carried out in a dialog form with the user. This dialog also includes an online dialog with an attached therapist and a chosen friend (helper) who can help the user understand the psychoeducational parts and conduct cognitive and practical exercises. The tests in the program are used both for reference to the individual treatment interventions in the program and for the continuous monitoring of treatment progression. If severe symptoms are detected, the user is referred for external psychological help in a clinic or hospital—in case of suicidal thoughts by call function directly to an acute clinic. The user sees graphical charts of the test result history to experience treatment progression. BetterLife has a toolbox with all tests and exercises in the program from where the user can choose to redo a test or exercise at any time. The primary test and exercise results are always visible on the main page of the BetterLife program.

The chance to use the WHO-5 questionnaire was provided every week to tailor to each user. Thus, the user could follow the improvement over time, even in a graph. If the test value is low (meaning low well-being), the user is referred to the depression (MDI) and anxiety (GAD-7) modules. When acceptable values of these tests are obtained (showing a low degree of depression and anxiety symptoms), the user automatically returns to the other parts of the program. Another feature to personalize the program to the user is the user’s own estimation of stressors in work and daily life. The user is guided to modules specifically concerning the problems pinpointed by the user. An overview of the program can be found in the supplements (Multimedia Appendices 1 and 2).

Statistical Analysis

All data were analyzed using SPSS (version 19.0, IBM Corp). The data presented here concern the intervention group compared to the waiting list group (controls) at baseline and follow-up. Data on the controls after using the BetterLife program and those on the program’s long-term effects are not presented. Baseline information, including demographic and psychological scales, was compared using the chi-square test, Fisher exact test, and independent t test. Intervention outcomes were measured using work-related scales and psychological scores. Changes in late days, early leave days, and absent days in the past month were also collected to determine whether the app could have a tangible effect on work life.

For all participant data, univariate logistic regression analyses were conducted to determine whether variables of independent factors—such as age, gender ratio, education level, marital status, alcohol consumption, smoking, BDI scores, BAI scores, type of work field, workdays, work hours, and work experience—could explain a significant amount of variance in the dependent variable (high perceived stress) after considering all other variables. Statistically significant independent factors were evaluated using a multivariate logistic regression analysis using a stepwise forward conditional method. High perceived stress was defined as a PSS score of 27 or higher.

Using 2-way repeated-measures analysis of covariance (ANCOVA; split-plot ANCOVA) with time as a within-factor and group as a between-factor, controlling covariates determined from the logistic regression and the changes in PSS, UWES, WHOQOL, BDI, and BAI scores were calculated. Bonferroni correction was used to compensate for multiple comparisons, and α=.05/10=.005. P<.005 was considered statistically significant. The number of late days, early leave days, and absent days were analyzed separately by 2-way repeated measure ANOVA with time as a within-factor and group as a between-factor since they were not collected as part of the initial intervention outcome.

Results

Comparison of Demographic Characteristics and Psychological Scale Scores between the Intervention and Control Groups

There were no significant differences in age, gender ratio, education level, marital status, alcohol consumption, smoking, company size, work field, type of employment, job grade, direct customer complaint handling, workdays, work hours, work experience, and late days, early leave days, and absent days in the past month between the intervention and control groups (Table 1). At baseline, there were no significant differences in PSS, UWES, WHOQOL, BDI, and BAI scores (Table 2).

Comparison of Symptom Improvement Between the Intervention and Control Groups

In logistic regression analysis, only baseline BDI scores in all participants were positively correlated with high perceived stress (B=0.20, Exp(B)=1.22, P<.001; Table 3). A separate paired t test of the repeated measures of the control group showed no significant differences after the Bonferroni correction (Multimedia Appendix 3).

In the split-plot ANCOVA, there was homogeneity of variances (P>.05) and covariances (P>.001), as assessed by the Levene test of homogeneity of variances and Box M test, respectively, in all categories except for the number of late days, early leave days, and absent days in the past month. The Mauchly test of sphericity was ignored because there were only two groups for each factor. Controlling baseline BDI scores, the intervention group showed greater improvement in the changes in PSS (F=24.33, P<.001, η²=0.17) and UWESK scores (F=8.32, P=.0046, η²=0.06) compared to the control group (Figure 2). WHOQOL scores also demonstrated a statistically significant interaction between treatment groups and pretests/posttests in overall QoL (F=8.19, P=.0049, η²=0.06) and physical health (F=8.87, P=.003, η²=0.07), psychological (F=13.32, P<.001, η²=0.10), social relationships (F=19.43, P<.001, η²=0.14), and environmental domains (F=10.14, P=.002, η²=0.08) but not overall health (F=1.68, P=.20; Table 4). Although BDI (F=7.17, P=.008) and BAI (F=6.00, P=.02) showed traditionally low P values, they did not survive the Bonferroni correction. Unadjusted ANOVA yielded similar results except for BDI (F=9.67, P=.002, η²=0.07), which showed a significant interaction between treatment groups and pretests/posttests (Multimedia Appendix 4).

‎View this figure

Similarly, in the repeated measures ANOVA, homogeneity of variances (P>.05) and covariances (P>.001) were assessed by the Levene test of homogeneity of variances and Box M test, and the Mauchly test of sphericity was ignored. There were no significant interactions in the number of late days, early leave days, and absent days in the past month between the treatment groups and pretests/posttests (Table 5).

Sensitivity and Dropout Analyses

A sensitivity analysis was performed with the 54 intervention group and 61 control group participants who completed the trial. These data were evaluated separately using per-protocol analysis. Results showed a significant interaction between treatment groups and pretests/posttests in all scales except for the overall health of WHOQOL. Compared to the intention-to-treat analysis, the per-protocol analysis showed overall more significant (lower) P values and higher effect sizes (Multimedia Appendix 5). Little difference was found between the 2 analyses on the number of late days, early leave days, and absent days (Multimedia Appendix 6).

Dropout analysis was also performed, comparing the baseline values of the 11 participants who dropped out and the 115 participants who completed the trial. In the demographic data, work hours per day were significantly lower in the dropout group (t=3.63, P<.001; Multimedia Appendix 7). When comparing the baseline work-related stress and psychological scales, the dropout group showed significantly lower PSS scores (t=3.63, P<.001) and social relationship scores in the WHOQOL (t=–2.19, P=.03; Multimedia Appendix 8). The number of early leave days was also significantly lower in the dropout group (t=4.25, P<.001).

Discussion

Principal Findings

The BetterLife app, a program developed for the management of stress, effectively improved the degree of stress in people with work-related stress measured by PSS compared with control groups on a waiting list. It also improved work engagement as determined by the UWES. Furthermore, QoL improved in all domains except for overall health, according to the WHOQOL. However, the total number of late days, early leave days, and absent days showed no improvement in the 2 groups.

Intervention Effectiveness on Stress Reduction and Work Engagement

The effects of CBT on stress management in online settings have been well documented in previous studies [8,35]. A meta-analysis of web- and computer-based stress management interventions showed that these interventions effectively reduced stress and, on average, had a moderate effect size (Cohen d=0.43) on stress reduction [8]. However, recent studies have shown larger effect sizes. Asplund et al [36] identified a significant reduction in stress with a moderate to large effect size through an RCT with a guided internet-based stress management intervention. Ebert et al [10] also observed a large effect size in stress reduction using an internet- and mobile-based stress management program [10]. Our study showed that the intervention group had a significantly higher reduction in the PSS score from baseline to posttreatment when compared to the control group with a large effect size (F=24.83, P<.001, η²=0.17). We estimate this large effect size to be because of meditation, emotional acceptance, and other third-wave CBT qualities incorporated in the program. Asplund et al [36] and Ebert et al [10] also used parts of third-wave CBT interventions, which could explain the large effect sizes of the results. Third-wave CBT helps participants to be more aware and accepting of their thoughts through ways such as meditation or behavior activation [37]. It has demonstrated strong efficacy in stress reduction [38].

The intervention group also showed improved work engagement, represented by improvements in the UWES. Work engagement, often considered the opposite of burnout, is a positive work-associated mindset with characteristics such as vigor, dedication, and absorption [39]. Extensive literature supports the positive benefits of work engagement. Halbesleben et al [40] demonstrated, through a study conducted on 587 employees in various occupations, that work engagement shared variance with job performance, implying that developing work engagement leads to positive outcomes in job performance and job retention. A meta-analysis of 7939 business units concluded that work engagement is related to meaningful business outcomes [41]. The authors of this study also implied that this improvement in employee work engagement and satisfaction could increase profits for businesses. Work engagement has also been known to affect absenteeism [42] and overall sickness absence [43]. However, our data did not show a significant difference in the number of sick days, early leave days, and absent days between baseline and postintervention. We estimate this to be because of the small number of sick, early leave, and absent days at baseline, subsequently requiring more statistical power than the trial’s original design. However, several similar studies failed to demonstrate a significant improvement in sick leave days even with symptom improvement [44], which could imply that symptom improvement through CBT alone might not be enough for a tangible improvement in sick leave.

Intervention Effectiveness on Quality of Life

Improvement of QoL in all domains, including overall QoL, was seen in the intervention group. However, the perception of overall health did not differ between the control and intervention groups. QoL improvement by CBT has been observed in literature: Hofmann et al [45] used a meta-analysis to observe the effect of CBT on QoL in anxiety disorders and found it to be effective. An RCT conducted using an internet-based self-guided stress management intervention for employees, similar to that in this study, also exhibited improvements in QoL [10], but only the mental health component of QoL improved (not the physical health component), which was similar to our findings. Although the physical domain of WHOQOL in our trial showed significant improvements, the effect size was the smallest among the 4 domains, and overall perception of health showed no significant improvement. This may be because improving physical health and related QoL is significantly more complex than improving mental health and related QoL, although possible. Therapies with a meditative component, such as yoga and mindfulness-based cognitive therapy, have been shown to improve health-related QoL [46]. CBT that improves sleep disturbance also induces an improvement in health-related QoL [47]. Furthermore, our program had separate modules dedicated to sleep improvement and meditation. A few participants gave positive feedback on the meditation module, which could have resulted in the mixed health-related QoL improvements. Future programs should emphasize this in CBT for further improvement in physical health.

The social relationship domain of QoL had the most significant effect size (η²=0.14), in addition to our primary result, perceived stress. We attribute this improvement to the helper system that asked participants to find a helper, someone who can talk to and confide in about work. Participants also provided feedback that using this app with a helper allowed them to be more engaged in using the program and made the work environment more enjoyable. Improved social relationships within the workplace are estimated to increase employee well-being and company efficiency [48], constituting one reason for including CBT.

Changes in Depression and Anxiety Levels and Other Characteristics of the Program

Although our initial analysis showed that the BetterLife program significantly improved BDI and BAI levels, this significance did not survive the Bonferroni correction. We assume that this is because of the baseline observation carried forward method used to create the missing data of the participants who dropped out, which is one of the most conservative ways to estimate missing data. Notably, the per-protocol analysis showed that BDI and BAI scores were significantly improved using the BetterLife program with moderate effect size (F=10.15, P=.002, η²=0.08 and F=8.21, P=.0049, η²=0.07, respectively). The program incorporated modules that directly help deal with depression and anxiety, as controlling psychological symptoms is essential in reducing perceived stress. The efficacy of web-based CBTs has been proven in patients with major depressive disorder and anxiety disorder with mild and moderate symptoms [13]. Although the current intervention was aimed at employees with work stress, future trials could be performed on clinical groups to explore the program’s effects.

The dropout rate of the intervention was 14.3%, slightly lower than the average dropout rate of internet-based CBTs according to a recent meta-analysis [49]. This, along with improved stress reduction, work engagement, and QoL, could result from the way the program was tailored for each participant, as described in the intervention section. The use of a helper combined with personalization as the user was guided through the program is a unique feature that makes BetterLife different from other internet-based programs dealing with stress. The feedback from the tests that participants took during the trial might also be a reason for the low degree of dropouts.

Although our study design did not include a direct comparison with face-to-face CBT, recent research comparing internet-based and face-to-face CBT showed similar results. Peter et al [14] showed similar efficacy in treating insomnia when comparing an online CBT with face-to-face CBT. A recent randomized noninferiority clinical trial showed online CBT to be noninferior to face-to-face CBT for health anxiety [50]. A systemic review and meta-analysis comparing online CBT and face-to-face CBT in psychiatric and somatic disorders reported that most disorders showed equivalent effects between the two groups [49]. These results emphasize the opinion that online CBT has become a viable and cost-effective option for face-to-face CBT, possibly because of advancements in technology and our familiarity with it. Unfortunately, we could not find any studies comparing online CBT with face-to-face CBT for stress reduction. Future studies are needed for stress management in this context.

Limitations

Our study has several limitations. First, only psychological scales were used in the trial. Although using objective measurements such as heart rate variance or blood cortisol levels would have complemented the results, we avoided this so that the participants would not have to visit a psychiatric clinic. Stigma regarding psychiatric interventions remains widespread in Korea, and having participants with work stress visit a psychiatric ward could negatively impact the trial [51]. Second, the average perceived stress was relatively high in both groups. Generalizing this result to a population with low perceived stress would be difficult with the current data. Third, most participants were female, and most had an education level of college diploma or higher. Although there are similar findings regarding demographics [12,46], this should be accounted for when applying the results to a broader population. Finally, only pre- and postintervention data were collected. Additional data from months after the intervention are needed to examine if the effects of the intervention persist over time.

Conclusions

BetterLife, a smartphone-based individually tailored CBT, effectively reduced stress and increased work engagement and QoL in people with work-related stress. This is a viable option for reducing workplace stress.