This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3114) Anxiety and Stress Disorders (1005) Behavior Change (653) Depression and Mood Disorders; Suicide Prevention (1282)

Published on in Vol 22, No 10 (2020): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/21336, first published .
Twelve-Month Follow-Up to a Fully Automated Internet-Based Cognitive Behavior Therapy Intervention for Rural Adults With Depression Symptoms: Single-Arm Longitudinal Study

Twelve-Month Follow-Up to a Fully Automated Internet-Based Cognitive Behavior Therapy Intervention for Rural Adults With Depression Symptoms: Single-Arm Longitudinal Study

Twelve-Month Follow-Up to a Fully Automated Internet-Based Cognitive Behavior Therapy Intervention for Rural Adults With Depression Symptoms: Single-Arm Longitudinal Study

Authors of this article:

Mark Schure1 Author Orcid Image ;   Bernadette McCrory2 Author Orcid Image ;   Kathryn Tuchscherer Franklin1 Author Orcid Image ;   John Greist3 Author Orcid Image ;   Ruth Striegel Weissman3 Author Orcid Image
Article Authors Cited by (8) Tweetations (18) Metrics

    Original Paper

    1Department of Health & Human Development, Montana State University, Bozeman, MT, United States

    2Department of Mechanical & Industrial Engineering, Montana State University, Bozeman, MT, United States

    3Center for Mental Health Research and Recovery, Montana State University, Bozeman, MT, United States

    Corresponding Author:

    Mark Schure, BSc, MSc, PhD

    Department of Health & Human Development

    Montana State University

    305 Herrick Hall

    Bozeman, MT, 59717

    United States

    Phone: 1 406 994 3248

    Email: mark.schure@montana.edu


    Related ArticleThis is a corrected version. See correction statement in: https://www.jmir.org/2020/10/e25146/

    Background: Internet-based cognitive behavior therapy (iCBT) interventions have the potential to help individuals with depression, regardless of time and location. Yet, limited information exists on the longer-term (>6 months) effects of iCBT and adherence to these interventions.

    Objective: The primary aim of this study was to evaluate the longitudinal (12 months) effectiveness of a fully automated, self-guided iCBT intervention called Thrive, designed to enhance engagement with a rural population of adults with depression symptoms. The secondary aim was to determine whether the program adherence enhanced the effectiveness of the Thrive intervention.

    Methods: We analyzed data from 181 adults who used the Thrive intervention. Using self-reports, participants were evaluated at baseline, 8 weeks, 6 months, and 12 months for the primary outcome of depression symptom severity using the Patient Health Questionnaire-9 (PHQ-9) scale and secondary outcome measures, namely, the Generalized Anxiety Disorder Scale-7 (GAD-7) scores, Work and Social Adjustment Scale (WSAS) scores, Conner-Davidson Resilience Scale-10 (CD-RISC-10) scores, and suicidal ideation (ninth item of the PHQ-9 scale) scores. The Thrive program adherence was measured using the numbers of program logins, page views, and lessons completed.

    Results: The assessment response rates for 8-week, 6-month, and 12-month outcomes were 58.6% (106/181), 50.3% (91/181), and 51.4% (93/181), respectively. By 8 weeks, significant improvements were observed for all outcome measures. These improvements were maintained at 12 months with mean reductions in severities of depression (mean –6.5; P<.001) and anxiety symptoms (mean –4.3; P<.001). Improvements were also observed in work and social functioning (mean –6.9; P<.001) and resilience (mean 4.3; P<.001). Marked decreases were observed in suicidal ideation (PHQ-9 ninth item score >1) at 6 months (16.5%) and 12 months (17.2%) compared to baseline (39.8%) (P<.001). In regard to the program adherence, cumulative counts of page views and lessons completed were significantly related to lower PHQ-9, GAD-7, and WSAS scores and higher CD-RISC-10 scores (all P values <.001 with an exception of page views with WSAS for which P value was .02).

    Conclusions: The Thrive intervention was effective at reducing depression and anxiety symptom severity and improving functioning and resilience among a population of adults from mostly rural communities in the United States. These gains were maintained at 1 year. Program adherence, measured by the number of logins and lessons completed, indicates that users who engage more with the program benefit more from the intervention.

    Trial Registration: ClinicalTrials.gov NCT03244878; https://clinicaltrials.gov/ct2/show/NCT03244878

    J Med Internet Res 2020;22(10):e21336

    doi:10.2196/21336

    Keywords

    internet-based cognitive behavior therapy; depression; anxiety; long-term outcomes; iCBT; CBT; therapy; mental health; outcome 


    Clinician-delivered cognitive behavior therapy (CBT) is a long-standing evidence-based psychotherapy for depression and anxiety symptoms and disorders [1,2]. Computerized forms of CBT were introduced three decades ago, paving the way for present efforts to implement such programs via the internet [3]. Compared to clinician-delivered CBT, internet-based cognitive behavior therapy (iCBT) programs have potential for greater reach and scalability, greater standardization of content delivery, and reduced risk of stigmatization [4-6]. Even more, they have demonstrated equivalent effectiveness for reducing depression and anxiety symptoms [7].

    Studies support the feasibility, acceptability, and effectiveness of self-guided (no supportive contacts by email, text, telephone, or face-to-face) iCBT interventions on depression and anxiety symptoms [4,6,8-10]. These findings are particularly promising for people living in rural and frontier communities, which, nationally and internationally, have greater behavioral health care access challenges [11-13]. Compared with urban residents, rural and frontier residents have fewer qualified mental and behavioral health care providers and longer travel times to clinical services; moreover, they report greater concerns about privacy and higher levels of stigma [8,14-16]. Outside of the United States, other identified barriers to care in rural settings include long wait times for appointments, cost of care, transportation, lack of education, and stigma toward seeking mental health care [17,18].

    A meta-analysis of 13 randomized controlled trials (RCTs), all conducted outside the United States [4], evaluated the efficacy of self-guided iCBT interventions for adults with depression symptoms. Compared to controls, iCBT was significantly more effective for the intervention groups on depressive symptoms severity. Furthermore, program adherence was significantly associated with reduction in depression symptoms. Another meta-analysis of 64 RCTs evaluated the effectiveness of iCBT, usually delivered with support from a health care provider or other individual, compared to the usual care, wait list, or placebo intervention in the treatment of depression or anxiety [6]. The meta-analysis reported superior outcomes among the intervention group compared to the control group. Most self-guided iCBT trials with outcomes of depression and anxiety have shorter follow-up assessment periods (≤6 months). We identified only 2 studies reporting longer-term effects of completely self-guided iCBT on depression symptoms. In Mira and colleagues’ trial [19], 12-month effect sizes for depression symptoms demonstrated a moderate within-group effect size (d= 0.67). Clarke and colleagues [20] reported on an 8-month follow-up on depression symptoms, indicating a comparable effect size of 0.72 (calculated by authors).

    In the context of online programming for mental health interventions, adherence has been defined as “the extent to which individuals experience or engage with content” [21]. However, operationalizing adherence to iCBT interventions poses challenges. For example, while in-person CBT typically involves treatment sessions (with a mutually agreed upon start and end time), electronic delivery of treatment has varying degrees of engagement and length of use. The content of iCBT may be designed to comprise numerous short videos that can be viewed in a variety of temporal orders and explored in a variety of timeframes different from what would typically be encountered in an in-person session. Hence, the concept of a session may not easily translate from in-person CBT to iCBT or from one iCBT program to another.

    Most published RCTs evaluating the efficacy of iCBT interventions have been implemented in non-US urban settings. Our research team evaluated an iCBT intervention, called Thrive, designed to help improve depression and anxiety symptoms for adults residing in the western rural communities in the United States [8,9]. Thrive is an interactive, fully automated, self-guided intervention that uses a video-based platform to deliver CBT curriculum and provide supportive feedback to users [22]. We implemented a pragmatic usual care waitlist control (WLC) RCT, enrolling 343 adults with at least mild depression symptoms (a Patient Health Questionnaire-9 [PHQ-9] score >5), who were randomized either to immediate access to the Thrive intervention or to 8-week delayed access [9]. In models adjusted for potential confounders, depression severity following 8 weeks of intervention was significantly lower for the immediate access group than for the WLC group (d= 0.63). Superior 8-week outcomes in the immediate access group versus WLC group were also observed for secondary outcomes, including anxiety symptoms (d=0.47), work and social functioning (d=0.39), and resilience (d=0.55). Although not statistically significant, the immediate access group was 45% less likely (odds ratio [OR] 0.55) to report suicidal ideation compared to the WLC group.

    Since there are few studies examining the long-term impact of self-guided iCBT interventions, the primary aim of this within-group analysis was to assess 6- and 12-month follow-up outcomes of trial participants who received immediate access to the Thrive intervention. Moreover, to date, there has been no consensus on the operationalization or impact of adherence on outcomes in iCBT interventions [21]. Thus, our secondary aim was to determine whether program adherence enhanced the effectiveness of the Thrive intervention.


    Trial Design

    A longitudinal study design was used to evaluate the effectiveness of the Thrive intervention with participants receiving immediate access to the iCBT program. Study participants were enrolled in the study between September 2017 and January 2018 and were provided free online access to the intervention for 1 year. The original study design was a waitlist RCT in which participants were randomly assigned to receive either an immediate access to the iCBT intervention or access to the intervention delayed 8 weeks. The WLC group received a link to the National Institute of Mental Health’s depression information webpage. All participants received a link to the resource webpage of Montana’s chapter of the National Alliance on Mental Illness. All participants were also permitted to continue or begin whatever usual care was available to them. The Montana State University Institutional Review Board (IRB) approved the protocol and all related materials (#MS033017-FC) prior to study initiation. The study is registered at ClinicalTrials.gov (NCT03244878).

    Study Recruitment

    We used several strategies for promoting the study to state residents. We first partnered with Montana State University Extension faculty to disseminate study brochures and flyers that guided potential participants to the study website, where they could learn more and sign up. Research team members also conducted 12 community meetings throughout all regions of the state, which were promoted by local extension agents. Other recruitment sources included public service announcements, local newspaper articles, social media (Facebook and Craigslist community pages), email listservs, large employers, and local health care providers.

    Study Eligibility and Participants

    Eligibility requirements for the study included age >18 years; Montana state residency; having regular access to broadband internet via a computer, tablet, or smartphone; and reporting at least mild depression symptoms (PHQ-9 score >5) at baseline. Potential participants were directed to a study website where they were informed about study participation; self-screened for eligibility; and, if eligible, guided through the informed consent, randomization, and online assessment process [23]. The Montana State University logo was displayed throughout the study website pages. Of the 573 individuals assessed for eligibility, 463 individuals were deemed study eligible and enrolled in the study; yet 109 were later identified as fraudulent and removed from the study. The sample for the current study included 181 eligible study participants who had immediate access to the Thrive intervention (see Multimedia Appendix 1 for the CONSORT (Consolidated Standards of Reporting Trials) flowchart).

    Intervention Description

    Thrive, developed by Waypoint Health Innovations [22], is a self-guided iCBT intervention for depression and anxiety that distills best practices from CBT and delivers them through a rich, structured, and guided curriculum. Thrive uses video, interactive tools, and sophisticated algorithms that dynamically adjust the individual’s course through the intervention. The intervention is comprised of 320 videos, averaging 80 seconds in length, to deliver content. Videos explain CBT concepts, demonstrate skills, provide feedback and recommendations, and portray actual case histories of individuals who used CBT skills to improve depression symptoms. The intervention also provides periodic PHQ-9 self-assessments and tailored feedback based on the scores. For this study, over a third of the demonstration and case history videos were replaced with new videos featuring rural characters, story lines, and settings. Other features of the Thrive program (ie, didactic and feedback videos, interactive tools, and algorithms) were not modified for this study. Thrive incorporates classic cognitive behavior therapy themes in modules (series of the didactic and feedback videos and interactive tools) on Constructive Thinking (cognitive restructuring), Pleasant Activities (behavioral activation), and Assertive Communication (social skills training). Each module has 10 lessons and suggested exercises for users to practice offline as homework pertinent to their own goals.

    The cost for an individual to use Thrive for 6 months is roughly equivalent to the cost of 1 session with a therapist.

    Assessments

    All participants were assessed at baseline, 8 weeks, 6 months, and 12 months after study enrollment for each outcome measure. Each participant received email reminders when assessments were due, and 2 additional reminders within 7 days were issued for those who had not yet completed their assessment. Data were designated as lost to follow-up when no assessment was completed within 10 days of the due date. Participants with completed interim assessments were rewarded with a US $25 Amazon gift code, and those with the completed final assessment (12 months) were rewarded with a US $30 Amazon gift code.

    Measures

    All outcome measures and other demographic and treatment measures were administered electronically via the study assessment portal. The primary outcome measure was depression symptom severity measured by the PHQ-9, (score range 0-27; higher scores indicate greater severity) [24]. Secondary outcome measures included anxiety symptom severity, daily functioning, resilience, and suicidal ideation. Anxiety symptom severity was measured with the Generalized Anxiety Disorder Scale-7 (GAD-7) (score range 0-21; higher scores indicate greater severity) [25]. Daily functioning was measured with the Work and Social Adjustment Scale (WSAS) (score range 0-40; higher scores indicate worse daily functioning) [26]. Resilience was measured with the Connor-Davidson Resilience Scale-10 (CD-RISC-10) (score range 0-40; higher scores indicate greater resilience) [27]. Frequency of suicidal ideation was measured with the ninth item of the PHQ-9 (score range 0-3; higher scores indicate greater suicidal ideation).

    We assessed rates of remission and relapse. Remission was defined as a treatment response in which an individual with mild, moderate, moderately severe, or severe depression at baseline (PHQ-9 scores ≥5) achieved a subsequent PHQ-9 score lower than 5 at 6 months and 12 months. Relapse was defined as a PHQ-9 score ≥10 for those who had achieved remission at 6 months or 12 months. Cumulative program adherence was measured with the following indicators: (1) number of logins, (2) number of page views, and (3) number of lessons completed within the program. Cumulative counts of each of these 3 program usage measures were assessed as explanatory variables for each of the 4 continuous outcome measures (PHQ-9, GAD-7, WSAS, CD-RISC-10) and the PHQ-9 ninth item, suicidal ideation.

    Demographic variables included age (years), gender (female vs male), race (White vs other), marital status (single vs married/domestic relationship), employment status (employed full-time, employed part-time, unemployed/retired/student), veteran status (yes vs no), educational attainment (≤ college without a degree, bachelor’s degree, ≥ master’s degree), health insurance (private, public, other, none), and rural classification (urban, large rural, small rural, isolated).

    Participant Safety

    Participants were encouraged to seek or continue other available care throughout the study. During all assessments, participants who reported any frequency of suicidal ideation (PHQ-9 ninth item score >0) on the assessment portal were encouraged to seek help from multiple sources. They were also asked whether they could keep themselves safe from self-harm. Those responding they could not keep themselves safe would be told not to continue in the study and were provided a list of things to do to seek professional help. All participants were provided a resource list for seeking additional support (Multimedia Appendix 2). However, none of the participants responded they could not be safe during any assessment. Additionally, in the Thrive intervention, any self-assessed PHQ-9 scores >20 and a PHQ-9 score >10 on the third self-assessment recommended seeking a clinician’s help. Participants were also provided contact information of the institution’s IRB director and the study’s principal investigator to report any adverse events. No adverse events were reported in this study.

    Statistical Analysis

    The longitudinal change over time in each continuous outcome was assessed using a linear mixed model analysis of repeated measures. Separate models were created for each outcome measure (ie, PHQ-9, GAD-7, WSAS, and CD-RISC-10) to assess the fixed effect of time adjusting for baseline scores and receiving therapy for depression. Similar separate models were used to assess the relationship program logins, page views, and lessons completed on each outcome, adjusting for baseline scores, therapy for depression, and time.

    The PHQ-9 ninth item, suicidal ideation, was treated as an ordinal outcome that ranged from 0 (“not at all,” no suicidal ideation) to 3 (“nearly every day”) where the cumulative probabilities were modeled over the higher-ordered suicidal ideation scale scores (indicating greater suicidal ideation). Using an ordinal logistic regression model within a generalized estimating equation framework, the PHQ-9 ninth item was assessed with the fixed effect of time adjusting for baseline scores and receiving therapy for depression. Similar separate ordinal logistic regression models were used to assess the relationship program logins, page views, and lessons completed, adjusting for baseline scores, therapy for depression, and time.

    Statistical analyses were performed using SAS software, version 9.4 (SAS/STAT 14.2, SAS Institute Inc). Maximum likelihood estimators allow efficient parameter estimation using only available data under an assumption of missing at random [28-30]. Sensitivity analyses were also conducted using only participants with complete data (ie, complete cases), and no significant differences were found between those who were lost to follow-up/noncompleters and those who completed the trial. Attrition/loss to follow-up was assessed to ensure the key covariates, and baseline measures did not differ from those that completed and those that did not complete the trial. The baseline primary and secondary outcomes each showed no significant differences between trial completers and noncompleters. There were no significant differences found among completers for each outcome variable of interest based on age, gender, marital status, employment, rural-urban commuting area codes, or baseline therapy usage. Due to extremely low sample sizes, variables of race, veteran status, and other or no insurance could not be assessed. The level of significance was set at α=.05 (two-tailed), and the Bonferroni method was implemented to control false positives over the multiple tests.


    Participant Characteristics

    A total of 181 immediate intervention group participants (iCBT) were included in this longitudinal outcome assessment. As detailed in Table 1, participants were on average 42 years old (SD 12.8); and most were female (88.9%), White (93.9%), and nonveterans (96.1%). A majority was married or in a domestic relationship (56.9%), employed full-time (61.9%), had obtained at least a bachelor’s degree (56.9%), and had private health insurance (77.3%). Nearly 15% of participants lived in urban, over 56% in rural, and nearly 29% in isolated communities. Nearly 57% of participants reported receiving clinical care for mental health.

    Table 1. Baseline characteristics of the analytic sample (N=181).
    CharacteristicsValues
    Age (years), mean (SD)42.1 (12.8)
    Female, n (%)160 (88.4)
    Race, n (%)

    		White170 (93.9)

    		Other11 (6.1)
    Marital status, n (%)

    		Single78 (43.1)

    		Married/domestic partnership103 (56.9)
    Employment status, n (%)

    		Employed full-time112 (61.9)

    		Employed part-time39 (21.6)

    		Unemployed/retired/student30 (16.8)
    Veteran, n (%)7 (3.9)
    Education, n (%)

    		Some college or less78 (43.1)

    		Bachelor\'s degree61 (33.7)

    		Graduate or professional degree42 (23.2)
    Health insurance, n (%)

    		Private140 (77.3)

    		Public32 (17.7)

    		Other5 (2.8)

    		None4 (2.2)
    Rural classificationa, n (%)

    		Urban27 (14.9)

    		Large rural42 (23.2)

    		Small rural60 (33.2)

    		Isolated52 (28.7)
    Receiving mental health treatmentb, n (%)103 (56.9)

    aDefined using the rural-urban commuting area codes.

    bDefined as receiving any clinical care or taking medication(s) for depression symptoms.

    Clinical Outcomes

    The assessment response rates for the 8-week, 6-month, and 12-month outcomes were 58.6%, 50.3%, and 51.4%, respectively. Thus, the respective attrition rates were 41.4%, 49.7%, and 48.6%. To assess remission and relapse, participants’ PHQ-9 scores were assessed for changes from baseline to 8 weeks, 6 months, and 12 months. Of the 107 participants who completed week 8 assessments, 42 (39.3%) achieved remission. Among the 42 participants who achieved remission, 22 (52.4%) and 24 (57.1%) maintained remission at 6 months and 12 months, respectively. Only 4 participants (9.5%) who had achieved remission at week 8 subsequently relapsed at 6 months (n=2) and 12 months (n=2).

    Further, 72.9% (78/107) participants had moderate or greater depression symptoms at baseline (PHQ-9 score ≥10). From this subgroup of 78, 28 (35.9%) achieved remission at week 8. Of these 28 participants who achieved remission within this subgroup, 11 (39.3%) and 13 (46.4%) maintained remission at 6 months and 12 months, respectively. Only 4 (14.2%) participants who had achieved remission at week 8 subsequently relapsed at 6 months (n=2) and 12 months (n=2).

    Longitudinal mean outcome scores and adherence metrics are presented in Table 2.

    Table 2. Longitudinal mean trend of clinical and adherence measures.
    MeasuresBaseline, mean (SD)8 weeks, mean (SD)6 months, mean (SD)12 months, mean (SD)Significancea






    		F test (df)P value (adj)
    Primary outcome measure (Depression symptom severity)b13.7 (5.0)7.2 (5.1)7.0 (5.4)7.2 (5.6)121.6 (3)<.001
    Secondary outcome measures

    		Anxiety symptom severityc10.3 (4.7)6.4 (4.7)6.2 (4.8)6.0 (5.4)50.4 (3)<.001

    		Work and social functioningd20.2 (8.0)15.2 (9.4)13.7 (9.7)13.3 (10.5)29.1 (3)<.001

    		Resiliencee22.2 (6.6)25.6 6.1)26.1 (6.3)26.5 (6.7)28.9 (3)<.001

    		Suicidal ideationf (score ≥1), n (%)72 (39.8)13 (12.2)15 (16.5)16 (17.2)48.9 (3)g<.001h
    Adherence measures

    		Logins (cumulative counts)N/Ai8.9 (9.6)11.1 (13.3)11.7 (14.3)N/AN/A

    		Page views (cumulative counts)N/A66.3 (70.4)80.9 (89.1)84.8 (93.5)N/AN/A
     Lessons completed (cumulative counts)N/A7.7 (7.9)8.9 (9.1)9.2 (9.3)N/AN/A

    aP value was associated with the test (F test, Type 3 tests of fixed effects) of the overall time period (week) difference. P values adjusted by the Bonferroni method (adj) are shown.

    bPatient Health Questionnaire-9 (PHQ-9) score range = 0 to 27.

    cGeneralized Anxiety Disorder Scale-7 (GAD-7) score range = 0 to 21.

    dWork and Social Adjustment Scale (WSAS) score range = 0 to 40.

    eConnor-Davidson Resilience Scale-10 (CD-RISC-10) score range = 0 to 40.

    fPHQ-9 Item 9, Suicidal Ideation, score range = 0 to 3.

    gChi-square statistic, χ2 (degrees of freedom).

    hP value was associated with the test (χ2, Type 3 generalized estimating equations analysis) of the overall time period (week) difference. P value adjusted by the Bonferroni method (adj) is shown.

    iN/A: not applicable.

    Mean effect of treatment over time by outcome measure with 95% CI is illustrated in Figure 1.

    By 8 weeks, significant improvements were observed for all outcome measures. These improvements were maintained at 6 and 12 months. We observed 6-month mean reductions and respective effect sizes in the severity of depression (mean –6.3; d=1.27) and anxiety symptoms (mean –4.1; d=0.86). Improvements were also observed in work and social functioning (Mean –6.5; d=0.73) and resilience (mean 3.9; d=0.60). A total of 23% fewer participants endorsed suicidal ideation (PHQ-9 ninth item score >1) at 6 months (16.5%) and 12 months (17.2%) compared to baseline (39.8%).

    We observed 12-month mean reductions and respective effect sizes in the severity of depression (mean –6.5; d=1.23) and anxiety symptoms (mean –4.3; d=0.93). Improvements were also observed in work and social functioning (mean –6.9; d=0.76) and resilience (mean 4.3; d=0.62). Marked decreases were observed on suicidal ideation (PHQ-9 ninth item score >1) from baseline (39.8%) to 6 months (16.5%) and 12 months (17.2%). Longitudinal trends from baseline for all outcome measures were statistically significant (P<.001).

    Table 3 presents the effects of program adherence on each clinical measure. The number of lessons completed was significantly associated to lower PHQ-9 (P<.001), GAD-7 (P<.001), and WSAS scores (P<.001) and higher CD-RISC-10 scores (P<.001). The number of page views was significantly associated to lower PHQ-9 (P<.001), GAD-7 (P<.001), and WSAS scores (P=.02) and higher CD-RISC-10 scores (P<.001) The number of logins was significantly associated only with the PHQ-9 (P=.02). No adherence metrics were significantly associated with suicidal ideation.

    ‎
    Figure 1. Mean effect of treatment over time by clinical measure with 95% CI. CD-RISC-10: Conner-Davidson Resilience Scale-10; GAD-7: Generalized Anxiety Disorder scale-7; PHQ-9: Patient Health Questionnaire-9; WSAS: Work and Social Adjustment Scale.
    View this figure
    Table 3. Program adherence metric effects on outcome measures.
    Adherence metricPHQ-9aGADS-7bWSAScCD-RISC-10dPHQ-9, item 9e

    		F test (df)P value (adj)fF test (df)P value (adj)fF test (df)P value (adj)fF test (df)P value (adj)fχ2 (df)P value (adj)g
    Logins7.89 (1).023.62 (1).231.00 (1)>.991.09 (1)>.991.18 (1)>.99
    Page views31.3 (1)<.00126.95 (1)<.0017.72 (1).0222.81 (1)<.0014.83 (1).11
    Lessons completed41.42 (1)<.00145.65 (1)<.00114.72 (1)<.00131.69 (1)<.0014.57 (1).13

    aPatient Health Questionnaire-9 (PHQ-9) score range = 0 to 27.

    bGeneralized Anxiety Disorder Scale-7 (GAD-7) score range = 0 to 21.

    cWork and Social Adjustment Scale (WSAS) score range = 0 to 40.

    dConnor-Davidson Resilience Scale (CD-RISC 10) score range = 0 to 40.

    ePHQ-9 Item 9, Suicidal Ideation, score range = 0 to 3.

    fP value was associated with the test (F test, Type III Tests of Fixed Effects) of the overall time period (week) difference. P values adjusted by the Bonferroni method (adj) are shown.

    gP value was associated with the test (χ2, Type 3 generalized estimating equations analysis) of the overall time period (week) difference. P values are adjusted by the Bonferroni method (adj) are shown.


    Principal Results

    This study evaluated the long-term (6- and 12-month) outcomes of a fully automated self-guided (no supportive contacts by email, text, telephone, or face-to-face) video-centric iCBT intervention called Thrive in a rural US community. We believe this is the first study to assess the long-term impacts of an iCBT program within an adult population in rural United States. These analyses focused on participants receiving immediate access to the Thrive intervention. Over the course of 8 weeks, over a third of participants with 8-week data achieved remission, and over half of those maintained remission at 6 months and 12 months. A very low relapse rate was one of the noteworthy observations of this study. In regard to long-term outcomes of primary and secondary measures, study findings demonstrated mean improvements in depression and anxiety symptoms, work and social functioning, and resilience from baseline to 8 weeks. These improvements were sustained at 6 and 12 months. Comparable sustained improvements were also observed with decreased percentages of participants reporting suicidal ideation.

    In regard to our adherence analyses, the number of page views and the number of lessons completed most consistently predicted greater sustained positive effects on all outcome measures. Both page views and lessons completed are reasonable markers to assess the program adherence with self-guided iCBT interventions like Thrive, as they indicate the extent to which the users progress through the program. In contrast, the number logins and progress through the program will expectedly vary because some users tend to spend more blocks of time in the program compared to others.

    Comparison With Prior Work

    There is limited evidence on the long-term impacts of self-guided iCBT interventions. To our knowledge, only a few studies have examined long-term effects on depression. Mira and colleagues’ study [19] with adults residing in Spain found similar 12-month sustained effects of an iCBT intervention. Clarke and colleagues’ study [20] examined 8-month effects of an iCBT intervention on depression among members of the US Pacific Northwest Kaiser Permanente organization.

    Substantial evidence exists regarding CBT’s short- and long-term effectiveness for depression and anxiety [31-33]. A growing number of studies report evidence of equivalent effectiveness of iCBT compared with clinician-delivered CBT [7]. Lorenzo-Luaces and colleagues’ [34] meta-analysis concluded comparable effectiveness of self-guided iCBT with that of antidepressants and in-person psychotherapy. In our trial, Thrive’s 8-week RCT between group (intervention vs control) effect size of 0.63 compared well with “mostly 8-week clinical trials of antidepressants for adults with unipolar major depression” between group (antidepressant vs placebo) effect sizes of 0.27 (Hamilton Depression Rating Scale) and 0.30 (Montgomery-Asberg Depression Rating Scale) in 109 antidepressant medication RCTs [35]. For longer-term outcomes, 12-month relapse rates from remission for Thrive at 8 weeks and STAR*D Phase I citalopram at 12 weeks [36] are 14.2% versus 33.5%, respectively. This comparison includes only Thrive participants with PHQ-9 baseline score 10, consistent with a diagnosis of major depressive disorder.

    Adherence in iCBT needs further exploration. In Beintner and colleagues’ review [21], most studies (85%) reported at least 1 adherence indicator, yet adherence metrics varied widely across this literature. The 10 most commonly reported adherence metrics were as follows: full intervention completion; completion of a minimum number of sessions/modules; average number of completed sessions/modules; specified point of discontinuation of the intervention (“dropout”); dropout without specifying a time point; number of participants who were randomized to an intervention group but never logged on; number of times a participant logged on to access the intervention program; total time spent on the program; number of entries into a diary; and number of messages sent to a coach. Recognizing that the specific metric(s) chosen will need to be appropriate to the specific components or delivery formats of online interventions, Beintner and colleagues [21] further recommend the use of multiple adherence metrics, which all studies should report on adherence, providing detailed information on its operationalization.

    In our study, we analyzed data on the number of logins, page views, and lessons completed. Page views and lessons completed were the consistent significant predictors of our outcome measures with the exception of suicidal ideation. Cuijpers and colleagues’ [33] meta-analysis of CBT depression studies assessed the number of sessions (comparable to lessons completed in Thrive) as an adherence measure to determine a dose-response effect. In contrast to our findings, they found no significant relationship with study effect sizes. Given the relative infancy of iCBT platforms, it is imperative that future studies critically devise relevant adherence metrics that fit the type of medium.

    Limitations

    Our findings need to be considered in light of several limitations. As commonly observed in iCBT studies, assessment completion rates were low with 50.3% and 51.4% of participants completing assessments at 6 and 12 months, respectively. Thus, our results may be skewed due to underlying responder biases. Within-group analyses are limited in that there is no control group with which to compare findings. Relying solely on self-assessments, a common practice in iCBT studies, is a potential weakness of the study; however, the use of validated, widely used instruments largely addressed this issue. The PHQ-9, GAD-7, and WSAS measures correlate well with clinician-administered instruments [24,25,37]; furthermore, they have been shown to be sensitive to treatment effects [26,38,39]. Additionally, self-assessments may underestimate the effect of iCBT compared to clinician-administered assessments [40]. As a community-based trial, our findings cannot be generalized to health care settings. In regard to adherence metrics, we limited our analyses to include the number of logins, page views, and lessons completed. Our study was not originally designed as a dose-response analysis; and therefore, our findings are limited to our post hoc analyses of adherence.

    Conclusions

    To our knowledge, this study is the first to demonstrate both short- and long-term positive impacts of a self-guided iCBT intervention on depression and anxiety symptoms, work and social functioning, and resilience among rural adults in the United States. iCBT interventions, such as Thrive, have the potential to provide help and clinical benefit to people with substantial barriers to traditional forms of care; they also present a cost-effective alternative [41] with additional benefits of confidentiality that may not be possible for those seeking traditional care in small rural communities. That iCBT interventions have the potential for greater accessibility, privacy, and affordability in many rural areas compared to in-person psychological treatment is particularly encouraging. Further research is warranted to identify effective and cost-effective dissemination strategies for expanded reach and to understand adoption patterns of iCBT and other internet-based therapies in rural American communities for ensuring optimal uptake of these promising interventions [42].

    Acknowledgments

    Research reported in this publication was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Numbers P20GM103474, U54GM115371, and 5P20GM104417. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors are grateful to Dr Bill Bryan for his tireless efforts to publicize Thrive across the state of Montana and Dr Paul Crits-Christoph of the University of Pennsylvania for his guidance.

    Conflicts of Interest

    JG previously held a financial interest in Waypoint Health Innovations, which developed the Thrive intervention evaluated in this work. Waypoint Health Innovations pays him a royalty based on revenue from Thrive use. He no longer has a direct financial interest in Waypoint Health Innovations but does retain a small interest in Waypoint Health Innovations through Healthcare Technology Systems where he is CEO and a shareholder. He is also a consultant to Waypoint on projects outside of the grant supporting this study. The terms of JG’s financial relationship with Waypoint Health Innovations have been reviewed by Montana State University, and his involvement with this research project has been approved in accordance with its conflict of interest policies.

    ‎
    Multimedia Appendix 1

    CONSORT (Consolidated Standards of Reporting Trials) flowchart.

    PDF File (Adobe PDF File), 103 KB
    ‎
    Multimedia Appendix 2

    Safety Protocol.

    PDF File (Adobe PDF File), 114 KB
    ‎
    Multimedia Appendix 3

    Consort-eHealth.

    PDF File (Adobe PDF File), 438 KB
    1. Chand S, Ravi C, Chakkamparambil B, Prasad A, Vora A. CBT for depresssion: What the evidence says. Current Psychiatry 2018;17(9):14-23.
    2. Deacon BJ, Abramowitz JS. Cognitive and behavioral treatments for anxiety disorders: a review of meta-analytic findings. J Clin Psychol 2004 Apr;60(4):429-441. [CrossRef] [Medline]
    3. Selmi PM, Klein MH, Greist JH, Sorrell SP, Erdman HP. Computer-administered cognitive-behavioral therapy for depression. Am J Psychiatry 1990 Jan;147(1):51-56. [CrossRef] [Medline]
    4. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of Self-guided Internet-Based Cognitive Behavioral Therapy in the Treatment of Depressive Symptoms: A Meta-analysis of Individual Participant Data. JAMA Psychiatry 2017 Apr 01;74(4):351-359. [CrossRef] [Medline]
    5. Mohr DC, Hart SL, Howard I, Julian L, Vella L, Catledge C, et al. Barriers to psychotherapy among depressed and nondepressed primary care patients. Ann Behav Med 2006 Dec;32(3):254-258. [CrossRef] [Medline]
    6. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
    7. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
    8. Schure MB, Howard M, Bailey SJ, Bryan B, Greist J. Exploring Perceptions of a Computerized Cognitive Behavior Therapy Program in a U.S. Rural Western State. Rural Ment Health 2018 Jul;42(3-4):174-183. [CrossRef] [Medline]
    9. Schure MB, Lindow JC, Greist JH, Nakonezny PA, Bailey SJ, Bryan WL, et al. Use of a Fully Automated Internet-Based Cognitive Behavior Therapy Intervention in a Community Population of Adults With Depression Symptoms: Randomized Controlled Trial. J Med Internet Res 2019 Nov 18;21(11):e14754 [FREE Full text] [CrossRef] [Medline]
    10. Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health 2017 Jun 06;4(2):e19 [FREE Full text] [CrossRef] [Medline]
    11. Robert WJF. Life in Rural America: Part II. Harvard School of Public Health. 2019.   URL: https://www.rwjf.org/en/library/research/2019/05/life-in-rural-america--part-ii.html [accessed 2020-06-01]
    12. Scheil-Adlung X. Global evidence on inequities in rural health protection: new data on rural deficits in health coverage for 174 countries. World Health Organization. 2015.   URL: https:/​/reliefweb.​int/​report/​world/​global-evidence-inequities-rural-health-protection-new-data-rural-deficits-health [accessed 2020-05-02]
    13. Titov N, Fogliati VJ, Staples LG, Gandy M, Johnston L, Wootton B, et al. Treating anxiety and depression in older adults: randomised controlled trial comparing guided . self-guided internet-delivered cognitive-behavioural therapy. BJPsych Open 2016 Jan;2(1):50-58 [FREE Full text] [CrossRef] [Medline]
    14. Mental Health Care Health Professional Shortage Areas (HPSAs). Kaiser Family Foundation. 2018 Dec 21.   URL: https:/​/www.​kff.org/​other/​state-indicator/​mental-health-care-health-professional-shortage-areas-hpsas [accessed 2019-12-05]
    15. Larson JE, Corrigan PW. Psychotherapy for self-stigma among rural clients. J Clin Psychol 2010 May;66(5):524-536. [CrossRef] [Medline]
    16. Heitkamp T, Nielsen S, Schroeder S. Mountain Health Technology Transfer Network. Promoting positive mental health in rural schools. 2019 Aug 12.   URL: https:/​/mhttcnetwork.​org/​sites/​default/​files/​2019-08/​promoting-positive-mental-health-in-rural-schools.​pdf [accessed 2020-08-04]
    17. De Silva T, Prakash A, Yarlagadda S, Johns MD, Sandy K, Hansen V, et al. General practitioners' experiences and perceptions of mild moderate depression management and factors influencing effective service delivery in rural Australian communities: a qualitative study. Int J Ment Health Syst 2017;11:54 [FREE Full text] [CrossRef] [Medline]
    18. Hull MJ, Fennell KM, Vallury K, Jones M, Dollman J. A comparison of barriers to mental health support-seeking among farming and non-farming adults in rural South Australia. Aust J Rural Health 2017 Dec;25(6):347-353. [CrossRef] [Medline]
    19. Mira A, Bretón-López J, García-Palacios A, Quero S, Baños RM, Botella C. An Internet-based program for depressive symptoms using human and automated support: a randomized controlled trial. Neuropsychiatr Dis Treat 2017;13:987-1006 [FREE Full text] [CrossRef] [Medline]
    20. Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleher C, et al. Overcoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res 2002 Dec;4(3):E14 [FREE Full text] [CrossRef] [Medline]
    21. Beintner I, Vollert B, Zarski A, Bolinski F, Musiat P, Görlich D, et al. Adherence Reporting in Randomized Controlled Trials Examining Manualized Multisession Online Interventions: Systematic Review of Practices and Proposal for Reporting Standards. J Med Internet Res 2019 Aug 15;21(8):e14181 [FREE Full text] [CrossRef] [Medline]
    22. Thrive. Waypoint Health Innovations. 2018.   URL: https://waypointhealth.com/thrive/ [accessed 2020-07-01]
    23. Feeling down, anxious, or stressed? Free online help is here for Montanans. Thrive for Montana. 2018 Feb 28.   URL: https://thriveformontana.com [accessed 2020-07-01]
    24. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
    25. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    26. Mundt JC, Marks IM, Shear MK, Greist JH. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. Br J Psychiatry 2002 May;180:461-464. [CrossRef] [Medline]
    27. Campbell-Sills L, Stein MB. Psychometric analysis and refinement of the Connor-davidson Resilience Scale (CD-RISC): Validation of a 10-item measure of resilience. J Trauma Stress 2007 Dec;20(6):1019-1028. [CrossRef] [Medline]
    28. Wolfinger R, Chang M. Comparing the SAS GLM and MIXED procedures for repeated measures. In: Comparing the SAS GLMMIXED procedures for repeated measures. Presented at: Proceedings of the Twentieth Annual SAS Users Group Conference; 1998 Presented at: Twentieth Annual SAS User Group; 1998; Cary, NC.
    29. Little RJ. Modeling the Drop-Out Mechanism in Repeated-Measures Studies. Journal of the American Statistical Association 1995 Sep;90(431):1112. [CrossRef]
    30. Rubin DB. Inference and Missing Data. Biometrika 1976 Dec;63(3):581-592. [CrossRef]
    31. Hollon SD, Stewart MO, Strunk D. Enduring effects for cognitive behavior therapy in the treatment of depression and anxiety. Annu Rev Psychol 2006;57:285-315. [CrossRef] [Medline]
    32. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev 2006 Jan;26(1):17-31. [CrossRef] [Medline]
    33. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry 2013 Jul;58(7):376-385. [CrossRef] [Medline]
    34. Lorenzo-Luaces L, Johns E, Keefe JR. The Generalizability of Randomized Controlled Trials of Self-Guided Internet-Based Cognitive Behavioral Therapy for Depressive Symptoms: Systematic Review and Meta-Regression Analysis. J Med Internet Res 2018 Nov 09;20(11):e10113 [FREE Full text] [CrossRef] [Medline]
    35. Hengartner MP, Jakobsen JC, Sørensen A, Plöderl M. Efficacy of new-generation antidepressants assessed with the Montgomery-Asberg Depression Rating Scale, the gold standard clinician rating scale: A meta-analysis of randomised placebo-controlled trials. PLoS One 2020;15(2):e0229381 [FREE Full text] [CrossRef] [Medline]
    36. Rush AJ, Trivedi MH, Wisniewski SR, Nierenberg AA, Stewart JW, Warden D, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry 2006 Nov;163(11):1905-1917. [CrossRef] [Medline]
    37. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care 2008 Mar;46(3):266-274. [CrossRef] [Medline]
    38. Zimmerman M, Walsh E, Friedman M, Boerescu DA, Attiullah N. Are self-report scales as effective as clinician rating scales in measuring treatment response in routine clinical practice? J Affect Disord 2018 Jan 01;225:449-452. [CrossRef] [Medline]
    39. Kertz S, Bigda-Peyton J, Bjorgvinsson T. Validity of the Generalized Anxiety Disorder-7 scale in an acute psychiatric sample. Clin Psychol Psychother 2013;20(5):456-464. [CrossRef] [Medline]
    40. Cuijpers P, Li J, Hofmann SG, Andersson G. Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: a meta-analysis. Clin Psychol Rev 2010 Aug;30(6):768-778. [CrossRef] [Medline]
    41. Thase M, McCrone P, Barrett M, Eells T, Wisniewski S, Balasubramani G, et al. Improving Cost-effectiveness and Access to Cognitive Behavior Therapy for Depression: Providing Remote-Ready, Computer-Assisted Psychotherapy in Times of Crisis and Beyond. Psychother Psychosom 2020;89(5):307-313 [FREE Full text] [CrossRef] [Medline]
    42. Taylor CB, Ruzek JI, Fitzsimmons-Craft EE, Sadeh-Sharvit S, Topooco N, Weissman RS, et al. Using Digital Technology to Reduce the Prevalence of Mental Health Disorders in Populations: Time for a New Approach. J Med Internet Res 2020 Jul 24;22(7):e17493 [FREE Full text] [CrossRef] [Medline]

    ‎
    CBT: cognitive behavior therapy
    CD-RISC-10: Conner-Davidson Resilience Scale-10
    CONSORT: Consolidated Standards of Reporting Trials
    GAD-7: Generalized Anxiety Disorder scale-7
    iCBT: internet-based cognitive behavior therapy
    IRB: institutional review board
    PHQ-9: Patient Health Questionnaire-9
    RCT: randomized controlled trial
    WLC: waitlist control
    WSAS: Work and Social Adjustment Scale

    Edited by G Eysenbach; submitted 11.06.20; peer-reviewed by S Pit, MDG Pimentel, K Mathiasen; comments to author 01.07.20; revised version received 14.08.20; accepted 02.09.20; published 02.10.20

    Copyright

    ©Mark Schure, Bernadette McCrory, Kathryn Tuchscherer Franklin, John Greist, Ruth Striegel Weissman. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 02.10.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.