This paper is in the following e-collection/theme issue:

e-Mental Health and Cyberpsychology (1386) Anxiety and Stress Disorders (999) Behavior Change (652) Innovations in Mental Health Systems (370) Transdiagnostic Mental Health Interventions (120) Depression and Mood Disorders; Suicide Prevention (1271) Telepsychiatry (143) Web-based and Mobile Health Interventions (3092)

Published on in Vol 24, No 3 (2022): March

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/29384, first published .
Internet-Delivered Cognitive Behavioral Therapy for Generalized Anxiety Disorder in Nationwide Routine Care: Effectiveness Study

Internet-Delivered Cognitive Behavioral Therapy for Generalized Anxiety Disorder in Nationwide Routine Care: Effectiveness Study

Internet-Delivered Cognitive Behavioral Therapy for Generalized Anxiety Disorder in Nationwide Routine Care: Effectiveness Study

Authors of this article:

Ville Ritola1 Author Orcid Image ;   Jari Olavi Lipsanen2 Author Orcid Image ;   Satu Pihlaja1 Author Orcid Image ;   Eero-Matti Gummerus1 Author Orcid Image ;   Jan-Henry Stenberg1 Author Orcid Image ;   Suoma Saarni1 Author Orcid Image ;   Grigori Joffe1 Author Orcid Image
Article Authors Cited by (9) Tweetations (10) Metrics

    Original Paper

    1Department of Psychiatry, Helsinki University Hospital and University of Helsinki, Helsinki, Finland

    2Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland

    Corresponding Author:

    Grigori Joffe, MD, PhD, Prof Dr

    Department of Psychiatry

    Helsinki University Hospital and University of Helsinki

    P.O. Box 100

    Helsinki, 00029 HUS

    Finland

    Phone: 358 405136500

    Email: grigori.joffe@hus.fi


    Background: Therapist-supported, internet-delivered cognitive behavioral therapy (iCBT) is efficacious for generalized anxiety disorder (GAD), but few studies are yet to report its effectiveness in routine care.

    Objective: In this study, we aim to examine whether a new 12-session iCBT program for GAD is effective in nationwide routine care.

    Methods: We administered a specialized, clinic-delivered, therapist-supported iCBT for GAD in 1099 physician-referred patients. The program was free of charge for patients, and the completion time was not predetermined. We measured symptoms with web-based questionnaires. The primary measure of anxiety was the GAD 7-item scale (GAD-7); secondary measures were, for pathological worry, the Penn State Worry Questionnaire and, for anxiety and impairment, the Overall Anxiety Severity and Impairment Scale.

    Results: Patients completed a mean 7.8 (SD 4.2; 65.1%) of 12 sessions, and 44.1% (485/1099) of patients completed all sessions. The effect size in the whole sample for GAD-7 was large (Cohen d=0.97, 95% CI 0.88-1.06). For completers, effect sizes were very large (Cohen d=1.34, 95% CI 1.25-1.53 for GAD-7; Cohen d=1.14, 95% CI 1.00-1.27 for Penn State Worry Questionnaire; and Cohen d=1.23, 95% CI 1.09-1.37 for Overall Anxiety Severity and Impairment Scale). Noncompleters also benefited from the treatment. Greater symptomatic GAD-7–measured relief was associated with more completed sessions, older age, and being referred from private or occupational care. Of the 894 patients with a baseline GAD-7 score ≥10, approximately 421 (47.1%) achieved reliable recovery.

    Conclusions: This nationwide, free-of-charge, therapist-supported HUS Helsinki University Hospital–iCBT for GAD was effective in routine care, but further research must establish effectiveness against other treatments and optimize the design of iCBT for GAD for different patient groups and individual patients.

    J Med Internet Res 2022;24(3):e29384

    doi:10.2196/29384

    Keywords

    CBT; iCBT; cognitive behavioral therapy; routine care; generalized anxiety disorder; internet; web-based; digital health; mental health 


    Background

    Generalized anxiety disorder (GAD) [1] is a common but underdiagnosed and undertreated condition [2,3]. The global annual prevalence of GAD in high-income countries may be 2.3%, and this disorder is associated with increased functional impairment [4].

    Pharmacotherapy and psychotherapy are considered first-line GAD treatments [5,6]. Among psychotherapies, cognitive behavioral therapy (CBT) is the most studied, with large effect sizes (Hedges g=0.90 [7]). However, traditional face-to-face CBT is relatively resource-consuming and limited in access.

    The need to resolve the accessibility and affordability challenges of face-to-face CBT led to the development of internet-delivered CBT (iCBT), with notable work occurring in Sweden, the Netherlands, Denmark, Australia, and Canada, for example [8]. A typical iCBT intervention is location-independent and includes a program with therapeutic content and homework. It is accessible 24/7 through a web-based platform or mobile app, with or without therapist support. In therapist-supported iCBT, remote therapist support is most often asynchronous. While the intensity of therapist support in iCBT can vary, it requires far less therapist time compared with face-to-face or real-time remote psychotherapy and thus may be more cost-effective [9,10].

    As demonstrated in recent meta-analyses of randomized controlled trials (RCTs) [9,11-16], therapist-supported iCBTs are efficacious in several psychiatric disorders and may be as efficacious as short-term face-to-face CBT. Therapist-supported iCBTs for depression and anxiety have higher efficacy and adherence than unguided programs do [12,17-19], which is likely due to the high level of client–therapist alliance, comparable with that in face-to-face therapies [20]. In general, iCBTs reduce symptomatic deterioration more than waiting lists (3.1% or 5.8% vs 17.4% [13,21]). For therapist-supported iCBT for GAD, efficacy has also been established for both diagnosis-specific and transdiagnostic programs [9,16].

    However, RCT efficacy studies, being the gold standard for clinical evidence, have their own shortcomings. Participants are more likely to be highly motivated, have had thorough screening, and have received more intensive treatment than in routine care. Therefore, RCT-validated treatments may or may not yield similar results or level of adherence in routine care depending on whether the criteria are lax [22-24]. Addressing these limitations, routine care studies are now accepted as valid scientific evidence for their clinical effectiveness and safety [25]. Although there is strong real-world evidence for the effectiveness of therapist-supported iCBT in general [8,9,13], studies focusing on GAD are limited.

    We identified 5 publications comprising 7 therapist-supported iCBT interventions focusing on GAD in routine care using 3 different programs at 2 clinics. ThisWayUp clinic in Australia performed 6 interventions comprising 2 programs [23,26-28], and Online Therapy Unit in Canada performed 1 intervention [29]. These studies mainly reported large intention-to-treat (ITT) effect sizes (0.91-1.30), although 1 outlier study reported effects as large as 2.06 and 2.10 [28]. ThisWayUp trials reported full completion rates of 36% to 55%. Although these 3 programs demonstrated the overall feasibility of iCBT for GAD, the recruitment schemes, program length, and therapist support intensity in these trials varied widely (see the Design Comparison section in the Discussion section), and the Online Therapy Unit’s trial was relatively small. More routine care studies are needed to confirm their conclusions and elucidate the optimal program design.

    The HUS Helsinki University Hospital has developed and is providing nationwide, original Finnish language therapist-supported iCBT programs for several psychiatric conditions (further referred to as HUS-iCBTs), including one for GAD. Intake for the 12-session program requires a physician’s referral, it is free of charge, and therapist support is provided centrally by a specialized clinic. As the combination of setting and design of the HUS-iCBT for GAD differs from those studied earlier, the effectiveness of these programs may differ.

    Objective

    The aim of this study is to examine the effectiveness of HUS-iCBT for GAD in routine care. We hypothesized that the intervention would have a large overall effect size and the patients’ completion rate would be comparable (36%-55%) with the reported routine care studies.


    Setting and Design

    The HUS-iCBTs were delivered centrally by the iCBT clinic at HUS Psychiatry. The interventions were free of charge, diagnosis-specific, and therapist-supported programs. All physicians in Finland can refer their patients to therapy. The referring physicians received support from the instructions on the web. A specially trained mental health professional screened all referrals centrally. We deliberately implemented these inclusion procedures to enable virtually unlimited nationwide access for the clinical population despite the limited number of therapists.

    The study was an observational, nationwide, open-label, real-world trial.

    Participants

    Participants were recruited from those entering HUS-iCBT for GAD between February 2016 and December 2018. To be accepted for the treatment, patients had to (1) be diagnosed with GAD (ICD-10 [1]), (2) have an email address, and (3) have an internet-based bank account or mobile ID (the means of official e-identification in Finland). Comorbidity (other than the exclusion criteria) and concomitant pharmacological and psychological treatments were allowed, as the intervention was part of routine care. Exclusion criteria were acute psychosis or mania, severe personality disorders, severe suicidality, or neurological or neuropsychiatric disorder with cognitive decline. These were screened centrally from the physician’s referral before the referral was accepted. The only additional inclusion criterion for the study was a baseline score of ≥8 on the GAD 7-item scale (GAD-7 [30]).

    Ethical Considerations

    The patients provided informed consent after the first log-in. The study protocol was approved by the ethics committee of the HUS and pertinent institutional authorities.

    Intervention and Procedure

    The new iCBT program for GAD was designed by an expert group to be diagnosis-specific because the existing evidence base was clearly the strongest at the time, in 2013. This new program consisted of 12 consecutive sessions and a follow-up session 3 months after treatment completion. The program was theoretically based on several models of GAD and anxiety, including aspects of the cognitive avoidance model [31], model of intolerance of uncertainty [32], metacognitive therapy [33], and Acceptance and Commitment Therapy [34], as well as social aspects, such as assertiveness training. The sessions included text and videos, as well as educational illustrations, example stories, therapeutic exercises, and homework (Textbox 1).

    After approval, the patients received an email and a letter prompting them to sign in and begin treatment. A schedule of 1 session per week was recommended, and a minimum 24-hour waiting period was enforced between sessions to encourage daily life practice. The program sent email prompts for arriving messages and, after 2 weeks of inactivity, log-in reminders. Nevertheless, no maximum completion time was required if the patient remained active in therapy.

    Although HUS-iCBT for GAD was therapist-supported, several persuasive elements used in unguided iCBT programs [35] were originally included and are listed in Textbox 2. In total, 7 elements were used (or 6, if discounting tunneling as the original authors did).

    Content by session.

    Session titles

    • Session 1: Introduction to HUS Helsinki University Hospital–internet-delivered cognitive behavioral therapy program, cognitive behavioral therapy model, and generalized anxiety disorder
    • Session 2: Bodily stress response, worry and relaxation
    • Session 3: Worry and avoidance, intolerance of uncertainty
    • Session 4: Experiential Avoidance, Core Beliefs
    • Session 5: Negative beliefs about worrying, challenging worrying
    • Session 6: Positive beliefs about worrying, challenging beliefs about worrying
    • Session 7: Challenging worrying
    • Session 8: Acceptance of worries, acceptance vs submission
    • Session 9: Intolerance of uncertainty and reaching for perfection and certainty
    • Session 10: Problem solving vs worrying, solvable vs unsolvable worries
    • Session 11: Social skills, needs in relationships, assertiveness
    • Session 12: Summarizing, warning signs, plan for the future, feedback
    Textbox 1. Content by session.
    Identified persuasive design principles.

    Principles and brief example

    • Reduction: simple stepped instructions; for example, worry diary or relaxation
    • Tunneling: logical thematic progression
    • Self-monitoring: symptom graphs
    • Simulation: example stories with avatars
    • Rehearsal: web-based worry diary and relaxation training
    • Reminders: log-in reminders
    • Normative influence: normalization of common generalized anxiety disorder features
    Textbox 2. Identified persuasive design principles.

    Therapist Support

    The therapists in the program were clinical psychologists, psychology students, or nurses with additional therapeutic training, all working at HUS Psychiatry. Each therapist received 1 day of training on internet delivery of CBT, text-based communication, the intervention protocol, and Good Clinical Practice. Therapists received regular group supervision and sought consultation with a senior psychotherapist at any time.

    To support patients’ progress, the therapists provided empathic asynchronous feedback with written messages 4 times or more often (if the patient requested) during the therapy. If the patient was inactive for 2 weeks, the therapist pursued contact through an SMS text message or iCBT program. If no reply arrived within a week, further contact was attempted by phone and thereafter by a letter including a 2-week deadline for continuing, after which access to therapy was discontinued.

    A web-based report of live data on individual therapists was created during the study to allow for supervision of compliance with the intervention protocol and to prompt support to those failing to ask for support. The report included data on the date of the last log-in and the number of patients not contacted or logged in within the last 2 weeks. The therapist time per patient was not subject to monitoring, but each therapist treated a minimum quota of patients per dedicated working hour. On average, this quota would mean 9 to 11 minutes of working time per patient per week.

    Measures

    Overview

    Symptoms were measured using web-based questionnaires. The primary measure of anxiety was the GAD-7; secondary measures were, for pathological worry, the Penn State Worry Questionnaire (PSWQ) and, for anxiety and impairment, the Overall Anxiety Severity and Impairment Scale (OASIS).

    Patients completed the GAD-7 at the beginning of each of the first 11 sessions and at the end of the final 12th session. The 2 secondary measures were filled at the beginning and end of treatment. Each participant received an invitation for a follow-up measurement 3 months after completing the treatment.

    GAD 7-Item Scale

    The GAD-7 is a short, 7-item self-report questionnaire developed to measure GAD diagnostic symptom criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition [30]. Its internal consistency (Cronbach α) has generally ranged between .88 in the clinical population and .92 in the general population [36,37]. The test–retest reliability in a study was 0.83 [30]. In this study, Cronbach α was .725 before the treatment and .905 after the treatment.

    Penn State Worry Questionnaire

    The PSWQ is a 16-item questionnaire designed to measure pathological worry, a signature feature of GAD [38]. Internal consistency has ranged from .86 to .95 among various samples [38-40]. PSWQ sensitivity to change is good but less salient than that of GAD-7 [41]. Test–retest reliability has been inconsistent (0.67-0.93) [39,42], so a midrange score of 0.80 was our choice. In our study, the Cronbach α was .865 before the treatment and .931 after the treatment.

    Overall Anxiety Severity and Impairment Scale

    The OASIS is a 5-item scale developed as a brief transdiagnostic measure of anxiety severity and impairment [43]. Internal consistency has ranged from .84 to .93 in 3 large clinical samples [44-46]. OASIS’s sensitivity to change has been established, but its diagnostic accuracy is low [45,47]. Test–retest reliability has been 0.73 in undergraduate samples and 0.82 in clinical samples [43,45] with a midrange value of 0.78 used. Cronbach α in this study was .736 before the treatment and .888 after the treatment.

    Adherence

    We measured the number of completed sessions and the time of therapy, defined as the time between the first and last GAD-7 in-therapy measurements. Only patients who completed all 12 sessions were categorized as completers. The program introduced the main theoretical framework in the first 4 sessions, and the later sessions focused mostly on further implementation of basic principles or introduced secondary content. Hence, we subcategorized noncompleters into two groups a priori: early dropouts (those discontinued before the completion of the fourth session) and late dropouts.

    Patient Characteristics

    We collected patient demographics such as age, registered sex, and municipality class. The municipality class was either urban or nonurban, according to the official Finnish classification [48].

    Statistical Analysis

    Our data were a convenience sample of all consented patients who entered treatment after February 2016. Our desired power was 80% and type I error was 0.05 for the main tests and 0.01 for post hoc tests. When comparing 2 independent means, a requirement to detect a small effect size, a minimum of 0.2, would be achieved with a sample of 435 for main tests and 647 for post hoc tests, assuming a 45%/55% completer/noncompleter distribution. For mixed model parameters, we set significance at a conservative P<.01, while ensuring that Akaike and Bayesian Information Criteria [49,50] do not indicate worsening fit. The final mixed model was built in a backward stepwise manner.

    The primary outcome analysis involved a linear mixed random model that allowed growth modeling to account for the changing pace of recovery at different time points during therapy, the use of all available data, and different intercept estimations for each individual. GAD-7 served as the dependent variable, and we modeled a growth curve using linear, quadratic, and cubic terms of the GAD-7 observation time. To control for the effect of dose of and time on therapy, we also tried a full model with the main effects and interactions of the GAD-7 observation session. We also input the main effects of gender, age, referral source, municipality class, and completion status, along with their linear combinations with time.

    The ITT and noncompleter groups’ effect sizes for GAD-7 were estimated with mixed model–estimated marginal means. This approach generally performs better than a pure last observation carried forward (LOCF) at handling dropout bias, especially in larger samples (n>400 [51]). The method used to calculate effect sizes was that of Morris and DeShon [52] (equations 13 and 12). Calculations also used the baseline SD and observed pre–post correlation.

    We calculated clinical change indexes using the LOCF values. Reliable change was defined as Reliable Change Index (RCI [53]), and recovery was defined as transitioning below the clinical cutoff on the primary outcome measure, GAD-7. The RCI and clinical cutoff together provide stringent criteria for both response and recovery [54]. Test–retest reliability obtained from earlier normative studies allowed the calculation of the RCI. We explored recovery for those with a baseline GAD-7 score of ≥10 to avoid confounding recovery rates with those of patients who had already fulfilled the criterion at baseline. In reliable recovery, the patient fulfilled the criteria for both clinical recovery and RCI. Full symptomatic recovery (remission) was defined as a posttreatment score <5 on the GAD-7.

    We used SPSS Statistics 22 (IBM Corporation) for the analyses [55].


    Patient Flow, Baseline Characteristics, and Adherence

    Of all referrals to the clinic, those rejected during the recruitment period amounted to only 0.6%. Of the 1912 patients who completed the consent form, 1488 (77.82%) provided their consent (Figure 1).

    Of the 1099 patients analyzed, 485 (44.13%) fully completed the HUS-iCBT, and a further 363 (33.03%) completed at least the first 4 sessions, meaning that 251 (22.84%) patients dropped out early. Those who completed from 4 to 11 sessions were considered as late dropouts. On average, patients completed 7.8 (SD 4.2; 65.1%) of the 12 sessions. The average time on therapy (time between pretreatment and last measurements) in the whole sample was 128 (SD 97) days, 171 (SD 95) days for completers, 36 (SD 44) days for early dropouts, and 131 (SD 84) days for late dropouts.

    At baseline, completers and noncompleters differed in their proportions regarding referral source, municipality class, average age, and OASIS scores (Table 1). The average age of completers was 4.9 years higher with their average OASIS score being 0.7 points higher than the score of noncompleters. Statistically significant differences did not emerge in gender distribution or in the average GAD-7 and PSWQ scores.

    ‎
    Figure 1. Patient flowchart. GAD-7: Generalized Anxiety Disorder 7-item scale.
    View this figure

    In post hoc analyses, completion was more likely among patients referred from private or occupational care (50.4%) than among those from primary care (39.1%; χ21=9.675; P=.004). Baseline OASIS scores differed significantly between early dropouts and both late dropouts (t612=4.297; P<.001) and completers (t734=6.041; P<.001), but not between late dropouts and completers (t846=1.625; P=.105). The average baseline OASIS scores were 13.3 for early dropouts, 12.3 for late dropouts, and 12.0 for completers.

    In total, 30 therapists treated an average of 37 (SD 38) study patients. Of the 1099 patients, 1097 (99.81%) received at least one message from their therapist, and 796 (72.43%) sent one or more messages to their therapist. Completers sent a message more often (420/485, 86.6% patients) than noncompleters (376/614, 61.2% patients). Therapists sent, on average, 8.4 (SD 4.7) messages and patients (those who did) sent 4.0 (SD 4.2) messages. Of those patients who did send messages, the average number of messages that completers sent (4.9, SD 4.8) was greater than that of noncompleters (3.1, SD 3.1). Therapists sent, on average, more messages to completers (11.6, SD 4.4) than to noncompleters (5.9, SD 3.1).

    Table 1. Baseline characteristics.
    CharacteristicsTotal sample (N=1099)Completers (n=485)Noncompleters (n=614)Completers vs noncompleters




    		Test statisticP value
    Gender, n (%)χ21=2.240.13

    		Female849 (77.3)385 (79.4)464 (75.6)


    		Male250 (22.7)100 (20.6)150 (24.4)

    Referral source, n (%)χ24=16.872.002

    		Primary care601 (54.7)235 (48.5)366 (59.6)


    		Private or occupational339 (30.8)171 (35.3)168 (27.4)


    		Student health care50 (4.5)24 (4.9)26 (4.2)


    		Psychiatry55 (5)25 (5.2)30 (4.9)


    		Unspecified54 (4.9)33 (6.8)21 (3.4)

    Municipality class, n (%)χ21=8.469.004

    		Urban966 (87.9)412 (84.9)554 (90.2)


    		Nonurban130 (11.8)73 (7.2)57 (5.7)

    Age (years), mean (SD)33.3 (12.2)36.0 (13.1)31.1 (12.2)t935=6.578<.001
    GAD-7a, mean (SD)13.2 (3.6)13.1 (3.6)13.3 (3.6)t1038=0.941.35
    PSWQb, mean (SD)64.4 (9.8)64.7 (9.9)64.2 (9.7)t1031=0.723.47
    OASISc, mean (SD)12.4 (2.8)12.0 (2.8)12.7 (2.8)t1046=4.216<.001

    aGAD-7: Generalized Anxiety Disorder 7-item scale.

    bPSWQ: Penn State Worry Questionnaire.

    cOASIS: Overall Anxiety Severity and Impairment Scale.

    Primary Outcome Model

    The building of the primary mixed model is described in Multimedia Appendix 1 and Textbox 1. The model was built backward stepwise, with all terms of interest entered into the model. Eliminated terms were gender, both in average effects and time interaction; municipality class, both in average effects and time interaction; age in average effects; and completion status, both in average effects and session interaction.

    The estimated fixed and random effects on GAD-7 are given in Table 2, and the model-estimated mean trajectory is shown in Figure 2. According to the model, after the average time between the pre- and posttreatment observations for completers (171, SD 95 days), the main effects of time on GAD-7 would amount to an average of 3.1 (95% CI 4.4-2.7) points of improvement. The main average effect of the session was 0.523 (95% CI 0.592-0.455) points of improvement per session, or 6.7 (95% CI 5.5-7.1) points after all 12 sessions. Time had a negative interaction with session, and over a complete therapy (12 sessions), an extra week of therapy would amount to an average of 0.20 (95% CI 0.16-0.24) points of deterioration. Older patients improved more than their younger counterparts; for example, 40-year-old patients improved on average 0.3 points more (95% CI 0.1-0.5) than their 20-year-old counterparts after an average course of therapy. Patients from private or occupational health care had similar overall symptoms as patients from student health care (95% CI −0.6 to 1.6) but milder than those from specialized psychiatric care (95% CI 1.0-3.1) and primary health care (95% CI 0.4-1.4). Patients from private and occupational health care benefited similarly as patients from psychiatric services (95% CI −0.12 to 0.17) but more than patients from student (95% CI 0.13-0.46) or primary health care (95% CI 0.05-0.19) per month on therapy. Completers and early dropouts benefited from the treatment similarly (95% CI −0.16 to 0.35), but late dropouts benefited less (95% CI 0.16-0.35) per month. Estimates of random effects indicated that patients likely had individual overall symptom levels (P<.001).

    Table 2. Mixed linear model parameter estimates.
    ParameterEstimate (SE; 95% CI)Wald Zt test (df)P value
    Fixed effectsN/Aa

    		Intercept12.422 (0.210; 12.01 to 12.83)
    		59.16 (1594.5)<.001

    		Observation time (time)−0.028 (0.004; −0.035 to −0.021)
    		−7.89 (8627.5)<.001

    		Time23.35×10−5 (1.32×10−5; 7.57×10−6 to 5.95×10−5)
    		2.53 (8308.9).01

    		Time3−3.39×10−8 (1.18×10−8; −5.70×10−8 to −1.07×10−8)
    		−2.87 (8117.6).004

    		Observation session (session)−0.523 (0.035; −0.592 to −0.455)
    		−15.00 (8652.5)<.001

    		Time × session0.002 (0.0003; 0.002 to 0.003)
    		9.11 (8240.2)<.001

    		Time × age−1.32×10−4 (4.10×10−5; 2×10−4 to −5.11×10−5)
    		−3.21 (8669.3).001

    		Referral source, main effectN/A


    		Private or occupationalN/A
    		N/AN/A


    		Psychiatry2.059 (0.530; 1.018 to 3.099)
    		3.88 (1289.4)<.001


    		Other1.409 (0.533; 0.363 to 2.454)
    		2.64 (1272.5).008


    		Student health care0.500 (0.559; −0.597 to 1.598)
    		0.89 (1336.0).37


    		Primary care0.919 (0.249; 0.430 to 1.408)
    		3.68 (1303.0)<.001

    		Referral source × timeN/A


    		Private or occupationalN/A
    		N/AN/A


    		Psychiatry0.001 (0.002; −0.004 to 0.006)
    		0.34 (8163.2).73


    		Other0.004 (0.002; −0.001 to 0.009)
    		1.58 (8140.2).11


    		Student health care0.010 (0.003; 0.004 to 0.015)
    		3.46 (8193.8).001


    		Primary care0.004 (0.001; 0.002 to 0.006)
    		3.51 (8219.9)<.001

    		Completion status × timeN/A


    		Completers, as referenceN/A
    		N/AN/A


    		Late dropouts0.005 (0.001; 0.002 to 0.008)
    		3.44 (8677.2).001


    		Early dropouts0.003 0.004; −0.005 to 0.012)
    		0.72 (8525.7).47
    Random effects

    		Residual9.40 (0.15; 9.11 to 9.70)61.76N/A<.001

    		Patient intercept10.56 (0.52; 9.59 to 11.63)20.34N/A<.001

    aN/A: not applicable.

    ‎
    Figure 2. Estimated mean marginal values based on linear mixed model versus observed values. Curves depict an average patient trajectory. GAD-7: Generalized Anxiety Disorder 7-item scale.
    View this figure

    Effect Sizes

    Treatment effect sizes are given in Table 3. The estimated ITT treatment effect size was 0.97, indicating a large effect on the GAD-7. The estimated scores indicated a small effect size in early dropouts (Cohen d=0.34) and a large effect size in late dropouts (Cohen d=0.85). The treatment effect observed for the completers was very large (Cohen d=1.39).

    At the 3-month follow-up, 111 completers were reached, which accounted for 10.1% of the whole sample and 22.9% of the completers. The observed change in the GAD-7 score was −6.8. For secondary measures, changes were −11.3 in PSWQ and −3.9 in OASIS.

    Table 3. Treatment effects based on estimated marginal means and observed values.
    Treatment effectBaseline, mean (SD)Post, mean (SD)Pre–post, changePre–post, correlationEffect size, Cohen d (95% CI)
    GAD-7 a

    		Completersb13.1 (3.6)7.0 (4.7)−6.10.2671.39 (1.25-1.53)

    		Estimated values


    		Early dropouts12.9 (3.6)11.5 (4.5)−1.30.377c0.34 (0.16-0.51)


    		Late dropouts12.9 (3.6)9.4 (5.1)−3.50.351c0.85 (0.70-1.00)


    		ITTd analysis12.9 (3.6)8.7 (4.7)−4.10.308c0.97 (0.88-1.06)
    PSWQe

    		Completersb64.7 (9.9)53.7 (13.6)−11.00.5261.14 (1.00-1.27)
    OASISf

    		Completersb12.0 (2.8)8.4 (3.9)−3.60.4521.23 (1.09-1.37)

    aGAD-7: Generalized Anxiety Disorder 7-item scale.

    bFewer patients completed the PSWQ and OASIS after the treatment (n=479).

    cOnly these correlations were calculated from before the treatment and the last observed values for those with ≥2 observations.

    dITT: intention to treat.

    ePSWQ: Penn State Worry Questionnaire. For PSWQ and Overall Anxiety Severity and Impairment Scale, only baseline scores were available for noncompleters; hence, only complete effect sizes were observed.

    fOASIS: Overall Anxiety Severity and Impairment Scale.

    Clinical Change Indexes

    Clinical change indexes based on the GAD-7 scores are provided in Table 4. After the treatment, 23.29% (256/1099) of patients achieved remission (GAD-7 score <5).

    Indexes for PSWQ and OASIS were reported only for completers because noncompleters had only one observation. Reliable change on PSWQ was a change of >12 points and 40.9% (196/479) reliably improved and 1% (5/479) reliably deteriorated. On OASIS reliable change was a change of >3 points and 48.6% (233/479) reliably improved and 1.5% (7/479) reliably deteriorated.

    Table 4. Clinical indexes at the final on-therapy observation.
    Clinical indexITTa, n (%)Completers, n (%)Late dropoutsb (sessions ≥4), n (%)Early dropoutsb (sessions <4), n (%)
    Baseline GADc-7 score ≥81099 (100)485 (100)363 (100)251 (100)

    		Reliable improvementd532 (48.4)306 (63.1)170 (46.8)56 (22.3)

    		Reliable deterioratione41 (3.73)12 (2.5)19 (5.2)10 (4)
    Baseline GAD-7 score ≥10894 (100)385 (100)295 (100)214 (100)

    		Reliable improvementd479 (53.6)268 (69.6)155 (52.5)56 (26.2)

    		Recoveryf486 (54.4)280 (72.7)149 (50.5)57 (26.6)

    		Reliable recoveryg421 (47.1)244 (63.4)129 (43.7)48 (22.4)

    		Reliable deterioratione27 (3)9 (2.3)11 (3.7)7 (3.3)

    aITT: intention to treat.

    bOn the basis of the last observed values.

    cGAD-7: Generalized Anxiety Disorder 7-item scale.

    dGAD-7 score drop of ≥5.

    eGAD-7 score increase of ≥5.

    fGAD-7 changed to <10 at after the treatment.

    gBoth reliably improved and recovered.


    Principal Findings

    This nationwide, free-of-charge, therapist-supported HUS-iCBT for GAD in routine care, with no predetermined maximum completion time, comprised 1099 patients referred by their physicians. To the best of our knowledge, this study has the largest reported real-world sample involving iCBT for GAD. The patients reported substantial improvement in ITT analyses amounting to effect sizes of Cohen d=0.97 on GAD-7; and for completers, Cohen d=1.39 on GAD-7, Cohen d=1.14 on PSWQ, and Cohen d=1.23 on OASIS. The full completion rate was 44.1% (485/1099), and the average session completion rate was 7.8 (65%) of the total 12 sessions. A reliable improvement occurred in 48.4% (532/1099) of the full sample. Of those with a baseline GAD-7 score ≥10, reliable improvement occurred in 53.6% (479/894), with a transition to recovery achieved by 54.4% (421/894). These results seemed to be enduring for 22% (111/485) of completers who answered the follow-up questionnaires. The average change on the GAD-7 at 3-month follow-up was still −6.8 points (vs −6.1 at after the treatment).

    Comparison With Earlier Work

    Symptomatic Change

    The change in GAD-7 score was comparable with that achieved in 3 of the 4 Australian ThisWayUp trials (Cohen d=0.91-1.18 [23,26,27]) lower than that in the fourth trial that used different inclusion criteria (Hedges g=2.06 and 2.10 [28]), and comparable with the Online Therapy Unit’s trial (Cohen d=1.07 [29]). The change in GAD-7 scores in our study was also not inferior to the within-group change in a recent meta-analysis of RCTs of transdiagnostic iCBT for anxiety and depression (95% CI 0.91-1.22 [15]). The effect size also seems encouraging when contrasted with change in comparison with a waiting list in another meta-analysis (95% CI 0.39-1.01 [9]).

    Interestingly, from both practical and theoretical viewpoints, the longer it took for patients to complete a given number of sessions, the less they improved. It is plausible that the effects of therapy become diluted if the patient’s commitment to therapy weakens. Therefore, methods of increasing engagement in therapy seem one of the most promising ways to increase the effectiveness of iCBT for GAD.

    The change in the PSWQ for completers (Cohen d=1.14) indicates a substantial improvement in worry. In a meta-analysis of 10 RCTs of iCBT for GAD, the ITT effect size on the PSWQ in iCBT was 0.71 [16]. Although the comparison is not direct, the results of this study agree with the existing RCT results.

    The OASIS-measured anxiety-related functional impairment demonstrated a large improvement in the completers (Cohen d=1.23). Of the ThisWayUp trials, Mahoney et al [28] found medium to large improvements in health-related disability and functioning, as measured by the World Health Organization Disability Assessment Schedule 2.0. Similarly, a trial conducted by the Online Therapy Unit [29] found that patients had a large decrease in functional impairment after therapy, as measured by the Work and Social Adjustment Scale. The combined results strongly suggest that iCBT for GAD can improve functional capacity.

    Our completers improved faster than late dropouts, unlike those in 2 earlier studies, which showed no or negligible differences. However, significant improvement was also observed in our noncompleters, which could imply heterogeneity in this population. Feedback offered by the patients who withdrew from the Online Therapy Unit intervention also suggests that several patients may have discontinued treatment as their symptoms improved [29]. In iCBT for depression, there seems to be a subgroup of patients who benefit from the therapy rapidly and discontinue treatment prematurely [56]. These patients may regard further treatment as unnecessary. Hence, discontinuation should not be considered, by default, a failure.

    Although not unequivocally proven, clinician-referred patients in iCBT tend to exhibit lower effect sizes than community-recruited or self-referred ones [13,57]. Thus, our results seem encouraging, as all patients in HUS-iCBT have been physician-referred.

    Recovery and Reliable Change

    In the ITT analysis using LOCF, of HUS-iCBT patients with a baseline GAD-7 score ≥10, 54.4% (486/894) of patients achieved recovery. This recovery rate is comparable with that reported earlier in face-to-face CBT for GAD (51.4% [54]). Owing to differing imputation methods, we cannot directly compare our ITT results with those of Australian studies. Nevertheless, Hobbs et al [26] reported a 70% recovery rate for completers, which is comparable with our 72.7% (280/385).

    Moreover, the criteria for reliable change in our study and previous studies differed, which prevented direct comparison. In one meta-analysis of RCTs of iCBT, reliable deterioration occurred in 3.1% (95% CI 1.5%-5.9%) of patients in trials for anxiety disorders [13]; in another meta-analysis of several disorders, reliable deterioration occurred in 5.8% of iCBT patients (vs 17.4% of patients on the waiting list) [21]. In HUS-iCBT, reliable deterioration occurred in 3.73% (41/1099) of patients. Thus, HUS-iCBT for GAD also appeared to be as safe as precious treatments and safer than no treatment.

    Adherence

    The full completion rate in our study (485/1099, 44.13%) was similar to the rates in previous routine care trials (36%-55%) but lower than that in a meta-analysis of routine care iCBT for depression and anxiety (61%) [13]. However, RCTs are likely to inflate adherence when compared with real-world trials. For instance, ThisWayUp’s trials have shown a dramatic decrease from very high full completion rates when transitioning from a research setting into primary care (75%-85% vs 38% [24]).

    The proportion of completed sessions in HUS-iCBT (65%) was marginally lower than that in the earlier trials (67%-77% in ThisWayUp’s and 72% in Online Therapy Unit’s trials) but somewhat higher than that in the previously mentioned meta-analysis (57% [13]). Compared with ThisWayUp’s program, a lower proportion of completed sessions may be expected in a therapy comprising twice as many sessions (12 vs 6). Interestingly, more patients in the HUS-iCBT completed ≥6 sessions compared with the Australian trials (61% vs 36%-55%). On the other hand, the Online Therapy Unit had slightly better results with a program of similar length both in sessions (n=12) and average days on therapy (135 vs our 128). Differences in the course of therapy, recruitment, and therapist protocols prevent straightforward comparison. The comparison is further hampered by the overlapping recruiting periods in the Australian trials, as the number of patients reported remains unique. Hence, the optimal number of iCBT sessions for GAD remains unknown.

    Compared with our noncompletion rate (614/1099, 55.86%), face-to-face individual psychotherapy for GAD has had a low dropout rate of 17% in a meta-analysis of RCT trials [58], and a meta-analysis of nonrandomized outpatient studies on CBT for adult anxiety disorders showed a similar dropout rate of 15% [59]. However, the criteria for completion in face-to-face psychotherapy are often more lenient, such as stopping when agreed upon with the therapist or completing a minimum number of sessions [59]. Thus, the adherence numbers are not comparable and are likely to be closer to iCBTs when analyzed with an equivalent metric.

    In our study, completers sent and received more messages than did noncompleters. Our data do not allow exploration of causality, but this finding may be intuitively explained by the completer’s longer time on therapy. In the Online Therapy Unit’s trial [29], both patients and therapists sent more messages (on average 9.6 and 20.0, respectively) than ours (4.2 and 8.4, respectively). This difference is not surprising, given the differences in intake protocol (both clinician referral and community recruitment and intake interview vs strictly a physician’s referral and only referral screening in HUS-iCBT) and therapist-contact intensity (weekly messages vs minimum 4 messages per therapy). These differences may also partly explain the higher adherence in the Online Therapy Unit’s trial.

    Patients’ Background Effects

    Older patients improved more and faster than younger patients. At first glance, this seems counterintuitive, as one could presume that young, digital native patients could feel more comfortable in the digital realm than the older ones. Nevertheless, 1 of the 3 ThisWayUp’s trials also found that older age predicted larger improvement [23], whereas another found no such relationship [26], and the third trial found a relationship but left its direction unreported [27]. Furthermore, in a transdiagnostic study of iCBT for anxiety and depression, older age predicted greater improvement [60], whereas in a meta-analysis of various psychotherapies for GAD, older participants improved less than younger ones [61].

    Moreover, younger patients were less likely to complete the program than their older counterparts. This relationship has been a common finding in iCBT for GAD in routine care [23,26,27] but not in face-to-face psychotherapy for GAD [58]. The average age in this study was 33 years, 6 or 7 years lower than that in the 3 ThisWayUp and single Online Therapy Unit study that reported their sample ages [23,26,27,29]. Overall, research on the age issue is scarce, and what still remains obscure is how age may or may not affect the results of different modes of therapy, such as face-to-face therapy or iCBT.

    Gender had no significant influence on adherence in our study, similar to the findings of Australian studies. Systematic reviews of RCTs on face-to-face psychotherapy for GAD or CBT for anxiety disorders have also found no significant gender-adherence relationship [54,58].

    The full completion rate was higher among patients referred from private or occupational health (171/339, 50.4%) than among those referred from primary care (235/601, 39.1%). Their symptoms improved faster than those referred from students or primary health care. These effects could be due to patient-group differences: patients referred from occupational health are, by definition, employed and presumably have a higher average level of functioning, higher socioeconomic status, and a lower likelihood of serious comorbidities. Moreover, in our recent study, physicians in occupational health displayed more interest in sending patients to iCBT than their primary care counterparts [62], which may transfer into patients’ own expectations and motivation.

    Patients from nonurban municipalities were more likely to complete the therapy (73/130, 56.2%) than those from urban municipalities (412/966, 42.7%). This relationship is somewhat opposed to the findings in the 2012 report by Mewton et al [23], where patients from urban settings were more likely to complete the therapy than were those from a rural setting. The causes of this difference are unknown and are likely to include multiple recently discovered factors, such as differences in technology adoption, financial concerns, and access to treatment [63]. Nevertheless, the municipality class did not demonstrate a significant impact on symptomatic improvement in the mixed model analysis.

    Design Comparison

    The results of this and earlier studies in routine care are, in general, comparable despite considerable differences in setting, design, and support.

    To the best of our knowledge, HUS-iCBT and the Online Therapy Unit’s programs had no predetermined maximum therapy completion time, whereas ThisWayUp applied a typical fixed maximum time restriction (90 days). The time from the first to last observation on HUS-iCBT (128 days) and from the first to last log-in at the Online Therapy Unit (135 days) were comparable and longer than the typical 90 days in other trials. Although the lack of a predetermined maximum time span may support adherence during changing life situations, it could also disengage some patients and dilute therapy effects, thereby leading to an increased dropout rate. As confounding design and sample features exist, such as different numbers of sessions and differences in sample average age, the optimal time span remains unknown.

    In the Australian studies, patients were referred to iCBT by their own independent clinicians, each of whom was required to register as a provider, to receive training, and to use an assessment toolkit. The Online Therapy Unit required a centralized diagnostic interview. Our HUS-iCBT only screened referrals from physicians (but not from other clinicians) with no obligatory registration or any specific assessment schema. The practice at HUS-iCBT was chosen as the middle ground to ensure proper diagnostics while maintaining a high intake flow. To maintain a low threshold, the inclusion criteria in HUS-iCBT were purposely loose, with only 0.6% (51/8394) of all physician referrals rejected during the recruitment period, and this may have contributed to a possible patient–therapy mismatch.

    In HUS-iCBT, the support was provided centrally by specially trained and supervised mental health professionals, as recommended by authors from 5 universities who argue that iCBT requires specialized expertise and processes [64]. In the ThisWayUp studies, support was provided by the referring clinicians, who rarely contacted their patients. A pre-existing alliance could compensate for the clinicians’ lack of specialized training.

    HUS-iCBT was free of charge for the patients, as was the treatment at the Online Therapy Unit. Patients in ThisWayUp paid approximately Aus $49 (US $37) for the treatment, which may have ensured motivation and therefore improved adherence [24] but may also serve as a barrier to treatment.

    Strengths and Limitations

    The main strengths of this study are its nationwide scope, the largest sample thus far reported, and its routine care setting. One could argue that reliance on self-report measures can inflate treatment effects, but 2 recent meta-analyses suggest that combining indexes based on self-reporting can be even conservatively biased [7,54]. Self-report measures also fulfill practical requirements for scalability.

    Other routine care studies did not use PSWQ or OASIS. Their addition in our study shows that iCBT for GAD can ease pathological worry and anxiety-related impairments in routine care. Although they both offer important information, they may not be as sensitive to change as the GAD-7.

    Concomitant treatments, including psychopharmaceuticals, are often offered to patients in routine care. Information on these treatments is not reliable. This limits our understanding of the possible interactions between different treatments.

    Not all comorbidities may have appeared in patient documents in referrals, and depressive symptoms were not measured. This issue is particularly relevant for GAD, as its annual comorbidity with major depression is around 41% [4]. One effectiveness study also indicated that depressive symptoms could decrease treatment effectiveness [26]. The lack of depression measurement does not weaken the study’s results but highlights the question of whether every one of our patients received optimal treatment.

    We could not, retrospectively, reliably identify the therapist behind each message sent to the patient. We know that due to vacation periods, illness, or leaving HUS, not all messages sent to a patient may have come from the same therapist. This prevented us from analyzing the effect of the individual therapist, their profession, or other relevant training.

    Only 23% (111/485) of completers were reached for the 3-month follow-up. Such high attrition limits our conclusions regarding long-term effectiveness. However, we did not include follow-up data in the linear mixed model analyses, and these limited data do not compromise the main results.

    Future Research

    Thus far, therapist-supported iCBT for GAD has not been compared with other active treatment forms or no treatment in routine care, making comparisons with active controls, coupled with health economic analyses, essential. Not all patients benefit from iCBT for GAD. Both transdiagnostic programs and tailoring content to individual needs have displayed promising results [15,64], although for anxiety, unguided transdiagnostic iCBTs may be less or no more efficacious than diagnosis-specific ones [19,35]. Tailored transdiagnostic treatments seem likely to be a significant trend in iCBT, but evidence for their added value still needs to be confirmed.

    Adherence to iCBT for GAD seems to be lower in primary care than in RCT trials. Evidence is emerging for adapted therapist support for patients at risk for dropout [65,66] and for an optimal therapist-contact schedule for iCBT in cases of depression and insomnia [67,68]. Machine learning applications could accelerate the identification of patients at risk for dropout and accelerate proactive support. However, this venue of research is still in its infancy.

    iCBT programs often retain a considerable number of text-based answers and messages. Text mining approaches could offer an exciting avenue for exploring qualitative phenomena. Such analyses could be beneficial for both theoretical research and the practical application of iCBT.

    The optimal time schedules for or the number of sessions in iCBT for GAD remain unclear. Further comparison studies should seek to establish more suitable, effective, and economic combinations. Age and other demographic variables may also influence the outcomes of iCBT for GAD. To provide equally effective care for everyone, future studies should investigate how to overcome hurdles presented by demographic variables.

    PSWQ and OASIS were not used in other routine care studies. They represent both diagnosis-specific and transdiagnostic processes related to GAD and anxiety in general. OASIS, a short self-rating scale, opens up a venue for further comparative research between iCBT for various psychiatric disorders. As PSWQ measures pathological worry, a theoretically important GAD feature, an in-depth psychometric examination of PSWQ may reveal important information on moderators and mediators of effectiveness.

    Since the collection of our data, HUS-iCBT for GAD has adopted a novel technical platform, a predetermined maximum completion time (20 weeks), pretreatment phone calls, and weekly therapist support. These changes may have practical implications that will enable further studies.

    Conclusions

    This nationwide, free-of-charge, therapist-supported HUS-iCBT for GAD with no predetermined maximum completion time was effective at improving symptoms and reducing worry and functional impairment in routine care. Overall, this therapy appears to be safer than no treatment and is at least as effective as other therapist-supported iCBTs for GAD. The observed gains at the 3-month follow-up should be confirmed in future studies. Future research needs to establish comparative effectiveness against other treatments and to optimize the benefit of iCBT for GAD in a variety of patient groups and individual patients.

    Acknowledgments

    This work was supported by the Government of Finland and the Hospital Region of Helsinki and Uusimaa.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Description of primary mixed model building and statistics.

    DOCX File , 15 KB
    1. International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Fifth Edition, 2016. Geneva: World Health Organization; 2015.
    2. Munk-Jørgensen P, Allgulander C, Dahl AA, Foldager L, Holm M, Rasmussen I, et al. Prevalence of generalized anxiety disorder in general practice in Denmark, Finland, Norway, and Sweden. Psychiatr Serv 2006 Dec;57(12):1738-1744. [CrossRef] [Medline]
    3. Revicki DA, Travers K, Wyrwich KW, Svedsäter H, Locklear J, Mattera MS, et al. Humanistic and economic burden of generalized anxiety disorder in North America and Europe. J Affect Disord 2012 Oct;140(2):103-112. [CrossRef] [Medline]
    4. Ruscio AM, Hallion LS, Lim CC, Aguilar-Gaxiola S, Al-Hamzawi A, Alonso J, et al. Cross-sectional comparison of the epidemiology of DSM-5 generalized anxiety disorder across the globe. JAMA Psychiatry 2017 May 01;74(5):465-475 [FREE Full text] [CrossRef] [Medline]
    5. Generalised anxiety disorder and panic disorder in adults: management. National Institute for Health and Care Excellence.   URL: https://www.nice.org.uk/guidance/cg113/ [accessed 2022-02-13]
    6. Anxiety Disorders. Current Care Guidelines. Working Group Set Up by the Finnish Medical Society Duodecim and the Finnish Psychiatric Association. Helsinki: The Finnish Medical Society Duodecim; 2019.   URL: https://www.kaypahoito.fi/hoi50119
    7. Cuijpers P, Sijbrandij M, Koole S, Huibers M, Berking M, Andersson G. Psychological treatment of generalized anxiety disorder: a meta-analysis. Clin Psychol Rev 2014 Mar;34(2):130-140. [CrossRef] [Medline]
    8. Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, et al. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv 2018 Sep;13:108-115 [FREE Full text] [CrossRef] [Medline]
    9. Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
    10. Ophuis RH, Lokkerbol J, Heemskerk SC, van Balkom AJ, Hiligsmann M, Evers SM. Cost-effectiveness of interventions for treating anxiety disorders: a systematic review. J Affect Disord 2017 Mar 01;210:1-13. [CrossRef] [Medline]
    11. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther 2018 Jan;47(1):1-18. [CrossRef] [Medline]
    12. Karyotaki E, Efthimiou O, Miguel C, Bermpohl FM, Furukawa TA, Cuijpers P, Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis. JAMA Psychiatry 2021 Apr 01;78(4):361-371 [FREE Full text] [CrossRef] [Medline]
    13. Etzelmueller A, Vis C, Karyotaki E, Baumeister H, Titov N, Berking M, et al. Effects of internet-based cognitive behavioral therapy in routine care for adults in treatment for depression and anxiety: systematic review and meta-analysis. J Med Internet Res 2020 Aug 31;22(8):e18100 [FREE Full text] [CrossRef] [Medline]
    14. Cuijpers P, Noma H, Karyotaki E, Cipriani A, Furukawa TA. Effectiveness and acceptability of cognitive behavior therapy delivery formats in adults with depression: a network meta-analysis. JAMA Psychiatry 2019 Jul 01;76(7):700-707 [FREE Full text] [CrossRef] [Medline]
    15. Păsărelu CR, Andersson G, Bergman Nordgren L, Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn Behav Ther 2017 Jan;46(1):1-28. [CrossRef] [Medline]
    16. Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered treatment for generalized anxiety disorder: a systematic review and meta-analysis. Internet Interventions 2015 Sep;2(3):272-282. [CrossRef]
    17. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry 2017 Apr 01;74(4):351-359. [CrossRef] [Medline]
    18. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    19. Domhardt M, Geßlein H, von Rezori RE, Baumeister H. Internet- and mobile-based interventions for anxiety disorders: a meta-analytic review of intervention components. Depress Anxiety 2019 Mar;36(3):213-224. [CrossRef] [Medline]
    20. Pihlaja S, Stenberg J, Joutsenniemi K, Mehik H, Ritola V, Joffe G. Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders - A systematic review. Internet Interv 2018 Mar;11:1-10 [FREE Full text] [CrossRef] [Medline]
    21. Rozental A, Magnusson K, Boettcher J, Andersson G, Carlbring P. For better or worse: an individual patient data meta-analysis of deterioration among participants receiving internet-based cognitive behavior therapy. J Consult Clin Psychol 2017 Feb;85(2):160-177. [CrossRef] [Medline]
    22. Hedman E, Ljótsson B, Kaldo V, Hesser H, El Alaoui S, Kraepelien M, et al. Effectiveness of internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord 2014 Feb;155:49-58. [CrossRef] [Medline]
    23. Mewton L, Wong N, Andrews G. The effectiveness of internet cognitive behavioural therapy for generalized anxiety disorder in clinical practice. Depress Anxiety 2012 Oct;29(10):843-849. [CrossRef] [Medline]
    24. Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther 2012 Aug;50(7-8):463-468. [CrossRef] [Medline]
    25. Framework for FDA’s real-world evidence program. U.S. Food and Drug Administration.   URL: https://www.fda.gov/media/120060/download [accessed 2022-02-13]
    26. Hobbs MJ, Mahoney AE, Andrews G. Integrating iCBT for generalized anxiety disorder into routine clinical care: treatment effects across the adult lifespan. J Anxiety Disord 2017 Oct;51:47-54. [CrossRef] [Medline]
    27. Newby JM, Mewton L, Andrews G. Transdiagnostic versus disorder-specific internet-delivered cognitive behaviour therapy for anxiety and depression in primary care. J Anxiety Disord 2017 Mar;46:25-34. [CrossRef] [Medline]
    28. Mahoney AE, Newby JM, Hobbs MJ, Williams AD, Andrews G. Reducing behavioral avoidance with internet-delivered cognitive behavior therapy for generalized anxiety disorder. Internet Interv 2019 Mar;15:105-109 [FREE Full text] [CrossRef] [Medline]
    29. Hadjistavropoulos HD, Pugh NE, Nugent MM, Hesser H, Andersson G, Ivanov M, et al. Therapist-assisted internet-delivered cognitive behavior therapy for depression and anxiety: translating evidence into clinical practice. J Anxiety Disord 2014 Dec;28(8):884-893 [FREE Full text] [CrossRef] [Medline]
    30. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    31. Borkovec T. The nature, functions, and origins of worry. In: Worrying: Perspectives on Theory, Assessment and Treatment. Hoboken, New Jersey, United States: John Wiley & Sons; 1994.
    32. Dugas MJ, Freeston MH, Ladouceur R. Intolerance of uncertainty and problem orientation in worry. Cognit Ther Res 1997;21(6):593-606. [CrossRef]
    33. Wells A. The metacognitive model of worry and generalised anxiety disorder. In: Worry and Its Psychological Disorders: Theory, Assessment and Treatment. Hoboken, New Jersey, United States: Wiley; 2006.
    34. Hayes S, Strosahl K, Wilson K. Acceptance and Commitment Therapy, Second Edition The Process and Practice of Mindful Change. New York, United States: Guilford Press; 2011.
    35. McCall HC, Hadjistavropoulos HD, Sundström CR. Exploring the role of persuasive design in unguided internet-delivered cognitive behavioral therapy for depression and anxiety among adults: systematic review, meta-analysis, and meta-regression. J Med Internet Res 2021 Apr 29;23(4):e26939 [FREE Full text] [CrossRef] [Medline]
    36. Beard C, Björgvinsson T. Beyond generalized anxiety disorder: psychometric properties of the GAD-7 in a heterogeneous psychiatric sample. J Anxiety Disord 2014 Aug;28(6):547-552. [CrossRef] [Medline]
    37. Johansson R, Carlbring P, Heedman A, Paxling B, Andersson G. Depression, anxiety and their comorbidity in the Swedish general population: point prevalence and the effect on health-related quality of life. PeerJ 2013 Jul 9;1:e98 [FREE Full text] [CrossRef] [Medline]
    38. Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behav Res Ther 1990;28(6):487-495. [CrossRef] [Medline]
    39. Zlomke KR. Psychometric properties of internet administered versions of Penn State Worry Questionnaire (PSWQ) and Depression, Anxiety, and Stress Scale (DASS). Comput Human Behav 2009 Jul;25(4):841-843. [CrossRef]
    40. Verkuil B, Brosschot JF. The online version of the Dutch Penn State Worry Questionnaire: factor structure, predictive validity and reliability. J Anxiety Disord 2012 Dec;26(8):844-848. [CrossRef] [Medline]
    41. Dear BF, Titov N, Sunderland M, McMillan D, Anderson T, Lorian C, et al. Psychometric comparison of the generalized anxiety disorder scale-7 and the Penn State Worry Questionnaire for measuring response during treatment of generalised anxiety disorder. Cogn Behav Ther 2011;40(3):216-227. [CrossRef] [Medline]
    42. Molina S, Borkovec T. The Penn State Worry Questionnaire: Psychometric properties and associated characteristics. In: Worrying: Perspectives on Theory, Assessment and Treatment. Hoboken, New Jersey, United States: John Wiley & Sons; 1994.
    43. Norman SB, Cissell SH, Means-Christensen AJ, Stein MB. Development and validation of an Overall Anxiety Severity And Impairment Scale (OASIS). Depress Anxiety 2006;23(4):245-249. [CrossRef] [Medline]
    44. Campbell-Sills L, Norman SB, Craske MG, Sullivan G, Lang AJ, Chavira DA, et al. Validation of a brief measure of anxiety-related severity and impairment: the Overall Anxiety Severity and Impairment Scale (OASIS). J Affect Disord 2009 Jan;112(1-3):92-101 [FREE Full text] [CrossRef] [Medline]
    45. Hermans M, Korrelboom K, Visser S. A Dutch version of the Overall Anxiety Severity and Impairment Scale (OASIS): psychometric properties and validation. J Affect Disord 2015 Feb 01;172:127-132. [CrossRef] [Medline]
    46. Ito M, Oe Y, Kato N, Nakajima S, Fujisato H, Miyamae M, et al. Validity and clinical interpretability of Overall Anxiety Severity And Impairment Scale (OASIS). J Affect Disord 2015 Jan 01;170:217-224. [CrossRef] [Medline]
    47. Moore SA, Welch SS, Michonski J, Poquiz J, Osborne TL, Sayrs J, et al. Psychometric evaluation of the Overall Anxiety Severity And Impairment Scale (OASIS) in individuals seeking outpatient specialty treatment for anxiety-related disorders. J Affect Disord 2015 Apr 01;175:463-470. [CrossRef] [Medline]
    48. Vuoden 2017 kuntien ja kuntaryhmistys-luokituksen välinen luokitusavain. Tilastokeskus.   URL: https://www2.tilastokeskus.fi/fi/luokitukset/corrmaps/kunta_1_20170101%23kuntaryhmitys_1_20170101/ [accessed 2022-02-13]
    49. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Control 1974 Dec;19(6):716-723. [CrossRef]
    50. Schwarz G. Estimating the dimension of a model. Ann Statist 1978 Mar;6(2):461-464. [CrossRef]
    51. Mallinckrodt CH, Clark WS, David SR. Accounting for dropout bias using mixed-effects models. J Biopharm Stat 2001;11(1-2):9-21. [CrossRef] [Medline]
    52. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods 2002 Mar;7(1):105-125. [CrossRef] [Medline]
    53. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [CrossRef] [Medline]
    54. Springer KS, Levy HC, Tolin DF. Remission in CBT for adult anxiety disorders: a meta-analysis. Clin Psychol Rev 2018 Apr;61:1-8. [CrossRef] [Medline]
    55. IBM Corp. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp; 2013.
    56. Arndt A, Lutz W, Rubel J, Berger T, Meyer B, Schröder J, et al. Identifying change-dropout patterns during an internet-based intervention for depression by applying the Muthen-Roy model. Cogn Behav Ther 2020 Jan;49(1):22-40. [CrossRef] [Medline]
    57. Romijn G, Batelaan N, Kok R, Koning J, van Balkom A, Titov N, et al. Internet-delivered cognitive behavioral therapy for anxiety disorders in open community versus clinical service recruitment: meta-analysis. J Med Internet Res 2019 Apr 17;21(4):e11706 [FREE Full text] [CrossRef] [Medline]
    58. Gersh E, Hallford DJ, Rice SM, Kazantzis N, Gersh H, Gersh B, et al. Systematic review and meta-analysis of dropout rates in individual psychotherapy for generalized anxiety disorder. J Anxiety Disord 2017 Dec;52:25-33. [CrossRef] [Medline]
    59. Hans E, Hiller W. A meta-analysis of nonrandomized effectiveness studies on outpatient cognitive behavioral therapy for adult anxiety disorders. Clin Psychol Rev 2013 Dec;33(8):954-964. [CrossRef] [Medline]
    60. Edmonds M, Hadjistavropoulos HD, Schneider LH, Dear BF, Titov N. Who benefits most from therapist-assisted internet-delivered cognitive behaviour therapy in clinical practice? Predictors of symptom change and dropout. J Anxiety Disord 2018 Mar;54:24-32 [FREE Full text] [CrossRef] [Medline]
    61. Carl E, Witcraft SM, Kauffman BY, Gillespie EM, Becker ES, Cuijpers P, et al. Psychological and pharmacological treatments for generalized anxiety disorder (GAD): a meta-analysis of randomized controlled trials. Cogn Behav Ther 2020 Jan;49(1):1-21 [FREE Full text] [CrossRef] [Medline]
    62. Taiminen HM, Saraniemi S, Joffe G, Stenberg J, Parkinson J. Reducing health inequalities trough digital options in mental health: a physician's perspective. Health Mark Q 2019;36(2):93-106. [CrossRef] [Medline]
    63. Moskalenko MY, Hadjistavropoulos HD, Katapally TR. Barriers to patient interest in internet-based cognitive behavioral therapy: informing e-health policies through quantitative analysis. Health Policy Technol 2020 Jun;9(2):139-145. [CrossRef]
    64. Titov N, Hadjistavropoulos HD, Nielssen O, Mohr DC, Andersson G, Dear BF. From research to practice: ten lessons in delivering digital mental health services. J Clin Med 2019 Aug 17;8(8):1239 [FREE Full text] [CrossRef] [Medline]
    65. Forsell E, Jernelöv S, Blom K, Kraepelien M, Svanborg C, Andersson G, et al. Proof of concept for an adaptive treatment strategy to prevent failures in internet-delivered CBT: a single-blind randomized clinical trial with insomnia patients. Am J Psychiatry 2019 Apr 01;176(4):315-323. [CrossRef] [Medline]
    66. Pihlaja S, Lahti J, Lipsanen JO, Ritola V, Gummerus E, Stenberg J, et al. Scheduled telephone support for internet cognitive behavioral therapy for depression in patients at risk for dropout: pragmatic randomized controlled trial. J Med Internet Res 2020 Jul 23;22(7):e15732 [FREE Full text] [CrossRef] [Medline]
    67. Hadjistavropoulos HD, Schneider LH, Edmonds M, Karin E, Nugent MN, Dirkse D, et al. Randomized controlled trial of internet-delivered cognitive behaviour therapy comparing standard weekly versus optional weekly therapist support. J Anxiety Disord 2017 Dec;52:15-24 [FREE Full text] [CrossRef] [Medline]
    68. Hadjistavropoulos HD, Schneider LH, Mehta S, Karin E, Dear BF, Titov N. Preference trial of internet-delivered cognitive behaviour therapy comparing standard weekly versus optional weekly therapist support. J Anxiety Disord 2019 Apr;63:51-60 [FREE Full text] [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    GAD: generalized anxiety disorder
    GAD-7: Generalized Anxiety Disorder 7-item scale
    iCBT: internet-delivered cognitive behavioral therapy
    ITT: intention to treat
    LOCF: last observation carried forward
    OASIS: Overall Anxiety Severity and Impairment Scale
    PSWQ: Penn State Worry Questionnaire
    RCI: Reliable Change Index
    RCT: randomized controlled trial

    Edited by A Mavragani; submitted 05.04.21; peer-reviewed by H Hadjistavropoulos, S Victor, H Gandhi; comments to author 18.06.21; revised version received 30.09.21; accepted 16.12.21; published 24.03.22

    Copyright

    ©Ville Ritola, Jari Olavi Lipsanen, Satu Pihlaja, Eero-Matti Gummerus, Jan-Henry Stenberg, Suoma Saarni, Grigori Joffe. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 24.03.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.