Published on in Vol 20, No 11 (2018): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10927, first published .
Guided Internet-Based Cognitive Behavioral Therapy for Adult Depression and Anxiety in Routine Secondary Care: Observational Study

Guided Internet-Based Cognitive Behavioral Therapy for Adult Depression and Anxiety in Routine Secondary Care: Observational Study

Guided Internet-Based Cognitive Behavioral Therapy for Adult Depression and Anxiety in Routine Secondary Care: Observational Study

Original Paper

1Department of Psychology, Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark

2Centre for Telepsychiatry, Mental Health Services of Southern Denmark, Odense, Denmark

3Department of Mental Health, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

4Department of Research and Innovation, Specialized Mental Health Care, GGZ InGeest, Amsterdam, Netherlands

Corresponding Author:

Kim Mathiasen, MSc

Department of Psychology

Faculty of Health Sciences

University of Southern Denmark

Campusvej 55

Odense, 5230

Denmark

Phone: 45 61677747

Email: kim.mathiasen@rsyd.dk


Background: Internet-based cognitive behavioral therapy (iCBT) is a promising new treatment method for depression and anxiety. However, it is important to determine whether its results can be replicated in routine care before its implementation on a large scale. Although many studies have demonstrated the efficacy of iCBT under controlled conditions, only a few studies have investigated its effectiveness in routine care. Furthermore, several effects of iCBT such as treatment effects in routine care are unclear.

Objective: This study aimed to evaluate the clinical effectiveness of iCBT for depression and anxiety in routine secondary care.

Methods: n a retrospective cohort study, we analysed patients treated for depression or anxiety in a dedicated iCBT clinic in secondary care in Denmark. Patients were examined before treatment and weekly thereafter by using the Patient Health Questionnaire-9 and the Generalized Anxiety Disorder-7 scales for the diagnoses of depression and anxiety, respectively. Primary analyses were conducted using a linear mixed-effects model with random slope and intercept. Secondary analyses were conducted using baseline characteristics as predictors (gender, age, highest level of education, occupational status, marital status, psychotropic medication use, consumption of alcohol, and leisure drugs). Additionally, logistic regression analyses were used to predict noncompletion of treatment.

Results: A total of 203 (depression, N=60; anxiety, N=143) patients were included. Participants were mainly female (78.3% with depression and 65.7% with anxiety), with a mean age of 36.03 (SD 10.97) years (range, 19-67 years) for patients with depression and 36.80 (SD 13.55) years (range, 19-69 years) for patients with anxiety. The completion rates were 62% (37) and 40% (57) for depression and anxiety treatments, respectively. The primary analyses revealed large and significant reductions in the symptom levels of depression (beta=-6.27, SE 0.83, P<.001, d=1.0) and anxiety (beta=-3.78, SE 0.43, P<.001, d=1.1). High baseline severity of the primary disorder was associated with high treatment gains (r=-0.31 for depression; r=-0.41 for anxiety). In patients with anxiety, high baseline severity also predicted a high risk of noncompletion (odds ratio=1.08, CI=1.01-1.16, P=.03). An increase in the baseline severity of the comorbid disorder slightly increased the risk of noncompletion for both disorders (depression: odds ratio=1.03, CI=1.01-1.06, P=.02; anxiety: odds ratio=1.08, CI=1.01-1.16, P=.03).

Conclusions: iCBT can be clinically effective in routine care. Since depression and anxiety are costly and debilitating disorders that are vastly undertreated, this finding is important. Additionally, iCBT may help bridge the gap between the need for treatment and its provision. Our results are comparable to the within-group results of efficacy and effectiveness studies. Our noncompletion rates are similar to those observed in psychotherapy but are higher than those reported in similar clinics. Multiple factors predicted outcome and noncompletion. However, all predictor effects were statistically weak.

J Med Internet Res 2018;20(11):e10927

doi:10.2196/10927

Keywords



Background

Anxiety and depression are highly prevalent and costly disorders [1-4]. Although effective treatments for these disorders exist [5], only about half of those in need receive professional help, and only approximately 16% receive minimally adequate evidence-based treatments [6,7]. Many studies have demonstrated the efficacy of guided internet-based cognitive behavioral therapy (iCBT) [8-13], which may help bridge the gap between the need for treatment and its provision.

Although effectiveness studies such as randomized controlled trials (RCTs) conducted in routine care settings [14] are limited in number, they show effect sizes comparable to those obtained under almost-ideal conditions of efficacy studies [15]. Owing to the increasing number of persons seeking help for mental disorders in Denmark, the Danish Agency for Digitisation included iCBT in a national action plan for telemedicine in 2012, which led to the establishment of the first iCBT clinic in Denmark. The clinic, named Internetpsykiatrien, has been in operation since 2014 and a part of routine care since 2015. Funded by the Danish healthcare system, it is free at the point of use.

Internetpsykiatrien is a part of secondary care in the Mental Health Services of Southern Denmark. It delivers guided iCBT [15] for adult patients with anxiety and depression and offers selfreferral; as such, patients can apply for treatment directly at the clinic from a secure and dedicated website, and no general practitioner is required [16]. Herein, we present the results of an observational study of the first routine outcome measurements of a cohort of adult patients who were treated at the clinic with iCBT for depression or anxiety.

This observational cohort study with a pre-post design based on routine outcome measurements is important, as treatments and patients in routine practice may differ from those in RCTs. Patients may differ with regard to their sociodemographic and baseline characteristics as well as their motivation or need for treatment. Additionally, adherence to strict research protocols and treatment manuals and continuous monitoring of clinician adherence are not part of the standard clinical practice in most settings. Thus, although effectiveness trials have higher external validity than efficacy trials, their clinical effect may differ from that in actual routine practice, for example, in the case of RCTs, even in a routine care setting [14,17]. Therefore, studies of the clinical effect of routine care are needed to increase knowledge on the issue in real-world settings [18]. Although efficacy studies are important, the external validity of an intervention should be the prime focus in order to convince decision makers and clinicians of its effectiveness, thereby ensuring its implementation and uptake [19]. In addition to efficacy and effectiveness studies, observational cohort studies can provide important insights into the impact of iCBT in real-life routine care of populations.

The literature on cohort studies in routine practice of comparable iCBT clinics is limited. However, some published studies from the Internetpsykiatrien clinic in Sweden, the eMeistring clinic in Norway, the MindSpot clinic in Australia, and a centralized clinic in Canada [16,20-25] describe dedicated iCBT clinics that deliver guided iCBT and are operated as routine mental health care services. In the Swedish clinic, large, significant within-group effect sizes were found in cohort studies of major depression (n=1203, d=1.27) [20], panic disorder (n=570, d=0.91) [21], and social anxiety (n=547, d=0.86) [22]. In Norway, results for panic disorder showed a large pre-post effect size (n=124, d=1.24) [23]. In the Sydney clinic, a cohort study from the first 30 months of operation revealed large, significant effect sizes (n=6149, d=1.3-1.4) [24] for transdiagnostic iCBT for anxiety and depression. The centralized unit in Canada offers guided iCBT with selfreferral and provides assistance to community mental health services; the clinic showed large, significant reductions in symptom levels of both anxiety and depression (n=260, d=1.2-1.4) [25]. The literature, although scarce, seems to suggest that guided iCBT may be a promising treatment format in routine practice.

Primary Aim

This study aims to improve knowledge of the clinical effect of guided iCBT in routine practice by presenting and discussing analyses of routine outcome measurements in a cohort of adults treated for anxiety or depression at the iCBT clinic Internetpsykiatrien in Odense, Denmark.

Secondary Aim

Our secondary aim was to determine predictors of treatment outcome and noncompletion for iCBT.


Design

This uncontrolled retrospective cohort study included all patients who had initiated treatment for depression (N=60) or anxiety (N=143) at the iCBT clinic Internetpsykiatrien and provided consent for analysis of their data. Data for all patients were collected from the time reliable data were available until the time of data extraction (June 1, 2017). For patients with anxiety, reliable data was available from March 1, 2015, and for depressed patients, from August 1, 2016. The study was observational in nature and did not interfere with the normal operation of the clinic. All data were collected as routine outcome measurements and part of continuous quality assurance by the clinic by using online questionnaires. This article adheres to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for reporting observational studies [26,27] and the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [28].

Interventions

Internetpsykiatrien operates as part of routine care in Southern Denmark and is funded by the Region of Southern Denmark, which is a tax-funded public authority. The clinic uses self-referral via a website [29], where patients fill out a secure application form. They do not need referral from other sources such as a general practitioner. Treatment including the use of the programs is free of charge for the patients. After an initial screening of their application, patients are contacted for clarification by telephone and, if appropriate (eg, in cases of increased suicidal risk), provided access to more relevant sources of assistance or invited to a video-based assessment interview with a licensed psychologist or a psychologist under the supervision of a licensed psychologist. Eligibility criteria for patients at the clinic are age ≥ 18 years; meeting the diagnostic criteria of the International Classification of Diseases and Related Health Problems, 10th edition, for major depressive disorder, panic disorder, agoraphobia, social phobia, specific phobia, or generalized anxiety disorder; not currently at high risk of suicide; no comorbid substance dependence, bipolar affective disorder, psychotic illness, or obsessive compulsive disorder; not currently under other psychological treatment for depression or anxiety; access to a personal computer and fast internet connection; and adequate understanding of spoken and written Danish.

After assessment, the patients were provided anxiety or depression treatment. Both treatments included access to an iCBT treatment program and weekly or biweekly clinical support from a licensed clinical psychologist or a psychologist under the supervision of a licensed psychologist. Support was provided via a secure text module for depression treatment and via telephone for anxiety treatment. In addition, support comprised technical assistance, help with interventions included in the programs, and encouragement to continue the treatment. The Internetpsykiatrien has been described in detail elsewhere, and its implementation has been compared to similar clinics internationally [30,31].

For depression treatment, the program NoDep by Context Consulting was used. This program was designed and developed as part of a Public Private Innovation project between the Region of Southern Denmark and Context Consulting when the Internetpsykiatrien was under development. The manuscript was written by Specialist Psychologist Krista Nielsen Straarup, who is a well-known expert in CBT for depression in Denmark. The program includes 6 mandatory modules and 2 optional modules and is based on CBT; its core components include psychoeducation, behavioral activation, cognitive restructuring, and relapse prevention. The optional modules include interventions for reducing rumination and restructuring dysfunctional beliefs [32].

The anxiety treatment uses the Danish version of FearFighter, which is distributed in Denmark by Context Consulting [33] and includes 9 mandatory modules based on CBT. FearFighter was originally developed by Professor Dr. Isaac Marks [34] in England. Its core components comprise psychoeducation, in vivo and interoceptive exposure exercises, cognitive restructuring, and relapse prevention. In both programs, patients are reminded to access the programs by SMS and email on a weekly basis, if they fail to do so. This feature can be switched off by the patient. The programs are accessed via the internet on patients’ own devices. Access link, username, and access codes are provided to the patient by the clinic.

In both programs, clinicians are provided with administrative access, which includes a status of the progress of patients, their exercises, their scores on the weekly measures, and a secure text module (for NoDep only). Further, the clinicians are notified of high scores on suicidal ideation and lack of progress. In both cases, patients are contacted personally by phone. In cases of suicidal risk, a standard procedure including risk assessment and possible referral to an acute ward is instigated.

Ethics

All the patients analyzed in the study provided informed consent for use of their data at the beginning of treatment. No participants were approached by the research team. The Regional Committees on Health Research Ethics for Southern Denmark waived the need for full approval owing to the retrospective nature of the study, no new intervention was performed, and no human biological material was used. The routine outcome measurement database was approved by the Danish Data Protection Agency.

Measures

The Patient Health Questionnaire-9 (PHQ-9) [35] was used to evaluate changes in the severity of depressive symptoms. This 9-item questionnaire has good psychometric properties [36,37] and can be used to monitor patients with depressive disorders. All items are scored on a 0- to 3-point scale with a total score of 0-27, and higher scores indicated more severe depression. One study of the Danish version confirmed unidimensionality and reliability but suggests collapsing the two middle response categories [38]. However, the scale used in the Internetpsykiatrien clinic followed the original structure of the questionnaire.

The Generalized Anxiety Disorder-7 (GAD-7) scale [39] was used to measure general symptoms of anxiety. It is a 7-item questionnaire originally developed to measure GAD in primary care; each item is scored 0-3, with a total score of 0-21. This scale has shown good psychometric properties, has been validated in primary care [39] and the general population [40], and has performed well as a measure of anxiety-symptom severity in specialized mental health care settings [41]. Thus far, there are no validation studies of the Danish version of the GAD-7 scale.

The Fear Questionnaire [42] was used to describe baseline levels of comorbid anxiety symptoms for patients with depression. Data were extracted from the questionnaire included in the application form. The Fear Questionnaire measures phobias across 6 subscales and provides a total phobia score of 0-120. Similar to the GAD-7 scale, this scale has demonstrated good psychometric properties [43,44].

All questionnaires were administered in an electronic format in the treatment program or at the initial application form. This was not an open survey, as data were obtained from patients included in treatment at the Internetpsykiatrien clinic. It was mandatory to fill out the questionnaires with no extra incentives, apart from the added value of the supporting psychologist who monitored patients’ progress. Completeness of the questionnaires was checked by the program, such that progress was hindered until all items were completed. Once the questionnaire was submitted, it could not be altered. All responses were unique to the individuals, as its administration was a part of the treatment program or linked to the application form. All participants viewed the questionnaires and participated by completing at least some of the items. All available data were analyzed, even if partly filled questionnaires were submitted.

Description of Participants

Baseline characteristics of patients were extracted from a questionnaire delivered as part of the online application form, which was used when patients applied for treatment, and from a pretreatment questionnaire package. The collected data on patient characteristics included gender, age, highest level of education, occupational status, marital status, baseline symptomatic levels of depression and anxiety (PHQ-9, GAD-7, Fear Questionnaire), units of alcohol consumed per week, and use of recreational drugs. The baseline characteristics were analyzed using descriptive statistics.

Primary Analyses

Outcome measures were extracted from the pretreatment questionnaire package and the last of a series of weekly measures administered online as an integrated part of the iCBT programs. The intermittent weekly measures were unfortunately unavailable to the research team.

Treatment effects were estimated using mixed-effects linear regression. The pre-post measurements were considered as repeated observations nested within patients. The slope estimates for the binary treatment variable (0, pre; 1, post) indicated change in the outcome measurements from baseline to follow-up. The primary outcome variables, that is, symptomatic measures of the primary disorder, were included as a fixed effect, whereas the binary treatment variable and subject were included as random effects (random slope and intercept). The slope-intercept correlations were estimated to control for baseline severity. The primary model estimated the overall treatment effect with one slope parameter, one intercept, and one correlation between slope and intercept. No control variables were included. The standardized effect sizes were calculated by taking into account the correlation between the pre-post measurement and expressed as d [45]. Multivariate imputation by chained equations was applied to manage missing data. All inferences assumed normally distributed error terms and heteroscedasticity, which were substantiated by visual inspection of the Q-Q normality plot and the plot of fitted values versus standardized residuals.

Predictor Analyses

Predictors of change in symptom severity were determined in subsequent univariate models; specifically, we employed one model per predictor by using the same model type as that in the primary analyses. Regression estimates for the interaction between the predictor variables and the binary treatment variable indicated the extent to which the overall treatment slope was affected by a particular predictor variable. Additionally, two multivariate models were performed. One model included all variables, and the other model included only significant variables from the univariate analyses.

Completion

Analyses of treatment effect among completers and non-completers were conducted using the same model type as that of the primary analyses. Logistic regression was used to predict the risk of noncompletion on the basis of all available baseline variables for each of the two patient groups, which was expressed as odds ratios (ORs). Treatment completion was defined as completion of at least 5 of 6 mandatory modules in the depression program and 8 of 9 modules in the anxiety program. These thresholds were applied because patients at these stages of the programs had completed all core active treatment interventions aimed at symptom reduction.

Statistical Analyses

All data were analyzed using R ver. 3.4.2 [46]. Mixed-effects linear regression was performed using the nlme package [47]. The glm function [46] was used for logistic regression, whereas the mice package was used for imputation [48]. P<.05 was set as the threshold value for significance in inferential statistics.


Description of Participants

Among patients with depression, 60 patients started treatment and consented to the analysis, of which 59 (98.3%) patients provided followup data. A mean of 5.4 (SD 2.2) of the 6 mandatory and 2 optional modules were completed. In addition, 143 patients with anxiety commenced treatment, of which 133 (93.0%) patients provided followup data. A mean of 5.9 (SD 2.6) modules were (Figure 1). Table 1 presents the characteristics of all the patients who commenced treatment in the inclusion periods and had quit the program or ended treatment by the point of data extraction.

Figure 1. Patient flow chart.
View this figure
Table 1. Description of patients.
VariableDepression (N=60)Anxiety (N=143)
Gender, n (%)


Female47 (78.3)94 (65.7)

Male13 (21.7)52 (34.3)
Age (years)


Mean (SD)36.03 (10.97)36.80 (13.55)

Range19-6719-69
Education, n (%)N=41N=48

Primary: 9 years in school6 (14.6)6 (12.5)

Gymnasium: 12 years in school5 (12.2)11 (22.9)

College or vocational school26 (63.4)31 (64.6)

University: 17 years in school0 (0.0)0 (0.0)

Other4 (9.8)0 (0.0)
Occupational status, n (%)N=41N=48

Employed/student23 (56.1)39 (81.3)

Unemployed5 (12.2)7 (14.6)

Sick leave5 (12.2)2 (4.2)

Retired0 (0.0)0 (0.0)

Other8 (19.5)0 (0.0)
Marital status, n (%)N=41N=48

Single11 (26.8)a18 (37.5)b

Married or de facto not living together5 (12.2)a2 (6.25)b

Married or de facto living together25 (61.0)a27 (56.3)b
Depression severity


PHQ-9c mean (SD)15.5 (5.7)9.94 (6.06)

PHQ-9a, range5-260-26
Anxiety severity


FQb score, total mean (SD)22 (22.8)cd

FQ total score, range0-81c

GAD-7e score, mean (SD)10.42 (5.11)

GAD-7 score, range0-21
Patients taking psychotropic medication, n (%)
N=136

No medication45 (75.0)95 (69.9)

<1 month1 (1.7)5 (3.7)

<2 months3 (5.0)2 (1.5)

>2 months11 (18.3)34 (25.0)
Units of alcohol comsumed weekly, n (%)
N=136

032 (53.3)60 (44.1)

0-525 (41.7)51 (37.5)

5-101 (2.7)17 (12.5)

10-202 (4.1)7 (5.1)

>200 (0.0)1 (0.7)
Recreational drugs, n (%)3 (5.0)0 (0.0)f

aPHQ-9: Patient Health Questionnaire-9.

bFQ: Fear Questionnaire.

cTaken from the application form.

dNot applicable.

eGAD-7: Generalized Health Anxiety questionnaire.

fn=136 because some data were not collected for all patients at the clinic.

Primary Analyses

Table 2 shows the results of the linear mixed-effects model for depression and anxiety treatments including beta-coefficients, standard errors, and P-values for intention-to-treat analyses, completer analysis, and non-completer analysis.

Primary analysis of patients with depression revealed a significant reduction in depressive symptoms on the PHQ-9 (beta=–6.27, SE 0.83, P<.001), with a large effect size (d=1.0). For patients with anxiety, the linear mixed-effects model demonstrated a significant reduction in the symptom level of anxiety on the GAD-7 (beta=–3.78, SE 0.43, P<.001) with a large effect size (d=1.1).

Predictor Analyses

Exploratory predictor analyses for interaction effects revealed that some baseline characteristics predicted a change in symptom severity of the primary disorder from pretreatment to posttreatment (Table 2). For patients with depression, comorbid anxiety predicted slightly lesser symptom reduction with a small, but significant, positive addition to the slope (beta=.08, SE 0.04, P=.03). Additionally, a very small, but significant, interaction between treatment and time spent in the program was observed (beta=–0.03, SE 0.01, P=.02), indicating that for each additional day spent in the program, the PHQ-9 score reduced by 0.03. Furthermore, a college education as the highest education level predicted a steeper slope from pretreatment to posttreatment on the PHQ-9 (beta=–5.06, SE 2.02, P=.02) as compared to primary school education. The baseline symptom level of depression (intercept) was negatively correlated with the slope of change (r=–0.31), indicating a larger symptomatic reduction if the baseline level of depressive symptoms was higher. However, when all variables were included in one multivariate model, none of the interactions were significant. Similar results were observed when only the significant interactions from the univariate analyses were included in one multivariate model.

For patients with anxiety, two baseline characteristics predicted outcome. Inclusion in the “Other” category significantly increased symptom reduction (beta=–3.49, SE 1.10, P=.002) as compared to inclusion in the “Employed/student” category. Additionally, a higher level of comorbid depressive symptoms at baseline predicted a slightly larger reduction in the symptom level of anxiety (beta=–0.18, SE 0.07, P=.01). Time spent in the program was not associated with the treatment outcome in patients with anxiety. Finally, similar to patients with depression, the baseline level of anxiety was negatively correlated with slope (r=–0.41), indicating a larger improvement with higher baseline levels. Multivariate analysis of all variables indicated that marriage or de facto not living together was significant (beta=–2.64, SE 1.32, P=.047), whereas marriage and de facto living together was not significant (beta=1.63, SE 1.12, P=.15). Occupational status included in the “Other” category (beta=–2.96, SE 0.083, P=.02) and baseline comorbidity (beta=–0.259, SE 0.083, P=.002) remained significant in the multivariate analysis. Similar results were obtained when only the significant interactions from the univariate analyses were included; however, in this case, no category under the variable “marital status” was significant.

Completion

Of all patients with depression, 37 (61.7%) completed the treatment in the depression program. Analysis of treatment effect showed that for patients with depression, the level of depressive symptoms on the PHQ-9 reduced significantly (beta=–7.41, SE 1.00, P<.001), with a large effect size (d=1.1). Similarly, noncompleters showed a significant decrease in symptom severity (beta=–4.43, SE 1.39, P=.004), albeit with a lower yet moderate-to-large effect size (d=0.7). A nonsignificant trend was noted in the interaction between treatment effect and noncompletion (beta=2.97, SE 1.68, P=.08), indicating a nonsignificant decrease in symptom change in cases of noncompletion.

Among patients with anxiety, 57 (39.9%) completed treatment. A significant reduction in the severity of anxiety was observed among patients who completed the program (beta=–4.72, SE 0.65, P<.001), with a large effect size (d=1.1). Similar results were observed for 86 (60.1%) patients who did not complete the program, albeit with a moderate effect size (beta=–3.22, SE 0.56, P<.001, d=0.6). Furthermore, no significant difference was observed in the interaction between severity of anxiety and noncompletion (beta=1.49, SE 0.88, P=.09), but a trend toward better outcome with completion was noted.

Table 2. Univariate linear mixed-effects model outcomes for severity of depression and anxiety symptoms. Baseline severity correlated with the main effect for depression (r=–0.31) and anxiety (r=–0.41).
VariablesDepression (PHQ-9)a, N=60Anxiety (GAD-7)b, N=143



Mean beta (SE)P valuecdMean beta (SE)P valuecd
Treatment effectd






Main effect–6.27 (0.83)<.0011.0–3.78 (0.43)<.0011.1

Completers–7.41 (1.00)<.0011.1–4.72 (0.65)<.0011.1

Noncompleters–4.43 (1.39).0040.7–3.22 (0.56)<.0010.6
Predicting outcomee






Gender







Time point–4.77 (1.79).01
–3.90 (0.73)<.001


Gender x time point–1.91 (2.02).35
0.19 (0.91).84

Age







Time point–3.89 (2.88).18
–3.77 (1.25).003


Age x time point–0.07 (0.08).39
0.00 (0.03).99

Highest education level







Time point–3.75 (1.73).03
–3.00 (1.54).05


Education level x time point








Gymnasium vs primary1.08 (2.99).72
0.28 (1.76).88



College vs primary–5.06 (2.02).02
–0.50 (1.69).77



University vs primary


–2.07 (2.03).31



Other vs primary0.45 (2.56).86
–2.21 (1.81).22

Marital status







Time point–4.86 (1.36)<.001
–4.33 (0.76)<.001


Marital status x time point








Married or de facto not living together vs single1.46 (2.39).55
–0.95 (1.09).39



Married or de facto living together vs single–3.42 (1.78).06
1.94 (1.00).05

Occupational status







Time point–6.46 (1.24)<.001
–2.54 (0.71)<.001


Income x time point








Unemployed vs employed or student1.46 (2.62).58
–1.20 (1.07).27



Sick leave vs employed or student1.21 (2.26).59
–0.46 (1.32).73



Other vs employed or student–1.20 (2.26).60
–3.49 (1.10).002

Baseline comorbidity







Anxiety (FQe total)








Time point–8.16 (1.18)<.001






Time point x FQ total0.08 (0.04).03





Depression (PHQ-9)








Time point


–2.02 (0.81).01



Time point x PHQ-9


–0.18 (0.07).01

Time spent in the program







Time point–3.95 (1.26).003
–3.34 (0.63)<.001


Time spend x time point–0.03 (0.01).02
–0.00 (0.00).35

aPHQ-9: Patient Health Questionnaire.

bGAD-7: Generalized Anxiety questionnaire.

cTwo-tailed P value.

dPretreatment to last observation.

ed values cannot meaningfully be calculated for the interaction effects.

fFQ: Fear Questionnaire.

For patients with anxiety, a higher level of baseline anxiety predicted a slightly higher risk of noncompletion (OR=1.08, CI=1.01-1.16, P=.03). Similar results were noted for higher levels of comorbid depression (OR=1.12, CI=1.05-1.20, P<.001). A college education as the highest education level decreased the risk of noncompletion compared to primary school education (OR=0.14, CI=0.02-0.61, P=.02). A complete overview of all analyses of predictors for treatment completion is presented in Multimedia Appendix 1.


Aim

The aim of the present study was to evaluate the clinical effect of guided iCBT for depression and anxiety in routine care by analyzing routine outcome measurements from the dedicated iCBT clinic Internetpsykiatrien in Odense, Denmark. From March 1, 2015, to June 1, 2017, 203 patients commenced treatment, including 60 patients who underwent treatment for depression and 143 patients who underwent treatment for anxiety, which demonstrates the need for and motivation to seek guided iCBT.

Participants

Of the patients with depression, 78.3% were female, which corresponds to the findings of previous epidemiological research [49-52]. Despite the new format of treatment delivery, the gender difference may be biased by treatment-seeking behavior, as previously shown by a male to female ratio of 2:1. The prevalence of anxiety disorders is not extensively researched in Denmark, but in general practice, more women experience anxiety disorders than men [52]. The mean age, marital status, and employment status of our sample resembles those in primary care [52], which is reasonable, as Internetpsykiatrien offers selfreferral and is thus likely to receive a patient population of patients with depression and anxiety in primary care. The distribution of the highest education level resembles that of the general Danish population: primary, 25.9%; gymnasium, 10.2%; college or vocational school, 51.7%; university, and 9.5; other, 2.65% [53]. However, there are some noticeable differences. In the present study, fewer patients had primary school and university education as the highest level of education and more patients had college or vocational school as the highest level of education. Nonetheless, overall, our sample seemed representative of the Danish population.

Primary Analyses

The main analyses of change in the symptom levels of primary disorder from pretreatment to posttreatment showed large and significant reductions in both depression (beta=–6.27, SE 0.83, P<.001, d=1.0), and anxiety (beta=–3.78, SE 0.43, P<.001, d=1.1). These results resemble within-group effect sizes of previous studies on guided iCBT for adult patients with anxiety and depression, including both efficacy studies conducted under controlled research conditions [54-56] and effectiveness studies conducted under routine care conditions [15]. Notably, the results of studies in clinics akin to the one under investigation, which employed similar study designs examining cohorts in routine care (d=0.9-1.4) [20-25], are in line with our results. Therefore, this study concludes that the clinic under investigation yields results similar to those seen in other comparable clinics, and the effects of iCBT are satisfactory as compared to within-group effect sizes of previous efficacy and effectiveness studies.

Predictor Analyses

Baseline severity of primary disorders influenced the effect of treatment. For anxiety and depression, a higher baseline level of primary disorder correlated with larger treatment gains, which is a correlation commonly observed in studies on guided iCBT [57-59]. This effect is interesting, because it is often assumed that guided selfhelp should be aimed primarily at milder symptom levels of the disorders, although factors other than severity should be examined as indicators for stratification of patients for different treatments. Some studies found substantial evidence that treatment effect was unrelated to the treatment outcome [60].

There was a difference in how the symptomatic level of a comorbid disorder affected treatment gain. For patients with depression, comorbid anxiety slightly hampered the effect of treatment; in contrast, for patients with anxiety, high levels of comorbid depression increased the effect of treatment to a small extent. Thus, comorbidity might influence the effect of treatment differently for the two disorders, which may add nuance to the findings of a previous study, which concluded that the severity of comorbid disorders predicts poor treatment outcome [59]. Our finding may also support the use of transdiagnostic treatment programs or personalized treatment approaches.

For patients with anxiety, occupational status in the “Other” category rather than the “Employed/student” category positively affected the outcome (beta=–3.49, SE 1.10, P=.002). Analysis of the reasons for selection of the “Other” category showed that this category was extensively heterogeneous and included several entries such as apprenticeships, other forms of benefit, or no income, which made it difficult to interpret the result. Neither this category nor any other category of occupational status had an effect on the treatment in patients with depression. However, compared to the employed or student status, the unemployed status was associated with a higher risk of noncompletion for patients with depression. A previous study showed that a poorer employment status was associated with worse treatment outcome in unguided iCBT [61], which was not observed in this study. This difference in results may have resulted from the different formats of iCBT delivery. The present study investigated guided iCBT; therefore, the guidance may serve as a protective factor for vulnerable patients. This assumption could be substantiated by emerging findings that indicate the importance of the therapeutic alliance in iCBT [62,63]. Further study of the influences of and processes involved in the guidance is needed.

We found that more time spent in the program predicted a greater reduction in symptom severity for patients with depression but not for patients with anxiety, which seems to indicate a dose-response effect of depression treatment, but not anxiety treatment. Multivariate analyses showed different results for both anxiety and depression. For depression, no significant interactions were observed, which might be a result of the small sample size. These predictions should therefore be interpreted carefully.

Completion

Of the patients who commenced treatment, 37 of 60 patients with depression and 57 of 143 patients with anxiety completed treatment, which yields noncompletion rates of 38.3% and 60.1%, respectively. A review of 152 studies on traditional face-to-face psychotherapy reported an average dropout rate of 46.9% [64], which is comparable to our result. However, the dropout rates of iCBT treatment usually range from 0% to 78%, with a weighted mean of 26.5% [65]. This finding is in line with the results from routine-care iCBT clinics in Sweden, Norway, and Canada, where rates of noncompletion ranged from 15.9% to 29.5% [66-68]. Therefore, in the Internetpsykiatrien clinic, the rate of noncompletion seems relatively high, particularly for patients with anxiety. However, it is possible that the cutoff chosen to signify completion of treatment in our study was overly conservative, requiring completion of 83% and 88% of the modules of depression and anxiety, respectively, which are considerably higher than the thresholds used in other studies [23], where merely 56% of the modules needed to be completed. This hypothesis may be substantiated by the fact that significant and moderate-to-large effect sizes were seen among patients who did not complete the programs in the present study, indicating that some patients may terminate their treatment because they are satisfied with their progress. In addition, the difference in effect size between intention-to-treat analyses and completer analyses was not significant. For both disorders, a higher baseline symptom level of comorbid disorders slightly increased the risk of noncompletion. Interestingly, the dropout rates differed between the two treatment groups, with higher dropout rates observed for patients with anxiety than for patients with depression (60.1% vs 38.3%). However, this does not appear to be the case in the literature [66-68], and the underlying reasons are unclear. This difference may be caused by technical variations such as those between programs or the format of delivery of support, for example, written support for depression treatment and telephone support for anxiety treatment. However, it could also be a result of the different symptomatology of the disorders; for example, anxiety is characterized by avoidant behavior. Nonetheless, it is important to determine how to improve adherence, particularly for patients with anxiety, and whether this goal can be attained by different methods for the two different disorders. Further studies should focus on examination of different user behavior and experiences from the programs during and after provision of support. Furthermore, examination of motivations for ending treatment is needed. Finally, research investigating process factors involved in completion or noncompletion is needed to further clarify individual differences in response patterns among patients receiving iCBT.

Limitations

Most of the results from the exploratory predictor analyses were weak and subject to type 1 error due to their exploratory nature; therefore, they should be interpreted with caution. Prediction of treatment outcomes from baseline characteristics is generally difficult, as reported in previous iCBT studies showing varying results [58-61]. Our results therefore need repetition, particularly in routine care, with a larger study sample.

Conclusion

The Internetpsykiatrien clinic has demonstrated its ability to implement iCBT in routine care as well as the existence of, need for, and motivation among patients to seek iCBT treatment for anxiety and depression in the Region of Southern Denmark. The clinic yields results comparable to those of similar international clinics and efficacy and effectiveness studies. As such, the Danish clinic shows good clinical effectiveness. These findings are important, because depression and anxiety are highly prevalent and costly disorders that are substantially undertreated. Our results support the hypothesis that iCBT can help bridge the gap between the need for treatment and its provision even in routine care. Further research is needed to investigate the processes of change and individual response patterns to iCBT in order to improve the ability to personalize treatment for the individual and understand the reasons for premature cessation of treatment.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Univariate logistic mixed-effects model interactions for prediction of noncompletion.

PDF File (Adobe PDF File), 25KB

  1. Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, et al. Prevalence of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr Scand Suppl 2004(420):21-27. [CrossRef] [Medline]
  2. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun;62(6):593-602. [CrossRef] [Medline]
  3. Olesen J, Gustavsson A, Svensson M, Wittchen H, Jönsson B, CDBE2010 SG, et al. The economic cost of brain disorders in Europe. Eur J Neurol 2012 Jan;19(1):155-162. [CrossRef] [Medline]
  4. Ustün TB. The global burden of mental disorders. Am J Public Health 1999 Sep;89(9):1315-1318. [Medline]
  5. Cuijpers P. Four decades of outcome research on psychotherapies for adult depression: An overview of a series of meta-analyses. Canadian Psychology/Psychologie canadienne 2017;58(1):7-19. [CrossRef]
  6. Alonso J, Codony M, Kovess V, Angermeyer MC, Katz SJ, Haro JM, et al. Population level of unmet need for mental healthcare in Europe. Br J Psychiatry 2007 Apr;190:299-306 [FREE Full text] [CrossRef] [Medline]
  7. Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry 2017 Dec;210(2):119-124 [FREE Full text] [CrossRef] [Medline]
  8. Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported Internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev 2015;3:CD011565. [CrossRef] [Medline]
  9. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One 2014;9(5):e98118 [FREE Full text] [CrossRef] [Medline]
  10. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
  11. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
  12. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
  13. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205. [CrossRef] [Medline]
  14. Streiner DL. The 2 “Es” of research: efficacy and effectiveness trials. Can J Psychiatry 2002 Aug;47(6):552-556. [CrossRef] [Medline]
  15. Andersson G, Hedman E. Effectiveness of Guided Internet-Based Cognitive Behavior Therapy in Regular Clinical Settings. Verhaltenstherapie 2013;23(3):140-148. [CrossRef]
  16. Folker A, Mathiasen K, Lauridsen SM, Stenderup E, Dozeman E, Folker MP. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: A comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv 2018 Mar;11:60-70 [FREE Full text] [CrossRef] [Medline]
  17. Shadish WR, Matt GE, Navarro AM, Siegle G, Crits-Christoph P, Hazelrigg MD, et al. Evidence that therapy works in clinically representative conditions. J Consult Clin Psychol 1997 Jun;65(3):355-365. [Medline]
  18. Andersson G, Carlbring P, Cuijpers P. Internet interventions: Moving from efficacy to effectiveness. E-Journal of Applied Psychology 2009;5(2):18-24 [FREE Full text]
  19. Glasgow RE, Green LW, Klesges LM, Abrams DB, Fisher EB, Goldstein MG, et al. External validity: we need to do more. Ann Behav Med 2006 Apr;31(2):105-108. [CrossRef] [Medline]
  20. Hedman E, Ljótsson B, Kaldo V, Hesser H, El AS, Kraepelien M, et al. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord 2014 Feb;155:49-58. [CrossRef] [Medline]
  21. Hedman E, Ljótsson B, Rück C, Bergström J, Andersson G, Kaldo V, et al. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand 2013 Dec;128(6):457-467. [CrossRef] [Medline]
  22. El AS, Hedman E, Kaldo V, Hesser H, Kraepelien M, Andersson E, et al. Effectiveness of Internet-based cognitive-behavior therapy for social anxiety disorder in clinical psychiatry. J Consult Clin Psychol 2015 Oct;83(5):902-914. [CrossRef] [Medline]
  23. Nordgreen T, Gjestad R, Andersson G, Carlbring P, Havik OE. The implementation of guided Internet-based cognitive behaviour therapy for panic disorder in a routine-care setting: effectiveness and implementation efforts. Cogn Behav Ther 2018 Jan;47(1):62-75. [CrossRef] [Medline]
  24. Titov N, Dear BF, Staples LG, Bennett-Levy J, Klein B, Rapee RM, et al. The first 30 months of the MindSpot Clinic: Evaluation of a national e-mental health service against project objectives. Aust N Z J Psychiatry 2017 Dec;51(12):1227-1239. [CrossRef] [Medline]
  25. Hadjistavropoulos HD, Nugent MM, Alberts NM, Staples L, Dear BF, Titov N. Transdiagnostic Internet-delivered cognitive behaviour therapy in Canada: An open trial comparing results of a specialized online clinic and nonspecialized community clinics. J Anxiety Disord 2016 Dec;42:19-29 [FREE Full text] [CrossRef] [Medline]
  26. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med 2007 Oct 16;4(10):e297 [FREE Full text] [CrossRef] [Medline]
  27. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med 2007 Oct 16;4(10):e296 [FREE Full text] [CrossRef] [Medline]
  28. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [CrossRef] [Medline]
  29. Mental HSOSD. Internetpsykiatrien. 2018.   URL: http://www.internetpsykiatrien.dk/wm507341 [accessed 2018-08-21] [WebCite Cache]
  30. Folker AP, Mathiasen K, Lauridsen SM, Stenderup E, Dozeman E, Folker MP. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: A comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv 2018 Mar;11:60-70 [FREE Full text] [CrossRef] [Medline]
  31. Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, et al. ICBT in routine care: A descriptive analysis of successful clinics in five countries. Internet Interv 2018 Sep;13:108-115 [FREE Full text] [CrossRef] [Medline]
  32. Mathiasen K, Andersen TE, Riper H, Kleiboer AAM, Roessler KK. Blended CBT versus face-to-face CBT: a randomised non-inferiority trial. BMC Psychiatry 2016 Dec 05;16(1):432 [FREE Full text] [CrossRef] [Medline]
  33. Mathiasen K, Riper H, Ehlers LH, Valentin JB, Rosenberg NK. Internet-based CBT for social phobia and panic disorder in a specialised anxiety clinic in routine care: Results of a pilot randomised controlled trial. Internet Interv 2016 May;4:92-98 [FREE Full text] [CrossRef] [Medline]
  34. Marks I, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan;34(1):9-17. [Medline]
  35. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline]
  36. Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weert HC. Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review. Gen Hosp Psychiatry 2007;29(5):388-395. [CrossRef] [Medline]
  37. Löwe B, Unützer J, Callahan CM, Perkins AJ, Kroenke K. Monitoring depression treatment outcomes with the patient health questionnaire-9. Med Care 2004 Dec;42(12):1194-1201. [Medline]
  38. Pedersen SS, Mathiasen K, Christensen KB, Makransky G. Psychometric analysis of the Patient Health Questionnaire in Danish patients with an implantable cardioverter defibrillator (The DEFIB-WOMEN study). J Psychosom Res 2016 Dec;90:105-112. [CrossRef] [Medline]
  39. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
  40. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care 2008 Mar;46(3):266-274. [CrossRef] [Medline]
  41. Beard C, Björgvinsson T. Beyond generalized anxiety disorder: psychometric properties of the GAD-7 in a heterogeneous psychiatric sample. J Anxiety Disord 2014 Aug;28(6):547-552. [CrossRef] [Medline]
  42. Marks IM, Mathews AM. Brief standard self-rating for phobic patients. Behav Res Ther 1979;17(3):263-267. [Medline]
  43. Gillis MM, Haaga DAF, Ford GT. Normative values for the Beck Anxiety Inventory, Fear Questionnaire, Penn State Worry Questionnaire, and Social Phobia and Anxiety Inventory. Psychological Assessment 1995;7(4):450-455. [CrossRef]
  44. Oei T, Moylan A, Evans L. Validity and clinical utility of the Fear Questionnaire for anxiety-disorder patients. Psychological Assessment 1991;3(3):391-397. [CrossRef]
  45. Dunlap WP, Cortina JM, Vaslow JB, Burke MJ. Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods 1996;1(2):170-177. [CrossRef]
  46. R Foundation for Statistical Computing. R: A language and environment for statistical computing   URL: https://www.r-project.org/ [accessed 2018-10-30] [WebCite Cache]
  47. Pinheiro J, Bates D, DebRoy S, Sarkar D. R Core Team. 2017.   URL: https://cran.r-project.org/web/packages/nlme/index.html [accessed 2018-10-30] [WebCite Cache]
  48. Yucel RM. State of the Multiple Imputation Software. J Stat Softw 2011 Dec;45(1) [FREE Full text] [Medline]
  49. Weissman MM, Bland RC, Canino GJ, Faravelli C, Greenwald S, Hwu HG, et al. Cross-national epidemiology of major depression and bipolar disorder. JAMA 1996;276(4):293-299. [Medline]
  50. Olsen LR, Mortensen EL, Bech P. Prevalence of major depression and stress indicators in the Danish general population. Acta Psychiatr Scand 2004 Feb;109(2):96-103. [Medline]
  51. Christensen Sven Bundgaard Per Bech O. Prevalence of clinical (major) depression in general practice using the DSM-IV version of PRIME-MD. Int J Psychiatry Clin Pract 2001;5(1):49-54. [CrossRef] [Medline]
  52. Toft T, Fink P, Oernboel E, Christensen K, Frostholm L, Olesen F. Mental disorders in primary care: prevalence and co-morbidity among disorders. results from the functional illness in primary care (FIP) study. Psychol Med 2005 Aug;35(8):1175-1184. [Medline]
  53. Befolkningens højst fuldførte uddannelse.: Danmarks Statistik   URL: https:/​/dst.​dk/​da/​Statistik/​emner/​uddannelse-og-viden/​befolkningens-uddannelsesstatus/​befolkningens-hoejst-fuldfoerte-uddannelse [accessed 2018-10-30] [WebCite Cache]
  54. Paxling B, Almlöv J, Dahlin M, Carlbring P, Breitholtz E, Eriksson T, et al. Guided internet-delivered cognitive behavior therapy for generalized anxiety disorder: a randomized controlled trial. Cogn Behav Ther 2011;40(3):159-173. [CrossRef] [Medline]
  55. Andersson G, Hesser H, Veilord A, Svedling L, Andersson F, Sleman O, et al. Randomised controlled non-inferiority trial with 3-year follow-up of internet-delivered versus face-to-face group cognitive behavioural therapy for depression. J Affect Disord 2013 Dec;151(3):986-994. [CrossRef] [Medline]
  56. Berger T, Caspar F, Richardson R, Kneubühler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther 2011 Mar;49(3):158-169. [CrossRef] [Medline]
  57. Bower P, Kontopantelis E, Sutton A, Kendrick T, Richards DA, Gilbody S, et al. Influence of initial severity of depression on effectiveness of low intensity interventions: meta-analysis of individual patient data. BMJ 2013;346:f540 [FREE Full text] [Medline]
  58. El Alaoui S, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, et al. Predictors of Symptomatic Change and Adherence in Internet-Based Cognitive Behaviour Therapy for Social Anxiety Disorder in Routine Psychiatric Care. PLoS One 2015;10(4):e0124258 [FREE Full text] [CrossRef] [Medline]
  59. Warmerdam L, Van SA, Twisk J, Cuijpers P. Predicting outcome of Internet-based treatment for depressive symptoms. Psychother Res 2013;23(5):559-567. [CrossRef] [Medline]
  60. Høifødt RS, Mittner M, Lillevoll K, Katla S, Kolstrup N, Eisemann M, et al. Predictors of Response to Web-Based Cognitive Behavioral Therapy With High-Intensity Face-to-Face Therapist Guidance for Depression: A Bayesian Analysis. J Med Internet Res 2015 Sep 02;17(9):e197 [FREE Full text] [CrossRef] [Medline]
  61. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an 'individual patient data' meta-analysis. Psychol Med 2015 Oct;45(13):2717-2726. [CrossRef] [Medline]
  62. Bergman Nordgren L, Carlbring P, Linna E, Andersson G. Role of the working alliance on treatment outcome in tailored internet-based cognitive behavioural therapy for anxiety disorders: randomized controlled pilot trial. JMIR Res Protoc 2013 Jan 18;2(1):e4 [FREE Full text] [CrossRef] [Medline]
  63. Mol M, Dozeman E, Provoost S, van Schaik A, Riper H, Smit JH. Behind the Scenes of Online Therapeutic Feedback in Blended Therapy for Depression: Mixed-Methods Observational Study. J Med Internet Res 2018 May 03;20(5):e174 [FREE Full text] [CrossRef] [Medline]
  64. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice 1993;24(2):190-195. [CrossRef]
  65. Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disorders. Br J Clin Psychol 2010 Nov;49(Pt 4):455-471. [CrossRef] [Medline]
  66. El Alaoui S, Ljótsson B, Hedman E, Svanborg C, Kaldo V, Lindefors N. Predicting Outcome in Internet-Based Cognitive Behaviour Therapy for Major Depression: A Large Cohort Study of Adult Patients in Routine Psychiatric Care. PLoS One 2016;11(9):e0161191 [FREE Full text] [CrossRef] [Medline]
  67. Hadjistavropoulos HD, Pugh NE, Hesser H, Andersson G. Predicting Response to Therapist-Assisted Internet-Delivered Cognitive Behavior Therapy for Depression or Anxiety Within an Open Dissemination Trial. Behav Ther 2016 Mar;47(2):155-165. [CrossRef] [Medline]
  68. Nordgreen T, Havik OE, Ost LG, Furmark T, Carlbring P, Andersson G. Outcome predictors in guided and unguided self-help for social anxiety disorder. Behav Res Ther 2012 Jan;50(1):13-21. [CrossRef] [Medline]


FQ: Fear Questionnaire
GAD-7: Generalized Anxiety Disorder-7
iCBT: Internet-based cognitive behavioral therapy
OR: odds ratio
PHQ-9: Patient Health Questionnaire-9
RCT: randomized controlled trial


Edited by G Eysenbach; submitted 01.05.18; peer-reviewed by J Apolinário-Hagen, L Li; comments to author 17.08.18; revised version received 11.09.18; accepted 12.09.18; published 28.11.18

Copyright

©Kim Mathiasen, Heleen Riper, Tonny E Andersen, Kirsten K Roessler. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 28.11.2018.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.