This paper is in the following e-collection/theme issue:

e-Mental Health and Cyberpsychology (1402) Web-based and Mobile Health Interventions (3114)

Published on in Vol 15, No 10 (2013): October

For Whom Does It Work? Moderators of Outcome on the Effect of a Transdiagnostic Internet-Based Maintenance Treatment After Inpatient Psychotherapy: Randomized Controlled Trial

For Whom Does It Work? Moderators of Outcome on the Effect of a Transdiagnostic Internet-Based Maintenance Treatment After Inpatient Psychotherapy: Randomized Controlled Trial

For Whom Does It Work? Moderators of Outcome on the Effect of a Transdiagnostic Internet-Based Maintenance Treatment After Inpatient Psychotherapy: Randomized Controlled Trial

Authors of this article:

David Daniel Ebert1, 2 ;   Mario Gollwitzer3 ;   Heleen Riper1, 4 ;   Pim Cuijpers1, 4 ;   Harald Baumeister5, 6 ;   Matthias Berking1, 2
Article Authors Cited by (47) Tweetations (7) Metrics

    Original Paper

    1Innovation Incubator, Division Health-Training.Online, Leuphana University Lueneburg, Lüneburg, Germany

    2Division of Clinical Psychology and Psychotherapy, Department of Psychology, Philipps University of Marburg, Marburg, Germany

    3Division of Methodology and Social Psychology, Department of Psychology, Philipps University of Marburg, Marburg, Germany

    4Department of Clinical Psychology & EMGO Institute, VU Amsterdam, Amsterdam, Netherlands

    5Department of Medical Psychology and Medical Sociology, Faculty of Medicine, University of Freiburg, Freiburg, Germany

    6Department of Rehabilitation Psychology and Psychotherapy, Institute of Psychology, University of Freiburg, Freiburg, Germany

    Corresponding Author:

    David Daniel Ebert, PhD

    Innovation Incubator

    Division Health-Training.Online

    Leuphana University Lueneburg

    Main Building, Ground Floor

    Rotenbleicherweg 67

    Lüneburg, 21335

    Germany

    Phone: 49 17616781057

    Fax:49 64212824053

    Email: Ebert@leuphana.de


    Background: Recent studies provide evidence for the effectiveness of Internet-based maintenance treatments for mental disorders. However, it is still unclear which participants might or might not profit from this particular kind of treatment delivery.

    Objective: The study aimed to identify moderators of treatment outcome in a transdiagnostic Internet-based maintenance treatment (TIMT) offered to patients after inpatient psychotherapy for mental disorders in routine care.

    Methods: Using data from a randomized controlled trial (N=400) designed to test the effectiveness of TIMT, we performed secondary analyses to identify factors moderating the effects of TIMT (intervention) when compared with those of a treatment-as-usual control condition. TIMT involved an online self-management module, asynchronous patient–therapist communication, a peer support group, and online-based progress monitoring. Participants in the control condition had unstructured access to outpatient psychotherapy, standardized outpatient face-to-face continuation treatment, and psychotropic management. Self-reports of psychopathological symptoms and potential moderators were assessed at the start of inpatient treatment (T1), at discharge from inpatient treatment/start of TIMT (T2), and at 3-month (T3) and 12-month follow-up (T4).

    Results: Education level, positive outcome expectations, and diagnoses significantly moderated intervention versus control differences regarding changes in outcomes between T2 and T3. Only education level moderated change differences between T2 and T4. The effectiveness of the intervention (vs control) was more pronounced among participants with a low (vs high) education level (T2-T3: B=–0.32, SE 0.16, P=.049; T2-T4: B=–0.42, SE 0.21, P=.049), participants with high (vs low) positive outcome expectations (T2-T3: B=–0.12, SE 0.05, P=.02) and participants with anxiety disorder (vs mood disorder) (T2-T3: B=–0.43, SE 0.21, P=.04). Simple slope analyses revealed that despite some subgroups benefiting less from the intervention than others, all subgroups still benefited significantly.

    Conclusions: This transdiagnostic Internet-based maintenance treatment might be suitable for a wide range of participants differing in various clinical, motivational, and demographic characteristics. The treatment is especially effective for participants with low education levels. These findings may generalize to other Internet-based maintenance treatments.

    Trial Registration: International Standard Randomized Controlled Trial Number (ISRCTN): 28632626; http://www.controlled-trials.com/isrctn/pf/28632626 (Archived by WebCite at http://www.webcitation.org/6IqZjTLrx).

    J Med Internet Res 2013;15(10):e191

    doi:10.2196/jmir.2511

    Keywords

    maintenance treatment; continuation treatment; Internet-based intervention; transdiagnostic treatment; mental disorders/inpatient psychotherapy; guided self-help; randomized controlled trial; relapse prevention; predictors; moderators 


    Despite strong evidence for the efficacy of psychotherapy for common mental health disorders [1,2], long-term outcome of psychotherapeutic interventions are still a major concern [3-6]. Psychological treatments following acute phase psychotherapy that aim to maintain achieved changes (ie, maintenance phase treatments) have been shown to enhance outcome sustainability (eg, major depressive disorder [7,8], obsessive compulsive disorder [9], and personality disorders [10,11]). However, such interventions are difficult to disseminate owing to high intervention costs and limited clinician availability.

    The use of the Internet to provide guided self-help maintenance phase treatments may help to overcome this unmet maintenance need. Internet-based guided self-help strategies for the maintenance phase of psychotherapies have several advantages over face-to-face maintenance approaches. These include (1) greater potential for the integration of acquired skills in daily life because of an emphasis on the patient’s active role in (guided) self-help treatment [12], (2) elimination of waiting periods between acute and maintenance treatment, (3) elimination of travel time and costs for both patients and clinicians, (4) access to the programs on a 24/7 basis, and (5) lower costs.

    Several studies have shown promising results with delivering maintenance phase treatments over the Internet [13-20]. For example, our group developed a form of Internet-based continuation phase psychotherapy, a transdiagnostic Internet-based maintenance treatment (TIMT) following inpatient psychotherapy [15,20]. TIMT was designed to increase long-term outcomes of inpatients treated in a routine care setting for common mental health disorders, such as major depressive, anxiety, posttraumatic stress, obsessive compulsive, eating, or somatoform disorders. Recently, TIMT was evaluated in a pragmatic randomized controlled trial (RCT), comparing TIMT in addition to treatment as usual (TAU) to TAU only (N=400). In this study, participants in the TIMT plus TAU condition showed a better maintenance of inpatient treatment effects (ie, differences in change of psychopathological symptom severity) from inpatient discharge to 3-month follow-up (between-group effect size: d=0.38, P<.001) and 12-month follow-up (between-group effect size: d=0.55, P<.001) than TAU-only controls [15].

    Although there is evidence for the general effectiveness of Internet-based maintenance phase treatments, little is known about which patients might or might not benefit from this particular kind of treatment delivery. Investigating the moderating effects of patient characteristics on Internet-based maintenance phase treatment effectiveness is crucial for identifying appropriate populations and for customizing interventions to the specific needs of patient subgroups. More knowledge regarding who is likely or unlikely to profit from these interventions should also help in identifying relevant mechanisms of change as well as allocating health care resources on an evidence-based level [21].

    Only a few studies to date have investigated moderators of Internet-based intervention outcomes for mental health problems. In 1 of these studies, Warmerdam and colleagues [22] explored moderators of Internet-based cognitive behavioral therapy (CBT) and Internet-based problem-solving therapy for depressive symptoms. None of the variables investigated in this study (demographic variables, illness severity, dysfunctional attitudes, and problem-solving skills) moderated the differential effectiveness of the 2 treatments. In a study comparing the effects of Internet-based CBT to group-based face-to-face CBT, Spek et al [23] found that participants high in altruism performed better in group CBT than in Internet-based CBT (no significant findings for age, gender, education, neuroticism, extraversion, agreeableness, openness and conscientiousness, pretreatment severity, previous episodes of depression, and marital status). When comparing responses to online CBT for depression compared to a waitlist control group Button et al [24] found that higher pretreatment severity of depressive symptoms were associated with a greater benefit of treatment. In another study, de Graaf and colleagues [25] explored pretreatment and short-term improvement variables as moderators of unsupported Internet-based CBT outcomes, usual primary care (TAU), and CBT combined with TAU for depression. They found that patients with higher levels of extreme positive responding to questionnaires had a better outcome in Internet-based CBT compared to TAU, whereas those with parental psychiatric history or with a major depressive disorder diagnosis had a better outcome in Internet-based CBT plus TAU compared to TAU.

    The aim of the present study was to identify moderating factors on the effects of TIMT after inpatient psychotherapy. Using data from a pragmatic RCT on the effectiveness of TIMT (ISRCTN:28632626) [15], we conducted secondary analyses to identify demographic, clinical, and motivational variables that moderate the effects of TIMT on change in psychopathological symptom severity.

    Given the current lack of data on moderators of Internet-based continuation phase treatment effects, we used an exploratory approach including a wide range of potential pretreatment moderators [21]. Our choice of moderators was based on (1) results of previous studies investigating moderators of face-to-face continuation treatment outcomes [26,27], (2) results of previous predictors/moderators in Internet-based intervention outcome studies [22,23,25,28,29], (3) predictors of relapse/long-term outcome studies [5,27,30-36], and (4) theoretical assumptions attributed to intervention characteristics. The final list of potential moderators investigated in the present study included (1) demographics, such as age, gender, education level, and computer/Internet literacy; (2) clinical characteristics, such as diagnoses, remission status, age of first onset, comorbid personality disorder, and reliable change during inpatient treatment, and (c) motivational variables, such as self-efficacy and positive outcome expectations.

    The primary research questions of this study were:

    1. Do any of the pretreatment factors included in this study moderate the effectiveness of TIMT compared with TAU?
    2. If moderating effects are found, do participants characterized by disadvantageous scores on identified moderators still benefit from TIMT?

    Study Design

    We performed secondary analyses using data from a pragmatic RCT comparing TIMT in addition to TAU following inpatient psychotherapy to TAU only (N=400) [15]. The RCT was conducted in a German clinic providing routine mental health care. Study outcomes were assessed by using self-report measures that were completed at inpatient admission (T1), end of inpatient treatment/beginning of TIMT (T2), 3 months after discharge/end of TIMT (T3), and 12 months after inpatient treatment completion (T4). The study was powered to find a small to moderate effect size in the main effect analyses, which was considered to be the smallest relevant difference to health care decision makers in this context. All procedures were approved by the university and the hospital institutional review boards. Design and results of the effectiveness trial are described in detail in a previously published study [15].

    Participants and Procedures

    We recruited potential participants from 2189 patients treated for a variety of mental disorders between July 2008 and October 2009 in the study hospital. Patients were eligible for the study if they (1) were age 18 years or older, (2) met criteria for a mental disorder according to the International Classification of Diseases, Tenth Revision (ICD-10) [37], (3) spoke German sufficiently, (4) had basic reading and writing skills, and (5) had access to a computer with an Internet connection. Exclusion criteria were (1) a psychotic diagnosis, (2) acute alcohol or substance dependence, and (3) a significant risk for suicide.

    Participants who gave full written informed consent were randomly assigned to receive TAU only (control) or TAU plus TIMT (intervention). In total, 58 of 400 (14.5%) participants did not complete the T3 assessment and 113 (28.5%) did not complete the T4 assessment. Participants who did not provide data at 1 of the follow-ups did not differ from participants without missing data on baseline psychopathological symptom severity scores or any other clinical characteristics (all P values >.10), except for age (noncompleters on average 2.21 years younger than completers, P=.02). No significant interactions were found between missing pattern and outcome using pattern mixture analyses [38]. Thus, missing data appear not to bias the results. Figure 1 summarizes participant enrollment and flow throughout the study [15].

    Interventions

    Inpatient Treatment

    Inpatient treatment was based on CBT [39]. Participants received 1 session of individual therapy (50 minutes) and an average of 6 sessions of group therapy (90 minutes) per week. Interventions were supplemented with sports therapy and physiotherapy, as well as medical treatment (including pharmacotherapy) when necessary. Treatment was delivered by 6 experienced therapists and 14 therapists in training. Duration of treatment ranged between 22 and 98 days (mean 46.30, SD 8.17).

    Treatment as Usual Condition

    Following inpatient treatment, all participants had unstructured access to outpatient psychotherapy and standardized outpatient group-based, face-to-face, maintenance treatment [40] as typically provided by the referring agencies. In addition, there was no restriction on the use of medication during the study period.

    Treatment as Usual Plus Transdiagnostic Internet-Based Maintenance Treatment Condition

    In addition to TAU, the intervention group had TIMT for 12 weeks. The main focus of TIMT is to support patients in the sustained utilization of skills acquired during treatment. For this purpose, TIMT works to help participants identify activities that they have found helpful and systematically integrate these into their daily life routines. Because TIMT aims to enhance whatever strategy patients experienced as helpful, it can be used to maintain treatment outcome regardless of which psychopathology the patient is suffering from and regardless of the kind of treatment the patient received before. TIMT consists of 5 core components. The first component is the generation of a personal development plan. This process is conducted during the last 10 days of inpatient treatment in which TIMT participants complete 3 sessions of blended (face-to-face and online) standardized goal-setting and action planning instead of inpatient TAU. Participants develop a detailed plan including (1) highly relevant personal goals they want to achieve during the intervention phase, and (2) implementation intentions [41], including details on how and when they will achieve these goals. The second and central component of TIMT is the completion of a structured Web diary in which participants evaluate the realization of their personal goals weekly and set specific goals for the next week. The third component of TIMT is an online peer support group. Subgroups consisting of 3 to 6 participants are asked to give asynchronous online feedback to one another on their Web diaries. The fourth component of TIMT is coach support, involving weekly asynchronous written online feedback from a therapist regarding a participants’ Web diary. Coaches differed in their level of formal training, ranging from master’s level psychology students (n=1) and psychotherapists-in-training (n=1) to experienced CBT-trained psychotherapists with more than 10 years of professional experience (n=3). Coaches were supervised once a week by a licensed senior therapist, as is usual in the study hospital. Coaches were advised not to spend more than 30 minutes per week on support per patient. Total duration of support rendered was 231 minutes on average per patient (range: 10-490, SD 128). Finally, TIMT included weekly online monitoring of psychopathological symptoms.

    Treatment Received

    The intervention and control group did not differ in types of treatment received except for frequency of sedatives taken. Participants in the intervention group were less likely to take sedatives than controls (P<.001) [15].

    Measures

    Moderators

    In total, we included 11 pretreatment participant characteristics: age, sex, education, main diagnosis, comorbid personality disorder, remission status at the end of inpatient treatment, reliable change in the primary outcome during inpatient treatment, years since first disorder onset, Internet/computer literacy, positive outcome expectations, and health-related self-efficacy.

    Information on sex, age, and education were extracted from the inpatient clinic patient files. All self-report data were assessed using an online-based assessment tool. Diagnoses and year of first disorder onsets were assessed during the intake interview. All interviewers were experienced psychotherapists who were either psychologists or physicians with a master’s degree or higher, trained extensively in administering the structured clinical interviews of the German version of the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) [42]. Participants were classified as being remitted at inpatient discharge (yes/no) when individual scores in psychopathological symptom severity (primary outcome measure as described subsequently) exceeded a raw score value of 0.685 [43]). Reliable change in symptom severity (yes/no) was determined according to the widely used reliable change index of Jacobson and Truax [44]. Individual reliable change scores less than -1.96 were considered to reflect reliable (positive) change; scores equal to or greater than -1.96 reflected no reliable change. Participants were classified as Internet/computer illiterate if they checked the not at all response category for the item “I am used to sending and receiving emails” (1=not at all; 4=completely true). All other participants were classified as Internet/computer literate. Participants were coded as having a low education level if they reported 9 years of school education, as a medium education level if they reported 10 years of school education and a corresponding degree, or as a high education level if they reported a minimum of 13 years of school/college education and a corresponding degree.

    Positive outcome expectations were assessed by using the respective subscale of the Patient Questionnaire on Therapy Expectation and Evaluation (PATHEV) [45]. This scale consists of 4 items measuring participants’ expectations regarding the effectiveness of their inpatient treatment (eg, “I think that finally my problems will be solved”). Response scales ranged from 0 (=do not agree) to 4 (=agree completely). Higher scores reflect higher positive outcome expectations. Construct validity of the scale was demonstrated in several studies [45]. In the present study, internal consistency (Cronbach alpha) was .77.

    Health-related self-efficacy was assessed by using the self-efficacy subscale of the 49-item short form of the Hamburg Modules for the Assessment of Psychosocial Health (HEALTH-49) questionnaire [43].This scale includes 5 items measuring expected persistence and success in several domains (eg, “Despite my discomfort, I achieve the personal goals that I set for myself” score inverted for scale calculation; 0=not true, 4=very true). Higher scores denote lower self-efficacy. Internal consistency (Cronbach alpha) was .86 in the present study.

    Dependent Variable

    The primary outcome from the effectiveness trial was change in general psychopathological symptom severity (symptom severity) from discharge (T2) to 3- and 12-month follow-ups (T3, T4). Symptom severity was assessed by using the HEALTH-49, a widely used measure of symptom severity in Germany [43]. The HEALTH-49 general psychopathological symptom severity scale consists of 18 items related to somatoform complaints (7 items), depressiveness (6 items), and phobic anxiety (5 items). Participants were asked to rate the severity to which they had suffered from the presented symptoms in the previous 2 weeks (0=not at all; 4=very much). Reliability and construct validity have been established in several studies based on large clinical and nonclinical samples (1548 psychotherapy inpatients, 5630 primary care patients, see [43]). In the present study, internal consistency (Cronbach alpha coefficients) at baseline was .87 for the overall general psychopathological symptom severity score, 0.90 for depressive symptoms, 0.86 for somatoform complaints, and 0.86 for phobic anxiety.

    Statistical Analyses

    Group differences regarding baseline characteristics were compared via chi-square tests for categorical variables and t tests for continuous variables. Interactions between pretreatment participant characteristics (moderators) and interindividual differences in intraindividual changes across measurement occasions were modeled and tested via multilevel mixed-effect models. Change in symptom severity over time was dummy coded and treated as a fixed level-1 (ie, within-subjects) effect (dummy 1: T1-T2, dummy 2: T2-T3, dummy 3: T2-T4). Treatment conditions (0=control condition, 1=intervention condition) was treated as a fixed level-2 (ie, between-subjects) effect. More important for the present purpose, interactions between moderator and treatment condition, all cross-level interaction effects (condition × T1-T2, T2-T3, T2-T4; moderator × T1-T2, T2-T3, T2-T4), and 3-way interaction effects (moderator × condition × T1-T2, T2-T3, T2-T4) were also included in the models. A 3-way interaction effect of moderator × condition × T2-T3 or moderator × condition × T2-T4 would indicate that the magnitude of the intervention effect varies as a function of the moderator. The model imposed no restrictions on the covariance matrix for measurement occasions. Thus, no model assumptions were tested. We standardized continuous predictors so that regression coefficients were estimated for participants with average scores on the putative moderator.

    To increase interpretability and allow for testing nonlinear effects, categorical variables with more than 2 categories (ie, diagnosis, years since first disorder onset, education) were recoded into a maximum of 3 meaningful categories. Because of low prevalence rates, we excluded diagnoses other than depression, anxiety disorders, and adjustment disorders. All continuous moderators (ie, age, self-efficacy, positive outcome expectations) were standardized so that regression coefficients refer to participants with average scores on each moderator.

    Aiming at an intention-to-treat (ITT) design, we included all participants randomly assigned to conditions. We employed a full information maximum likelihood (FIML) estimation, which allows for all available data to be included without replacement or imputation of missing values. The FIML estimation for mixed models is especially robust with respect to missing data [46].

    Additionally, we conducted follow-up simple slope analyses for each significant 3-way interaction effect [47] to probe the relevant lower-order effects. In this method, the slope and the significance of the intervention main effect is evaluated for conditional values of the moderator. For significant 3-way interactions of continuous moderators, simple slopes were calculated for the mean and one standard deviation above and below the mean [48].

    Effect sizes for each significant moderator were calculated based on comparing the effect of control versus intervention groups on symptom severity scores, with participants grouped by the significant moderator variable. Cohen’s d scores [49] were calculated by standardizing the differences between baseline and follow-up by the pooled standard deviation of baseline scores.

    To verify whether the results of the ITT analyses would be sustained among the intervention completers sample only, we subsequently repeated all mixed-effects models with participants who stayed within key treatment parameters (completed at least 6 of 12 Web diary entries or more than 25 posts, n=177).

    To clarify the generalizability of our findings, we assessed all potential moderators also from patients who were treated during the recruitment period in the study center, but did not participate in the trial (not invited, declined to participate, not fulfilling inclusion criteria) but gave informed consent to use their data for research purposes (n=1789). Study participants and nonparticipants were compared using chi-square tests for categorical variables and t tests for continuous variables.

    Finally, if a significant moderator effect contradicted our a priori expectations, we conducted post hoc simple slope analyses for the control and intervention groups separately to identify the reasons for the effect. All analyses were performed with SPSS 19 (IBM Corp, Armonk, NY, USA).

    Descriptive Data

    Table 1 shows descriptive statistics for the dependent variable general psychopathological symptom severity. Table 2 shows descriptive data for all moderator variables. Table 1 and parts of Table 2 have been reported in previous studies [15]. Consistent with random assignment, no differences were found between intervention and control group on any of the pretreatment variables.

    ‎
    Figure 1. Participant flow and study dropouts at each stage of the study.
    View this figure
    Table 1. Descriptives for primary trial main outcome, psychopathological symptom severity as measured by the general psychopathological symptom severity subscale of the HEALTH-49 questionnaire.
    Assessment pointsTime of AssessmentIntervention (n=200)Control (n=200)


    		MeanSDMeanSD
    T1Inpatient admission1.500.691.490.71
    T2Inpatient discharge0.830.640.830.66
    T33-month follow-up0.710.610.960.69
    T412-month follow-up0.780.691.120.84
    Table 2. Descriptives for pretreatment moderator variables.
    VariablesIntervention (n=200)Control (n=200)Nonparticipantsa (n=1789)
    Age, mean (SD)45.09 (8.88)45.45 (9.80)47.12 (9.45)
    Sex (female), n (%)147 (73.5)151 (75.5)1360 (76.0)
    Education, n (%)



    		High80 (40.0)78 (39.0)498 (27.8)

    		Medium93 (46.5)91 (45.5)779 (43.5)

    		Low26 (13.0)31 (15.5)509 (28.5)
    Existing Internet literacy (%)178 (89.0)167 (83.5)1132 (67.5)b
    Disorder, n (%)



    		Mood disorder108 (54.0)113 (56.5)918 (51.3)

    		Anxiety19 (9.5)18 (9.0)206 (11.5)

    		Adjustment53 (26.5)38 (19.0)405 (22.6)

    		Other20 (10.0)31 (15.5)260 (14.5)
    Comorbid personality disorder, n (%)20 (10.0)22 (11.0)175 (9.8)
    Years since first disorder onset (years) n (%)



    		< 144 (22.0)47 (23.5)430 (24.2)

    		1-555 (27.5)44 (22.0)444 (24.9)

    		> 596 (48.0)105 (52.5)906 (50.9)
    Reliable change during inpatient treatment, n (%)100 (50.0)90 (45.0)1052 (58.8)
    Remission at discharge, n (%)94 (47.0)93 (46.5)787 (44.0)
    Self-efficacy, mean (SD)1.47 (0.83)1.49 (0.87)1.58 (0.90)
    Positive outcome expectations, mean (SD)3.86 (0.74)3.92 (0.66)3.72 (0.78)

    aAll differences between conditions were nonsignificant. If percentages do not reach 100, it is due to missing data.

    bn=1676.


    Moderators of Treatment Outcome

    Overview

    The subsequent tables show the mixed-effect model results based on ITT for the interactions between pretreatment participant characteristics (moderators), intervention condition, and changes in symptom severity. Intercepts represent the estimated level of symptom severity at baseline (discharge, T2). The regression coefficient of the moderator represents differences in symptom severity between participants differing in 1 unit of the hypothesized moderator at baseline. The regression coefficient of T1-T2 represents the average difference in symptom severity between inpatient admission (T1) and inpatient discharge (T2) in the control group, the regression coefficient of T2-T3 represents the average difference in symptom severity between discharge (T2) and 3-month follow-up (T3) in the control group, and the regression coefficient of T2-T4 represents the average difference in symptom severity between discharge (T2) and 1-year follow-up (T4) in the control group. The regression coefficient of the condition represents differences in symptom severity between the intervention and the control condition at discharge (T2). The cross-level interactions condition × T1-T2, T2-T3, T2-T4 represent intervention versus control group differences in changes over time.

    As expected, we found (1) a significant decrease in symptom severity between T1 and T2 in both conditions (T1-T2), (2) no interaction between T1-T2 and the intervention condition, (3) a significant condition × T2-T3 interaction effect showing that symptom severity remained low in the intervention group between T2 and T3 but increased in the control group, and (4) a significant T2-T4 × condition interaction effect showing that symptom severity remained low in the intervention group between T2 and T4 but increased in the control group (Table 4). The regression coefficients of the moderator × T1-T2, T2-T3, T2-T4 interaction effects represent moderator effects on changes in symptom severity across measurement occasions. Finally, the regression coefficient of condition × moderator × T1-T2, T2-T3, T2-T4 interaction effects represent moderator effects on intervention versus control condition differences on change scores over time.

    Dichotomous Moderator Variables

    Table 3 shows results for dichotomous moderator variables. The results revealed no moderator effects on intervention versus control group differences on changes in symptom severity over time (see Table 3, condition × moderator × T1-T2, T2-T3, T2-T4). Thus, none of the dichotomous moderators reliably altered the effectiveness of intervention versus control on symptom severity over time. The intervention was superior to control with regard to outcome sustainability, irrespective of sex, Internet literacy, reliable changes during inpatient treatment, comorbid personality disorder, or remission status at T2.

    Table 3. Multilevel results of the interactions between pretreatment participant characteristics (dichotomic moderator variables), intervention condition, and change in psychopathological symptom severity (dummy coded) for the intention-to-treat sample (N=400) using full maximum likelihood estimation.
    Interaction termsSexaInternet literacybReliable changecComorbid PDdRemission statuse

    		BSEPBSEPBSEPBSEPBSEP
    Interceptf0.940.09<.0010.910.11<.0010.980.06<.0010.780.05<.0010.300.04<.001
    Moderator–0.160.11.13–0.110.12.39–0.340.09<.0010.420.15.0040.980.06<.001
    T1-T2 (dummy 1)g0.600.08<.0010.690.10<.0010.250.03<.0010.660.04<.0010.840.06<.001
    T2-T3 (dummy 2)h0.200.08.0080.190.09.040.000.05.990.160.04<.0010.360.05<.001
    T2-T4 (dummy 3)i0.480.10<.0010.400.12.0010.120.06.050.300.05<.0010.440.07<.001
    Conditionj–0.350.13.006–0.130.18.480.050.09.580.020.07.780.010.06.90
    Condition×T1-T20.040.12.7500.070.16.66–0.060.05.250.040.06.53–0.040.08.61
    Condition×T2-T3–0.240.11.03–0.300.14.04–0.220.07.002–0.240.06<.001–0.300.07<.001
    Condition×T2-T4–0.450.14.001–0.650.18<.001–0.410.10<.001–0.340.07<.001–0.300.10.002
    Moderator×T1-T20.090.10.35–0.030.11.800.920.05<.0010.020.13.87–0.310.08<.001
    Moderator×T2-T3–0.040.09.61–0.020.10.830.360.07<.0010.040.12.75–0.370.07<.001
    Moderator×T2-T4–0.220.11.05–0.110.13.410.410.09<.0010.080.16.61–0.250.10.01
    Condition×moderator0.460.15.0020.140.19.47–0.070.13.57–0.230.21.28–0.020.09.83
    Cond×mod×T1-T2k–0.030.14.84–0.060.17.720.050.07.47–0.250.19.200.110.12.36
    Cond×mod×T2-T3k–0.020.12.890.060.15.71–0.080.10.46–0.120.18.490.120.10.22
    Cond×mod×T2-T4k0.120.16.470.340.20.090.060.14.67–0.220.23.34–0.110.14.41

    aSex (0=female; 1=male).

    bExisting Internet literacy (0=no; 1=yes).

    cReliable change: reliable change during inpatient treatment (0=no; 1=yes).

    dComorbid PD: comorbid personality disorder (0=no; 1=yes).

    eRemission status: remission status at baseline (T2) (0=in remission; 1=not in remission).

    fIntercept: general psychopathological symptom severity in control at baseline (T2).

    gT1-T2: dummy-coded change in general psychopathological symptom severity from T1 to T2.

    hT2-T3: dummy-coded change in general psychopathological symptom severity from T2 to T3.

    iT2-T4: dummy-coded change in general psychopathological symptom severity from T2 to T4.

    jCondition (0=control; 1=intervention).

    kCond × mod: condition × moderator.

    Trichotomous Moderator Variables

    Table 4 shows mixed-effects model results for the 3 trichotomous moderator variables education level, diagnoses, and years since first disorder onset. Three significant 3-way interaction effects were found. Education dummy 2 (low vs high education) interacted with condition × T2-T3, and with condition × T2-T4. These interactions indicate that a greater intervention effect was found among participants with low compared to high education level (see Figure 2). Participants low in education showed a larger intervention vs control condition difference on changes in symptom severity between discharge and 3-month follow-up and between discharge and 1-year follow-up. Post hoc analyses demonstrated that although simple slopes for the intervention main effects (condition × T2-T3, T2-T4) were lower among high-educated participants compared to low-educated participants, the intervention main effect was still significant (simple slope high-educated participants T2-T3: B=–0.17, SE 0.08, P=.04; T2-T4: B=–0.25, SE 0.11, P=.03; simple slope low-educated participants T2-T3: B=–0.49, SE 0.14, P<.001; T2-T4: B=–0.66, SE 0.18, P<.001).

    Moreover, diagnoses dummy 1 (mood disorders vs anxiety disorders) interacted with condition × T2-T3. Participants diagnosed with an anxiety disorder showed a larger intervention versus control group difference on changes in symptom severity between discharge and 3-month follow-up than participants diagnosed with a mood disorder (see Figure 3). Post hoc analyses demonstrate that although simple slopes for the intervention main effect (condition × T2-T3) were lower among participants with a mood disorder compared to participants with an anxiety disorder, the intervention main effect was significant in both groups (simple slope mood disorder T2-T3: B=–0.21, SE 0.07, P=.004; simple slope anxiety disorder T2-T3: B=–0.64, SE 0.02, P<.001). Diagnoses dummy 1 did not moderate the association between treatment and change in symptom severity from discharge to 1-year follow-up.

    Years since disorder onset did not moderate the effect of treatment on any intervention versus control group differences on change scores. Thus, transdiagnostic Internet-based maintenance treatment is effective irrespective of years since first disorder onset.

    ‎
    Figure 2. Estimated course of symptoms based on simple slope mixed-effect model analysis for significant moderators effect of education (0=high education, n=159; 1=low education, n=57) at inpatient admission (T1), inpatient discharge/begin transdiagnostic Internet-based maintenance treatment (T2), 3-month follow-up/end transdiagnostic Internet-based maintenance treatment (T3), and 12-month follow-up (T4).
    View this figure
    ‎
    Figure 3. Estimated course of symptoms based on simple slope mixed-effect model analyses for significant moderator effect of diagnoses (0=mood disorder, n=221; 1=anxiety disorder, n=37) at inpatient admission (T1), inpatient discharge/begin transdiagnostic Internet-based maintenance treatment (T2), 3-month follow-up/end transdiagnostic Internet-based maintenance treatment (T3), and 12-month follow-up (T4).
    View this figure
    Table 4. Multilevel results for interactions between pretreatment participant characteristics (trichotomous moderator variables), intervention condition, and change in psychopathological symptom severity (dummy coded) for the intention-to-treat sample (N=400) using full maximum likelihood estimation.
    Interaction termsEducation levelaDiagnosesbYears since onsetc

    		BSEPBSEPBSEP
    Interceptd0.880.07<.0010.870.06<.0010.880.09<.001
    Moderator dummy 1–0.160.10.110.440.16.005–0.040.11.687
    Moderator dummy 20.080.14.55–0.340.12.004–0.130.13.313
    T1-T2e0.630.07<.0010.730.06<.0010.640.09<.001
    T2-T3f0.170.06.0050.160.05.0020.100.08.18
    T2-T4g0.220.08.0040.360.06<.0010.310.10.002
    Conditionh–0.140.10.17–0.020.08.82–0.280.13.02
    Condition×T1-T20.030.09.76–0.020.08.850.090.12.43
    Condition×T2-T3–0.170.08.04–0.210.07.004–0.200.10.06
    Condition×T2-T4–0.250.11.03–0.380.09<.001–0.330.14.02
    Moderator dummy 1×T1-T20.030.09.72–0.040.15.80–0.040.11.70
    Moderator dummy 1×T2-T3–0.040.08.640.020.15.890.040.09.68
    Moderator dummy 1×T2-T40.070.11.480.010.19.96–0.030.12.80
    Moderator dummy 2×T1-T20.170.13.17–0.090.11.400.200.12.10
    Moderator dummy 2×T2-T30.100.11.350.030.10.750.140.11.19
    Moderator dummy 2×T2-T40.350.14.02–0.190.13.140.020.14.88
    Cond×mod×dummy 1i0.220.14.11–0.070.22.740.360.15.02
    Cond×mod×dummy 2i0.190.20.340.080.16.630.270.18.13
    Cond×mod×dummy 1×T1-T2i–0.030.13.82–0.070.22.75–0.070.15.63
    Cond×mod×dummy 1×T2-T3i–0.080.11.50–0.430.21.04–0.030.13.82
    Cond×mod×dummy 1×T2-T4i–0.120.15.42–0.240.26.370.010.17.93
    Cond×mod×dummy 2×T1-T2i0.050.18.770.100.15.52–0.140.17.43
    Cond×mod×dummy 2×T2-T3i–0.320.16.049–0.030.14.83–0.090.15.56
    Cond×mod×dummy 2×T2-T4i–0.420.21.0490.150.18.410.030.19.89

    aEducation level dummy 1 (0=high education level; 1=medium education level), education level dummy 2 (0=high education level; 1=low education level).

    bDiagnoses dummy 1 (0=mood disorder; 1=anxiety disorder), diagnoses dummy 2 (0=mood disorder; 1=adjustment disorder).

    cYears since onset: years since disorder onset dummy 1 (0=1-5 years; 1=>5 years), years since disorder onset dummy 2 (0=1-5 years; 1=<1 year).

    dIntercept: general psychopathological symptom severity in control at baseline (T2).

    eT1-T2: dummy-coded change in general psychopathological symptom severity from T1 to T2.

    fT2-T3: dummy-coded change in general psychopathological symptom severity from T2 to T3.

    gT2-T4: dummy-coded change in general psychopathological symptom severity from T2 to T4.

    hCondition (0=control; 1=intervention).

    iCond × mod × dummy: condition × moderator × dummy.

    Continuous Moderator Variables

    Table 5 shows mixed-effect model results for the continuous moderator variables age, self-efficacy, and positive outcome expectations. One significant 3-way interaction was found. Positive outcome expectations interacted with condition × T2-T3. This interaction indicates that more positive outcome expectations were associated with stronger intervention effects between discharge and 3-month follow-up (see Figure 4). Follow-up analyses revealed that although simple slopes for the intervention main effect (condition × T2-T3) were lower among participants with a moderate (mean) positive outcome expectation than for participants with a high (mean + 1 SD) positive outcome expectations, the intervention effect was still significant (simple slope moderate positive outcome expectations T2-T3: B=–0.25, SE 0.05, P<.001; simple slope high positive outcome expectations T2-T3: B=–0.36, SE 0.07, P<.001). For participants with a low positive outcome expectations (mean – 1 SD), the simple slope for the intervention main effect (condition × T2-T3) was lower and no longer significant (simple slope low positive outcome expectations T2-T3: B=–0.13, SE 0.07, P=.09). Only 14.4% of participants (57/400) expressed low positive outcome expectations. Therefore, the drop to nonsignificance was likely because of low power. Moreover, simple slope analyses for this participant group showed that the intervention main effect on change in symptom severity from discharge to 1-year follow-up was significant (simple slope low positive outcome expectations T2-T4: B=–0.38, SE 0.10, P<.001). Although short-term effects were not significant, participants with low positive outcome expectations benefited in the long term from the intervention. There was no interaction between positive outcome expectations and change in symptom severity from discharge to 1-year follow-up and no interaction effect including the other continuous variables age and self-efficacy. Thus, TIMT seems to be effective irrespective of age and self-efficacy.

    Table 5. Multilevel results for interactions between pretreatment participant characteristics (continuous moderator variables), intervention condition, and change in psychopathological symptom severity (dummy coded) for intention-to-treat sample (N=400) using full maximum likelihood estimation.
    Interaction termsAgeaSelf efficacyaPositive outcome expectationsa

    		BSEPBSEPBSEP
    Interceptb0.830.05.<0010.820.04<.0010.840.04<.001
    Moderatorc–0.120.04.0040.440.03<.001–0.230.05<.001
    T1-T2d0.670.04<.0010.670.04<.0010.660.04<.001
    T2-T3e0.170.04<.0010.170.04<.0010.160.04<.001
    T2-T4f0.310.05<.0010.310.05<.0010.300.05<.001
    Conditiong–0.010.06.850.010.05.89–0.010.06.84
    Condition×T1-T20.020.06.790.010.06.880.020.06.75
    Condition×T2-T3–0.240.05<.001–0.250.05<.001–0.250.05<.001
    Condition×T2-T4–0.350.07<.001–0.360.07<.001–0.350.07<.001
    Moderator×T1-T20.010.04.79–0.180.04<.0010.080.04.08
    Moderator×T2-T30.070.04.07–0.160.04.0020.070.04.08
    Moderator×T2-T40.050.05.26–0.050.05.260.030.05.62
    Condition×moderator0.030.06.60–0.040.05.420.120.06.06
    Condition×moderator×T1-T20.030.06.600.050.06.39–0.100.06.09
    Condition×moderator×T2-T30.040.05.500.070.05.22–0.120.05.02
    Condition×moderator×T2-T40.000.07.98–0.100.07.150.030.07.65

    aAll continuous variables standardized.

    bIntercept: general psychopathological symptom severity in control at baseline (T2).

    cModerators (0=mean; 1=mean + 1 SD).

    dT1-T2: dummy-coded change in general psychopathological symptom severity from T1 to T2.

    eT2-T3: dummy-coded change in general psychopathological symptom severity from T2 to T3.

    fT2-T4: dummy-coded change in general psychopathological symptom severity from T2 to T4.

    gCondition (0=control; 1=intervention).

    ‎
    Figure 4. Estimated course of symptoms based on simple slope mixed-effect model analyses for significant moderator positive outcome expectations (mean vs mean – 1 SD vs mean + 1 SD) at inpatient admission (T1), inpatient discharge/begin transdiagnostic Internet-based maintenance treatment (T2), 3-month follow-up/end transdiagnostic Internet-based maintenance treatment (T3), and 12-month follow-up (T4).
    View this figure
    Effect Sizes

    Effect sizes (Cohen’s d) for each significant moderator were calculated based on comparing the effect of control versus intervention condition on symptom severity, with participants grouped by parameter values on each significant moderator variable. A mean effect size of d=0.22 was found for participants with high education and d=0.80 for participants with low education for control versus intervention group differences in change of psychopathological symptom severity from discharge to 3-month follow-up. For change from discharge to 1-year follow-up, a mean effect size of d=0.30 for high-educated participants and a mean effect size of d=0.57 for low-educated participants was found. With diagnoses as the moderator, control versus intervention group differences in change from discharge to 3-month follow-up were d=0.33 for participants with a mood disorder and d=1.02 for participants with an anxiety disorder. With positive outcome expectations as moderator, control versus intervention group differences in change from discharge to 3-month follow-up were d=0.58 for participants with high positive outcome expectations, d=0.39 for participants with mean positive outcome expectations, and d=0.20 for participants with low positive outcome expectations.

    Intervention Completers Sample

    The results of the following intervention completers analyses closely paralleled those of the ITT analyses. Most of the significant 3-way interactions were also significant in the completers sample (B=–0.45 to –0.12, SE 0.05-0.21, P=.03-.046). Only the interaction of education dummy 2 with condition × T2-T4 was no longer significant at follow-up (B=–0.43, SE 0.22, P=0.05). None of the nonsignificant interactions in the ITT analyses was significant in the completers sample (B=–0.11 to 0.02, SE 0.07-0.20, P=.08-.97).

    Generalizability

    As shown in Table 1 and partly reported in previous studies [15], study participants did not differ from nonparticipants (n=1789) regarding sex (χ21=0.4, P=.52), years since first disorder onset (χ22=0.1, P=.93), existing comorbid personality disorder (χ21=0.5, P=.46), or remission status at the end of inpatient treatment (χ21=1.0, P=.32) or initial psychopathological symptom severity at inpatient admission (study participant symptom severity T1: mean 1.49, SD 0.70; nonparticipant symptom severity T1: mean 1.52, SD 0.84, t679.03=–0.69, P=.54). Study participants were significantly younger than nonparticipants (with an average difference of 1.7 years, t2135=–3.54, P<.001), had higher self-efficacy (t2182=–2.11, P=.04, d=0.15) had a slightly higher education level (χ22=40.81, P<.001, Kendall’s tau coefficient=0.11), had higher positive outcome expectations (t625.6=4.07, P<.001, d=0.27). Compared to nonparticipants, a greater percentage of participants had access to the Internet (χ21=47.3, P<.001, phi coefficient=0.15) were Internet literate (χ21=62.7, P<.001, phi coefficient=0.17), and relatively fewer showed reliable change during inpatient treatment (χ21=5.3, P=.02, phi coefficient=0.05).

    Post Hoc Analyses

    The moderator effect of education contradicted our a priori expectation of higher educated participants benefiting to a greater extent from the Internet-based intervention than lower educated participants. Thus, we conducted further post hoc simple slope analyses for the control group and the intervention group separately to identify possible explanations for this effect. For participants in the control group, we found no significant interaction between education and changes in symptom severity from discharge to 3-month follow-up (education dummy 2 × T2-T3 interaction, B=0.10, SE 0.11, P=.35), but we found a significant interaction between education and changes from discharge to 12-month follow-up (education dummy 2 × T2-T4, B=0.35, SE 0.14, P=.02). Less-educated participants had a greater risk for deterioration from discharge to 1-year follow-up than more-educated participants did. In contrast, we found no significant interaction of low compared to high education level in the intervention group, neither for changes in symptom severity from discharge to 3-month follow-up (B=0.21, SE 0.17, P=.07) nor for changes from discharge to 12-month follow-up (B=0.06, SE 0.15, P=.68). In contrast to the control group, less-educated intervention participants did not show a greater risk for deterioration in symptom severity than more-educated participants, indicating that participating in TIMT can effectively reduce this risk factor.


    Principal Results and Comparison With Prior Work

    In the present study, we aimed to identify moderators of treatment outcome for TIMT following inpatient psychotherapy. Education level, positive outcome expectations, and mental health diagnoses were identified as significant moderators of TIMT’s effects on psychopathological symptom severity. Findings indicate that the effects of TIMT on general psychopathological symptom severity were more pronounced among participants with a low (vs high) education level. Participants with high positive outcome expectations profited in the short term (until 3-month follow-up) more than participants with low positive outcome expectations. However, this effect was not significant at 1-year follow-up. Moreover, participants with a mood disorder benefited less from the intervention than did participants with an anxiety disorder; however, this effect was also not significant at 1-year follow-up. Simple slope analyses revealed that even when some groups profited less from participating, treatment effects in these subgroups were still significant, except for the subgroup of participants with low positive outcome expectation at 3-month follow-up.

    Other pretreatment variables did not interact with TIMT’s effects indicating that TIMT might be superior to TAU only with regard to outcome sustainability irrespective of age, gender, comorbid personality disorder, years since disorder onset, self-efficacy, remission status at the end of inpatient treatment, reliable change in psychopathological symptom severity during inpatient treatment, and Internet literacy. However, given that these analyses were exploratory and the study was not powered to find small interaction effects, these null findings should be interpreted with caution.

    The finding that participants with low education benefited more from using TIMT than participants with high education contrasts with findings from a study investigating moderators in face-to-face continuation phase psychotherapy in which education did not interact with treatment outcome [27]. Moreover, the finding is also in contrast to 3 other studies, that found that high education was associated with a better treatment outcome in Internet-based intervention studies [23,29,50]. There are several possible explanations for the contrast between the current findings and findings from previous studies: First, these differences can be explained with variances in treatment type (acute vs maintenance phase; disorder-specific vs transdiagnostic), different type of acute phase treatment (outpatient vs inpatient), study population, and design. Second, it could also be hypothesized that inpatients with low education might display a higher risk for deterioration after inpatient discharge than those with high education because of their more pronounced difficulties with transferring the acquired skills into their daily life. Therefore, they might profit to a greater extent from a maintenance intervention than participants with high education. This assumption is in-line with a risk-reduction model of continuation phase treatments [8], assuming that such concepts may effectively reduce an increased risk for relapse or recurrence because of a nonchangeable vulnerability (eg, education, genetic predisposition, developmental conditions) by helping participants to reduce the consequences of such risk factors. Post hoc simple slope analyses revealed that in this study the control group of participants with low education were more likely to deteriorate compared to highly educated participants, whereas in the intervention group no such interaction could be found, indicating that participating in TIMT can effectively reduce this risk factor. Moreover, the inpatient treatments present the rather unique opportunity to introduce patients to the online-based intervention face-to-face and to teach them the necessary skills for using the intervention successfully. Therefore, 1 possible mechanism responsible for the findings in studies in which participants with low education profited less from Internet-based treatments (ie, low Internet skills) no longer has any effect. However, as this study is 1 of the first studies investigating moderators of outcomes in maintenance phase treatments following inpatient psychotherapy, future studies are clearly needed to further clarify the moderating role of education for treatment outcome.

    On the basis of our data, we can only speculate on possible explanations as to why participants with anxiety disorder profited to a greater extent (in the short term) than participants with depression. These results are consistent with findings showing that effect sizes are typically larger for Internet interventions targeting anxiety than interventions targeting depression. In a review of 26 RCTs, Griffiths and colleagues [51] found that effect size differences ranged from 0.42 to 0.65 for interventions involving participants with clinically significant symptoms of depression, and 0.29 to 1.74 for interventions involving participants with a diagnosed anxiety disorder. Unlike guidelines for the treatment of depression [52,53], current guidelines for the treatment of anxiety disorders [1,54] do not recommend continuation phase psychological treatments following acute phase psychotherapy. Our findings, however, suggest that participants with anxiety disorder can benefit from an Internet-based maintenance treatment following inpatient psychotherapy. With regard to the subgroup effects for depression, future studies should try to examine treatment strategies to improve TIMT’s outcome, especially for this high-risk group [3]

    The significant finding for positive outcome expectancies regarding change differences from discharge to 3-month follow-up is consistent with the idea that high expectancies for change are associated with better treatment outcome [45,55]. However, these change differences turned insignificant at 1-year follow-up. Therefore, its use as a predictive indicator for treatment allocation seems limited.

    Limitations

    To validly interpret the results of this study, several limitations should be considered. First, as in most moderator studies, the analyses in this study were exploratory with participants not being randomized based on potential moderators of interest. Despite the limitations of this procedure, a growing recognition among methodologists has developed about its importance for fostering empirically founded hypotheses to be tested in future studies before clinical application [21]. Second, additional unmeasured variables (eg, participants’ genetic markers, developmental histories, self-regulation skills, coping strategies, attribution style, personality traits) may also moderate TIMT’s effects, which should be considered in subsequent studies. Third, as in most longitudinal studies, missing values had to be considered a relevant threat to the validity of the analyses. However, the adjustment used to address missing data (FIML) is especially robust with regard to missing data in mixed models [46]. Fourth, TIMT was a multicomponent intervention (ie, personal development plan, Web diary, peer support group, coach support, monitoring). Thus, the extent that the effects of specific components were moderated by studied variables is still unclear. Fifth, the sample size did not provide sufficient power to detect significant findings for potential moderator variables with subgroups of small sizes. Because of this limitation, other diagnoses in addition to mood disorder, adjustment, and anxiety disorders as potential moderating variables could not be included in the conducted analyses. Therefore, no generalization can be made for participants with other diagnoses. Likewise, the sample size did not provide sufficient power to examine moderators separately for different diagnostic status or gender. Therefore, it remains unclear whether moderators of TIMT’s effects vary across subpopulation (eg, different moderators for male and female or for different primary diagnoses). Compared with the nonparticipant cohort, individuals with low education were underrepresented in the study sample. Thus, the finding that TIMT was especially effective for participants with a low education level may only be generalizable to low-educated participants who are interested in participating in such an intervention. Finally, the sample included in this study was recruited in only 1 inpatient hospital, which clearly limits the generalizability of the findings to other patient populations.

    Strengths

    Strengths of the study include (1) its large sample size compared to other studies, (2) a TAU control condition, which allowed us to specify which participants might and might not benefit from TIMT compared to treatment provided by routine health care services, (3) inclusion and exclusion criteria were kept to a minimum to maximize the ecological validity, and (4) generalizability of findings was assessed by comparing the moderator sample with a large sample of participants representing basically all patients treated in the study site.

    Conclusions

    Transdiagnostic Internet-based guided self-help interventions may represent a cost-effective, far-reaching method for implementing maintenance phase treatments. Findings from the current study suggest that TIMT following inpatient psychotherapy helps patients differing in various characteristics to maintain treatment outcome. It is especially effective for participants with low education levels. Although some subgroups were identified as having profited less from the intervention than others, all subgroups benefited significantly. Future studies should replicate our results before clinical application.

    Acknowledgments

    David Ebert and Matthias Berking contributed to the design of the study. David Ebert and Mario Gollwitzer contributed to the analysis of data. All authors contributed to the interpretation of data, to the final draft of the paper, and approved the final version for publication. This study was funded by the Dr Ebel Fachkliniken–Vogelsbergklinik, Grebenhain, Germany and the European Union (EFRE: CCI 2007DE161PR001).

    Conflicts of Interest

    The first author (DE) developed the intervention under study.

    ‎

    Multimedia Appendix 1

    CONSORT-EHEALTH checklist V1.6.2 [56].

    PDF File (Adobe PDF File), 1MB
    1. American Psychiatric Association (APA). Practice Guideline for the Treatment of Patients with Panic Disorder, 2nd ed. Washington, DC: American Psychiatric Association (APA); 2009:1-90.
    2. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev 2006 Jan;26(1):17-31. [CrossRef] [Medline]
    3. Vittengl JR, Clark LA, Dunn TW, Jarrett RB. Reducing relapse and recurrence in unipolar depression: a comparative meta-analysis of cognitive-behavioral therapy's effects. J Consult Clin Psychol 2007 Jun;75(3):475-488 [FREE Full text] [CrossRef] [Medline]
    4. Durham RC, Chambers JA, Power KG, Sharp DM, Macdonald RR, Major KA, et al. Long-term outcome of cognitive behaviour therapy clinical trials in central Scotland. Health Technol Assess 2005 Nov;9(42):1-174 [FREE Full text] [Medline]
    5. Halmi KA, Agras WS, Mitchell J, Wilson GT, Crow S, Bryson SW, et al. Relapse predictors of patients with bulimia nervosa who achieved abstinence through cognitive behavioral therapy. Arch Gen Psychiatry 2002 Dec;59(12):1105-1109. [Medline]
    6. Chung T, Maisto SA. Relapse to alcohol and other drug use in treated adolescents: review and reconsideration of relapse as a change point in clinical course. Clin Psychol Rev 2006 Mar;26(2):149-161. [CrossRef] [Medline]
    7. Bockting CL, Schene AH, Spinhoven P, Koeter MW, Wouters LF, Huyser J, et al. Preventing relapse/recurrence in recurrent depression with cognitive therapy: a randomized controlled trial. J Consult Clin Psychol 2005 Aug;73(4):647-657. [CrossRef] [Medline]
    8. Jarrett RB, Vittengl JR, Clark LA. Preventing recurrent depression. In: Whisman MA, editor. Adapting Cognitive Therapy for Depression: Managing Complexity and Comorbidity. New York: Guilford Press; 2008:132-156.
    9. McKay D, Todaro JF, Neziroglu F, Yaryura-Tobias J. Evaluation of a naturalistic maintenance program in the treatment of obsessive-compulsive disorder: A preliminary investigation. Journal of Anxiety Disorders 1996 May;10(3):211-217. [CrossRef]
    10. Wilberg T, Karterud S, Pedersen G, Urnes O, Irion T, Brabrand J, et al. Outpatient group psychotherapy following day treatment for patients with personality disorders. J Pers Disord 2003 Dec;17(6):510-521. [Medline]
    11. Leirvåg H, Pedersen G, Karterud S. Long-term continuation treatment after short-term day treatment of female patients with severe personality disorders: Body awareness group therapy versus psychodynamic group therapy. Nord J Psychiatry 2010 Apr;64(2):115-122. [CrossRef] [Medline]
    12. Klaw E, Humphreys K. Facilitating client involvement in self-help groups. In: Koocher GP, Norcross JC, Hill SS, editors. Psychologists’ Desk Reference, 2nd ed. New York: Oxford University Press; 2005:502-505.
    13. Barnes C, Harvey R, Mitchell P, Smith M, Wilhelm K. Evaluation of an online relapse prevention program for bipolar disorder. Disease Management & Health Outcomes 2007;15(4):215-224. [CrossRef]
    14. Bauer S, Wolf M, Haug S, Kordy H. The effectiveness of Internet chat groups in relapse prevention after inpatient psychotherapy. Psychother Res 2011 Mar;21(2):219-226. [CrossRef] [Medline]
    15. Ebert D, Tarnowski T, Gollwitzer M, Sieland B, Berking M. A transdiagnostic Internet-based maintenance treatment enhances the stability of outcome after inpatient cognitive behavioral therapy: a randomized controlled trial. Psychother Psychosom 2013;82(4):246-256. [CrossRef] [Medline]
    16. Golkaramnay V, Bauer S, Haug S, Wolf M, Kordy H. The exploration of the effectiveness of group therapy through an Internet chat as aftercare: a controlled naturalistic study. Psychother Psychosom 2007;76(4):219-225. [CrossRef] [Medline]
    17. Gulec H, Moessner M, Mezei A, Kohls E, Túry F, Bauer S. Internet-based maintenance treatment for patients with eating disorders. Professional Psychology: Research and Practice 2011;42(6):479-486. [CrossRef]
    18. Holländare F, Johnsson S, Randestad M, Tillfors M, Carlbring P, Andersson G, et al. Randomized trial of Internet-based relapse prevention for partially remitted depression. Acta Psychiatr Scand 2011 Oct;124(4):285-294. [CrossRef] [Medline]
    19. Kordy H, Theis F, Wolf M. [Modern information and communication technology in medical rehabilitation. Enhanced sustainability through Internet-delivered aftercare]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2011 Apr;54(4):458-464. [CrossRef] [Medline]
    20. Ebert DD, Hannig W, Tarnowski T, Sieland B, Götzky B, Berking M. Web-based rehabilitation aftercare following inpatient psychosomatic treatment. Rehabilitation (Stuttg) 2013 Jun;52(3):164-172. [CrossRef] [Medline]
    21. Kraemer HC, Wilson GT, Fairburn CG, Agras WS. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry 2002 Oct;59(10):877-883. [Medline]
    22. Warmerdam L, van Straten A, Jongsma J, Twisk J, Cuijpers P. Online cognitive behavioral therapy and problem-solving therapy for depressive symptoms: Exploring mechanisms of change. J Behav Ther Exp Psychiatry 2010 Mar;41(1):64-70. [CrossRef] [Medline]
    23. Spek V, Nyklícek I, Cuijpers P, Pop V. Predictors of outcome of group and Internet-based cognitive behavior therapy. J Affect Disord 2008 Jan;105(1-3):137-145. [CrossRef] [Medline]
    24. Button KS, Wiles NJ, Lewis G, Peters TJ, Kessler D. Factors associated with differential response to online cognitive behavioural therapy. Soc Psychiatry Psychiatr Epidemiol 2012 May;47(5):827-833. [CrossRef] [Medline]
    25. de Graaf LE, Hollon SD, Huibers MJ. Predicting outcome in computerized cognitive behavioral therapy for depression in primary care: A randomized trial. J Consult Clin Psychol 2010 Apr;78(2):184-189. [CrossRef] [Medline]
    26. Bockting CL, Spinhoven P, Koeter MW, Wouters LF, Visser I, Schene AH, DELTA study group. Differential predictors of response to preventive cognitive therapy in recurrent depression: a 2-year prospective study. Psychother Psychosom 2006;75(4):229-236. [CrossRef] [Medline]
    27. Vittengl JR, Clark LA, Jarrett RB. Moderators of continuation phase cognitive therapy's effects on relapse, recurrence, remission, and recovery from depression. Behav Res Ther 2010 Jun;48(6):449-458 [FREE Full text] [CrossRef] [Medline]
    28. Andersson G, Bergström J, Holländare F, Ekselius L, Carlbring P. Delivering cognitive behavioural therapy for mild to moderate depression via the Internet: predicting outcome at 6-month follow-up. Verhaltenstherapie 2004;14(3):185-189. [CrossRef]
    29. Riper H, Kramer J, Keuken M, Smit F, Schippers G, Cuijpers P. Predicting successful treatment outcome of web-based self-help for problem drinkers: secondary analysis from a randomized controlled trial. J Med Internet Res 2008;10(4):e46 [FREE Full text] [CrossRef] [Medline]
    30. Brown TA, Barlow DH. Long-term outcome in cognitive-behavioral treatment of panic disorder: clinical predictors and alternative strategies for assessment. J Consult Clin Psychol 1995 Oct;63(5):754-765. [Medline]
    31. Katschnig H, Amering M. The long-term course of panic disorder and its predictors. J Clin Psychopharmacol 1998 Dec;18(6 Suppl 2):6S-11S. [Medline]
    32. Jarrett RB, Kraft D, Doyle J, Foster BM, Eaves GG, Silver PC. Preventing recurrent depression using cognitive therapy with and without a continuation phase: a randomized clinical trial. Arch Gen Psychiatry 2001 Apr;58(4):381-388 [FREE Full text] [Medline]
    33. Judd LL, Paulus MP, Zeller P. The role of residual subthreshold depressive symptoms in early episode relapse in unipolar major depressive disorder. Arch Gen Psychiatry 1999 Aug;56(8):764-765. [Medline]
    34. Judd LL, Akiskal HS, Maser JD, Zeller PJ, Endicott J, Coryell W, et al. Major depressive disorder: a prospective study of residual subthreshold depressive symptoms as predictor of rapid relapse. J Affect Disord 1998 Sep;50(2-3):97-108. [Medline]
    35. ten Doesschate MC, Bockting CL, Koeter MW, Schene AH, DELTA Study Group. Prediction of recurrence in recurrent depression: a 5.5-year prospective study. J Clin Psychiatry 2010 Aug;71(8):984-991. [CrossRef] [Medline]
    36. Bockting CL, Spinhoven P, Koeter MW, Wouters LF, Schene AH, Depression Evaluation Longitudinal Therapy Assessment Study Group. Prediction of recurrence in recurrent depression and the influence of consecutive episodes on vulnerability for depression: a 2-year prospective study. J Clin Psychiatry 2006 May;67(5):747-755. [Medline]
    37. Dilling H, Mombour W. In: Schmidt MH, editor. Internationale Klassifikation psychischer Störungen, ICD-10. Deutsche Ausgabe. [International classification of mental diseases, ICD-10. German edition]. Bern: Huber; 1991.
    38. Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods 1997;2(1):64-78. [CrossRef]
    39. Barlow DH. Clinical Handbook of Psychological Disorders: A Step-by-Step Treatment Manual, 4th edition. New York: Guilford Press; 2008.
    40. Kobelt A, Grosch E, Lamprecht F, Hentschel HJ. In: Schmid-Ott G, editor. Ambulante psychosomatische Nachsorge: Integratives Trainingsprogramm nach stationärer Rehabilitation. [Outpatient psychosomatic maintenance treatment: integrative training program after inpatient rehabilitation]. Stuttgart: Schattauer; 2002.
    41. Gollwitzer PM, Sheeran P. Implementation intentions and goal achievement. A meta-analysis of effects and processes. Adv Exp Soc Psychol 2006;38:69-119. [CrossRef]
    42. Wittchen HU, Zaudig M, Fydrich T. SKID-I/II Strukturiertes Klinisches Interview für DSM-IV. Deutsche Version. [SKID-I/II Structured Clinical Interview for DSM-IV. German version]. Weinheim: Beltz; 1997.
    43. Rabung S, Harfst T, Kawski S, Koch U, Wittchen HU, Schulz H. [Psychometric analysis of a short form of the "Hamburg Modules for the Assessment of Psychosocial Health" (HEALTH-49)]. Z Psychosom Med Psychother 2009;55(2):162-179. [Medline]
    44. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline]
    45. Schulte D. Messung der Therapieerwartung und Therapieevaluationvon Patienten (PATHEV). Zeitschrift für Klinische Psychologie und Psychotherapie 2005 Jul 1;34(3):176-187. [CrossRef]
    46. Little RJA. Modeling the Drop-Out Mechanism in Repeated-Measures Studies. Journal of the American Statistical Association 1995 Sep;90(431):1112. [CrossRef]
    47. Preacher KJ, Curran PJ, Bauer DJ. Computational Tools for Probing Interactions in Multiple Linear Regression, Multilevel Modeling, and Latent Curve Analysis. Journal of Educational and Behavioral Statistics 2006 Jan 01;31(4):437-448. [CrossRef]
    48. Cohen J, Cohen P, West SG, Aiken LS. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Mahwah, NJ: L Erlbaum Associates; 2003.
    49. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: L Erlbaum Associates; 1988:567.
    50. Warmerdam EH. Online treatment of adults with depression: Clinical effects, economic evaluation, working mechanisms and predictors. 2010.   URL: http://dare.ubvu.vu.nl//handle/1871/16313 [accessed 2012-05-08] [WebCite Cache]
    51. Griffiths KM, Farrer L, Christensen H. The efficacy of Internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust 2010 Jun 7;192(11 Suppl):S4-11. [Medline]
    52. American Psychiatric Association (APA). Practice Guideline for the Treatment of Patients with Major Depressive Disorder, 3rd ed. Arlington, VA: American Psychiatric Association (APA); 2010:1-128.
    53. National Collaborating Centre for Mental Health. Depression: The Treatment and Management of Depression in Adults. Clinical Guideline no 90. London, UK: National Institute for Health and Clinical Excellence (NICE); 2009:64.
    54. National Collaborating Centre for Mental Health, National Collaborating Centre for Primary Care. Generalised Anxiety Disorder and Panic Disorder (with or Without Agoraphobia) in Adults. Clinical Guideline no 113. London, UK: National Institute for Health and Clinical Excellence (NICE); 2011:56.
    55. Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 2000 Jun;31(2):73-86. [Medline]
    56. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]

    ‎
    CBT: cognitive behavior therapy
    FIML: full information maximum likelihood
    ITT: intention-to-treat
    RCT: randomized controlled trial
    TAU: treatment as usual
    TIMT: transdiagnostic Internet-based maintenance treatment

    Edited by G Eysenbach; submitted 23.12.12; peer-reviewed by M Wolf, Z Zhuohong; comments to author 18.01.13; revised version received 15.07.13; accepted 29.07.13; published 10.10.13

    Copyright

    ©David Daniel Ebert, Mario Gollwitzer, Heleen Riper, Pim Cuijpers, Harald Baumeister, Matthias Berking. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 29.08.2013.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.