This paper is in the following e-collection/theme issue:

Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1514) Digital Health Reviews (1289) Reviews in Digital Mental Health (243) Innovations in Mental Health Systems (390)

Published on in Vol 19, No 9 (2017): September

Blending Face-to-Face and Internet-Based Interventions for the Treatment of Mental Disorders in Adults: Systematic Review

Blending Face-to-Face and Internet-Based Interventions for the Treatment of Mental Disorders in Adults: Systematic Review

Blending Face-to-Face and Internet-Based Interventions for the Treatment of Mental Disorders in Adults: Systematic Review

Authors of this article:

Doris Erbe1 Author Orcid Image ;   Hans-Christoph Eichert1 Author Orcid Image ;   Heleen Riper2 Author Orcid Image ;   David Daniel Ebert3 Author Orcid Image
Article Authors Cited by (325) Tweetations (25) Metrics

    Original Paper

    1Department of Special Education and Rehabilitation, University of Cologne, Cologne, Germany

    2Department of Clinical Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

    3Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander University Erlangen-Nuremberg, Erlangen, Germany

    Corresponding Author:

    David Daniel Ebert, PhD

    Department of Clinical Psychology and Psychotherapy

    Friedrich-Alexander University Erlangen-Nuremberg

    Nägelsbachstr. 25a

    Erlangen, 91052

    Germany

    Phone: 49 9131 85 67566

    Fax:49 9131 85 67576

    Email: david.ebert@fau.de


    Background: Many studies have provided evidence for the effectiveness of Internet-based stand-alone interventions for mental disorders. A newer form of intervention combines the strengths of face-to-face (f2f) and Internet approaches (blended interventions).

    Objective: The aim of this review was to provide an overview of (1) the different formats of blended treatments for adults, (2) the stage of treatment in which these are applied, (3) their objective in combining face-to-face and Internet-based approaches, and (4) their effectiveness.

    Methods: Studies on blended concepts were identified through systematic searches in the MEDLINE, PsycINFO, Cochrane, and PubMed databases. Keywords included terms indicating face-to-face interventions (“inpatient,” “outpatient,” “face-to-face,” or “residential treatment”), which were combined with terms indicating Internet treatment (“internet,” “online,” or “web”) and terms indicating mental disorders (“mental health,” “depression,” “anxiety,” or “substance abuse”). We focused on three of the most common mental disorders (depression, anxiety, and substance abuse).

    Results: We identified 64 publications describing 44 studies, 27 of which were randomized controlled trials (RCTs). Results suggest that, compared with stand-alone face-to-face therapy, blended therapy may save clinician time, lead to lower dropout rates and greater abstinence rates of patients with substance abuse, or help maintain initially achieved changes within psychotherapy in the long-term effects of inpatient therapy. However, there is a lack of comparative outcome studies investigating the superiority of the outcomes of blended treatments in comparison with classic face-to-face or Internet-based treatments, as well as of studies identifying the optimal ratio of face-to-face and Internet sessions.

    Conclusions: Several studies have shown that, for common mental health disorders, blended interventions are feasible and can be more effective compared with no treatment controls. However, more RCTs on effectiveness and cost-effectiveness of blended treatments, especially compared with nonblended treatments are necessary.

    J Med Internet Res 2017;19(9):e306

    doi:10.2196/jmir.6588

    Keywords

    mental health; Internet; psychotherapy; blended treatment 


    Background

    Empirical evidence suggests that Internet-based psychological interventions can be used to effectively treat adults, adolescents, and children for various mental disorders such as depression, anxiety, or problematic substance use [1-6]. Such interventions have several advantages over conventional face-to-face (f2f) interventions. For example, Internet-based interventions can be administered over long distances, may save therapists’ time, allow both patients and clinicians to work at their own pace, save traveling time, and reduce the stigma of having a mental disorder or going to a psychologist or therapist [7,8].

    On the other hand, Internet-based interventions may also have disadvantages when compared with face-to-face therapies. For instance, Internet interventions may require certain abilities such as computer and Internet skills, reading and writing skills, and, in comparison with traditional therapy settings, more self-reflection and eloquence when talking about one’s thoughts and feelings. Furthermore, it has been argued that this type of intervention may make it difficult for therapists to adequately react to crisis situations such as suicidality because nonverbal cues are missing as additional information when assessing whether dissociation of suicidal thoughts is possible [9]. Also, negative effects such as frustration due to failure or time pressure might be associated with the Internet treatment format [10-13]. These disadvantages often lead to stand-alone Internet treatments being regarded as low-threshold interventions for milder cases of mental disorders, whereas face-to-face therapy and pharmacotherapy are often regarded as options of choice for more severe symptoms [14]. In addition to guided or unguided Internet treatment as a stand-alone intervention, a newer treatment approach combines face-to-face sessions with Internet-based sessions into one integrated treatment. This approach is usually called blended treatment and aims at retaining the positive aspects associated with both forms of therapy while mitigating the disadvantages [15-17]. There are different potential advantages of blending Internet and face-to-face treatments. For example, viewed from a cost-effectiveness perspective, blended treatments could possibly diminish the number of face-to-face contacts and thereby decrease the overall costs of treatment. Blended treatments could also increase the effectiveness by, for example, increasing frequency from one to two sessions per week [18] through adding Internet sessions to face-to-face interventions, thereby increasing intensity and success without additional costs. Furthermore, adding Internet interventions might improve transfer to everyday life as Internet or mobile elements could be used to support behavior change during face-to-face sessions and thereby increase effectiveness of face-to-face psychotherapy. Blended format could potentially also reach individuals for whom either purely delivered Internet-based or pure face-to-face approaches are not a suitable treatment option and thereby increase the utilization of effective treatments.

    As both face-to-face and Internet-based psychotherapy have advantages and disadvantages, combining the two approaches in a blended treatment might combine the best of two worlds.

    Definition of Blended Interventions

    As a clear definition of blended interventions is still missing [17], we define blended interventions in this study as treatment programs that use elements of both face-to-face and Internet-based interventions, including both the integrated and the sequential use of both treatment formats. Nonblended interventions comprise face-to-face treatments only or stand-alone Internet treatments. Within blended treatments, face-to-face contacts may be added to Internet interventions or, vice versa, the Internet-based part may be arranged as an adjunct to existing face-to-face programs. Internet interventions may also be arranged as an aftercare or maintenance element after acute phase treatment or as an early step within a stepped care program. Stepped care refers to a treatment program where interventions start with the least intensive and least costly treatment that is likely to work, progressing step-by-step with more intensive interventions for those patients insufficiently helped by the first or previous intervention. Blended stepped care treatments involve Internet-based treatments as one step within the sequence. See Textbox 1 for a subject index of types of blended interventions.

    Subject index

    Nonblended interventions: Face-to-face (f2f) treatments or stand-alone Internet-treatments only

    Blended interventions: Treatment programs that use elements of both face-to-face and Internet-based interventions, including sequential use of both forms of treatment

    • Integrated blended interventions: Blended treatments where the Internet-based intervention part is arranged as an adjunct to face-to-face programs or vice versa so that face-to-face and Internet-based elements are provided within the same period. In integrated blended interventions, the focus can be either on the face-to-face treatment or on the Internet-based intervention.
    • Sequential blended interventions: Blended treatments where the Internet-based intervention part is arranged before or after the face-to-face treatment such as within stepped care approaches or aftercare interventions that directly follow the face-to-face intervention.
    Textbox 1. Subject index

    Objective

    This systematic review supplies an overview of research into blended interventions for mental health. Specifically, we focus on the following questions: (1) Which blended intervention concepts have been proposed in the researched literature in the treatment of common mental disorders (anxiety disorders, depression, and substance abuse)? (2) In which stage of treatment (such as first step, acute phase treatment, and maintenance phase) do the Internet interventions take place? (3) Which types of problem and target group do the blended interventions focus on? (4) What is the objective in combining face-to-face and Internet-based approaches? and (5) What evidence is there for the effectiveness of blended interventions?


    Search Strategy

    Studies of potential relevance were identified using a systematic search in the MEDLINE, PsycINFO, Cochrane, and PubMed databases. All studies up to December 2015 were included. Searches were performed using keywords indicating face-to-face interventions (“inpatient,” “outpatient,” “face to face,” or “residential treatment”), which were combined with terms indicating Internet treatment (“internet,” “online,” or “web”) and terms indicating mental disorders (“mental health,” “depression,” “anxiety,” or “substance abuse”). We focused on three of the most common mental disorders: depression, anxiety, and substance abuse. The bibliographies of the identified studies revealed additional sources.

    Study Selection

    Studies were included if they met the following inclusion criteria: (1) the study was on an intervention that was based on both an Internet and a face-to-face treatment element that was either integrated or delivered sequentially, (2) the study involved treatment for adults with depression, anxiety, or substance abuse, and (3) the study was published in English or German. Studies with mere self-help interventions were excluded.

    After the initial database search and removal of duplicates, the title and abstract of the remaining studies were rated for the inclusion criteria independently by the first and second author. Interrater reliability was good (kappa, κ=.825, P<.001). The first and second authors then used the inclusion criteria to independently review all full-text publications judged as relevant in the title and abstract screening. Consensus through discussion was sought in cases of disagreement. If no agreement could be reached through discussion, the last author made a decisive judgment. This was the case for 2 studies.

    In November 2016, we started a second search in the database of PubMed, searching for studies citing the studies we had found in the initial search. The extraction strategies as of the initial search were used for those studies, including the independent rating of studies by the first and second authors in the first and second steps.

    Quality Assessment

    New quality assessment criteria were created since there are no current guidelines for assessing the quality of blended intervention studies. The quality of each study was rated on five aspects: study design, randomization of study conditions, report of statistics, sample size (studies powered to detect effect sizes of a minimal important difference between blended and nonblended interventions of a priori defined Cohen d of 0.35 were classified as high quality), and existence of a nonblended active control group. Each aspect was rated on a 3-point scale (0-2), providing a score with a range from 0 to10 (see Table 1 for detailed descriptions of quality assessment). Studies obtaining a total score above two-thirds of the maximum score (ie, >7) were considered high quality studies.

    Table 1. Study assessment criteria for scoring.
    AspectScoring
    Study designWhich design did the study have? 2=controlled trial, 1=pre-post, 0=case study or unclear.
    RandomizationWere participants randomized to conditions (depending on design)? 2=yes, 0=no.
    Report of statisticsAre relevant statistics reported? 2=mean and standard deviation for outcome measures, effect sizes, and P values for significant differences are reported, 1=is lacking any of these, 0=lack several of these.
    Sample sizeWas the sample size adequate to detect effect sizes of d=0.35? 2=264 or more, 1=more than 132, 0=132 or less
    Nonblended control groupDid the design involve a nonblended active control group? 2=nonblended control group with same number of sessions, 1=active control group without assured same number of sessions, 0=no active control group

    Study selection

    See Figure 1 for a flowchart of included and excluded studies. In total, we identified 64 publications describing 44 studies. Eight of the included studies were study protocols that had not yet published results in November 2016.

    Of the included studies, 27 were randomized controlled trials (RCTs), 4 were non-RCTs, 5 were pre-post studies without a control group, 4 were case studies, 3 were preliminary evaluation or acceptability studies, and 1 was a qualitative study. See Multimedia Appendix 1 for detailed study characteristics.

    Of the 44 studies, 8 were rated high quality studies (18%). A control group was involved in 31 (70%, 18/44) of the studies. Twelve studies (27%, 12/44) involved a nonblended active control group with the same number of sessions as the intervention group. Only 6 studies (14%, 6/44) involved a sample of 264 or more. All relevant statistics, including effect sizes, were reported by 17 studies (39%, 17/44). See Multimedia Appendix 3 for all quality assessment scores.

    Most studies were conducted or planned in the United States (n=12), followed by the Netherlands (n=9) and Germany (n=8). Six studies were conducted in Australia, four in Norway, three in the United Kingdom, and two in Sweden.

    The Internet part of the studies’ blended interventions generally used Web-based programs with modules combining techniques such as cognitive, behavioral, and/or emotion-focused interventions, some of them with email support. However, there were two exceptions. One study [23] reports on using emails and Internet sites on psychoeducation and other issues to improve organization and to assist with common problems related to depression. The other study [44] reports offering weekly Internet group chats with a therapist (in open groups of 8-10 participants) to focus on problems arising in readjusting to everyday life after inpatient treatment.

    ‎
    Figure 1. Flowchart of included and excluded studies.
    View this figure

    Disorders Addressed

    Of the included 44 studies, 20 studies focused on treating depression only, eight focused on anxiety disorders only, and eight focused on substance abuse only. One study described the treatment of comorbid depression and substance abuse; three studies treated both depression and anxiety. The remaining four studies had a transdiagnostic concept involving depression, anxiety, and other mental disorders. See Multimedia Appendix 1 for details on disorders addressed.

    Concept of Blended Care and Stage of Treatment

    Following the study selection process, we clustered the studies into the following types of blended care (Multimedia Appendix 1):

    1. Integrated blended interventions with face-to-face focus: These blended interventions are based on an face-to-face intervention that is complemented or partly replaced by an Internet intervention; face-to-face and Internet-based elements are provided within the same period.
    2. Integrated blended interventions with Internet focus: These blended interventions are based on Internet interventions that are partly replaced or complemented by face-to-face sessions; face-to-face and Internet-based elements are provided within the same period.
    3. Sequential blended interventions with Internet, then face-to-face: These blended interventions arrange the Internet intervention part before the face-to-face treatment, such as within stepped care.
    4. Sequential blended interventions with face-to-face, then Internet: These blended interventions arrange the Internet intervention part after the face-to-face treatment as in an aftercare program.

    The majority of studies (n=29) used a concept of integrated blended intervention, with face-to-face and Internet-based elements being provided within the same period. Among those, 18 studies focused on the face-to-face intervention, considering the Internet intervention as a replacement of some of the face-to-face sessions or as an adjunct, whereas 11 studies focused on the Internet intervention as the basis of treatment where the face-to-face sessions served as an adjunct, for example, for increasing adherence to the Internet-based modules.

    The remaining 15 studies presented sequential blended interventions, arranging the Internet intervention part before or following the face-to-face treatment. Nine of them placed the Internet intervention before the face-to-face treatment either as part of a stepped or matched care program (n=4) or for bridging waiting time for referrals on waiting lists for face-to-face psychotherapy (n=5). Six studies placed the Internet intervention after the face-to-face treatment as an aftercare concept.

    Aims

    The studies’ aims for choosing to use a blended intervention can be classified by the concepts that were used (see Multimedia Appendix 1 for details). As some studies mentioned various aims, we clustered the interventions by their main objective. See Figure 2 for a summary of aims of blended therapy.

    Integrated Blended Interventions With Face-to-Face (F2F) Focus

    Among the 18 integrated blended interventions with face-to-face focus, seven aimed at delegating some elements of face-to-face therapy to Internet-based cognitive behavioral therapy (iCBT) and thereby, saving clinician time and reducing overall costs [15,51,54,58,62,69,75]. Nine of them aimed at supporting face-to-face therapy by delivering additional Internet elements and thereby increasing effectiveness of face-to-face psychotherapy [23-25,37,40,55,60,61,77,78]. Two studies stated as their aim integrating Internet elements with face-to-face psychotherapy to establish a proactive and long-term approach to the management of chronic conditions, thus providing long-term support for patients with chronic or recurrent mental diseases beyond the acute phase of face-to-face treatment [59,68].

    Integrated Blended Interventions With Internet Focus

    Five of the 11 integrated blended interventions with an Internet focus aimed at improving the delivery of evidence-based treatment in primary care [31,48,50,65,72], for instance, by assisting GPs in providing evidence-based mental health care programs [65]. Three of the studies aimed at integrating face-to-face sessions to maximize the effectiveness of iCBT [19,64,70]. Other aims of these studies were to offer an Internet intervention to all participants with complementary face-to-face sessions as needed by the individual participant and as such to increase flexibility to meet the needs of different clients concerning face-to-face support [79], or, through face-to-face support, to motivate participants to persist with iCBT [45,74].

    Sequential Blended Interventions With Internet, Then F2F

    The nine sequential blended interventions that started with the Internet intervention either aimed at bridging waiting time (n=5) with iCBT until face-to-face therapy started [53,57,67,73,80] or worked with a blended stepped care concept, aiming at delivering low-threshold iCBT as an early step and thereby reducing costs for subsequent face-to-facepsychotherapy (n=4) (by reducing either the number of v sessions or the number of patients treated in the subsequent face-to-facephase) [20,26,46,47].

    ‎
    Figure 2. Aims of blended therapy.
    View this figure
    Sequential Blended Interventions With F2F, Then Internet

    Five of the six sequential blended interventions that started with the face-to-face intervention were designed as aftercare programs aiming at maintaining therapeutic benefits of face-to-faceresidential or inpatient psychotherapy through subsequent iCBT [22,41,44,56,76]. Another study aimed to monitor patient progress beyond the acute phase of face-to-face treatment for a long time and to maintain the therapeutic relationship in the absence of face-to-face contacts in a remote setting [71].

    Outcome

    See Multimedia Appendix 2 for details. Given the variety of different study types (eg, study protocols, case studies, and qualitative studies) and that outcome measures, study designs, and aims differ substantially, it is not currently possible to summarize the effects using meta-analytic techniques.

    Of the 44 identified studies, eight were study protocols and 36 had been completed. Among the 36 completed studies, 23 involved a control group. Out of the 23 completed studies with a control group, four studies compared the blended intervention with no treatment, for instance, a waiting-list group [19,41,44,48]. Nine studies compared the blended intervention group with a nonblended face-to-face intervention with an equal amount or more sessions [24,32,50,53-55,57,62,75], six studies compared the blended intervention with treatment-as-usual without controlling the number of sessions [20,26,32,37,40], and two studies compared the blended intervention with a nonblended Internet intervention [64,65]. The remaining two studies involved both Internet-based and face-to-face nonblended control groups [55,70].

    Among the eight study protocols, two studies compared the blended intervention with TAU without controlling for the number of sessions [46,51,59], whereas four studies compared the blended intervention-group with a nonblended face-to-face intervention with an equal number or more sessions [15,53,60,69]. One study compared the blended intervention (in that case, aftercare after inpatient treatment) with a placebo (inpatient treatment plus access after discharge to an Internet platform with information on depression [76]), and one study compared it with no treatment [67].

    Although cost-effectiveness or cost-savings was in some way considered by almost all 44 studies, only three of the completed studies elaborated on it and evaluated potential cost-effectiveness or cost-savings [33,52,81]. However, three additional study protocols claimed to be planning to evaluate cost-effectiveness [15,46,51,69]. See Figure 3 for a summary of aims of blended therapy in completed studies.

    Integrated Blended Interventions With F2F Focus

    Of the six studies that were delegating some elements of face-to-face therapy to iCBT, three studies were able to show that, by doing that, 50% to 86 % of clinician time could be saved without reducing the therapeutic outcome of depression and anxiety treatment [54,62,75]. The other three were study protocols [15,51,69].

    Six of the nine studies aiming at supporting face-to-face therapy by delivering Internet elements were able to show that adding Internet elements can lead to lower dropout rates and/or greater abstinence rates of patients with substance abuse compared with stand-alone face-to-face interventions [24,40,55,77]. One study had a pre-post design [61] and showed that its blended concept led to a reduction in symptomatology that was maintained for 12 months. One was a case study where a patient diagnosed with depression as well as her therapist were able to make creative use of Web-based resources for purposes such as psychoeducation or job search [23]. One was a study protocol [60].

    ‎
    Figure 3. Aims of blended therapy of studies with published outcome.
    View this figure

    One study aiming at establishing a proactive and long-term approach to the management of chronic mental diseases beyond the acute phase of face-to-face treatment [68] showed large and significant pre-post effect sizes. One study with a similar aim [59] showed that an Internet-delivered adjunct to TAU may reduce the number of unwell weeks in patients with recurrent depression compared with TAU and thus, reduce the lifelong burden of depression.

    Integrated Blended Interventions With Internet Focus

    Regarding the aim of integrating face-to-face sessions into Internet interventions to maximize effectiveness of iCBT, Sethi et al [70] found the blended intervention to be superior in reducing symptoms of depression, anxiety, and/or automatic negative thoughts in comparison with three control groups: nonblended Internet intervention, nonblended face-to-faceintervention, and no intervention. One study treating depression [64] did not find a difference between the blended and the nonblended intervention, whereas a third study did not have a nonblended control group but found the blended intervention to be superior compared with a waiting-list control group [19].

    Using blended interventions to improve the delivery of evidence-based treatment in primary care has been successful in acceptability and in reducing symptomatology [31,48,50,65,72]. However, two controlled studies [50,65] could not show a superiority of the blended intervention over an active nonblended control group.

    In a study aiming at increasing the flexibility of an Internet intervention by offering complementary face-to-face sessions as needed by the individual participant [79], the average number of face-to-face sessions participants needed was equal to or less than 3.7, with the program still substantially and significantly reducing depression symptomatology. This was a pilot study without a control group, yet the authors cite a study that treated participants with similar pre- and post-Beck depression inventory scores as their study [82]. The other study’s participants needed an average of 11.6 face-to-face sessions for a similar symptom reduction, and the authors therefore claim that their treatment needs about eight fewer individual sessions per client.

    A study aiming at motivating participants to persist with iCBT through face-to-face support [74] used a qualitative design. It found that for persistence in a blended treatment, acknowledgment may be related to flexibility and feedback from a qualified therapist in the face-to-face consultations beneath personal resources such as a sense of belonging toward partners, family, and friends. A case study with a similar aim [45] showed good adherence and symptom remission in a woman suffering from antepartum depression, who was thus able to avoid antidepressant use during pregnancy.

    Sequential Blended Interventions With Internet, Then F2F

    Of the five studies aiming at bridging waiting time with iCBT until face-to-face therapy starts, two studies [53,57] showed significant between-group effect sizes between the iCBT waiting-list group and the control waiting-list group. One study failed to show a superiority of the iCBT waiting-list group compared with a control condition with participants who remained on the wait-list with a self-help booklet [83]. One of the studies showed in a pre-post design that the Internet intervention could substantially and significantly reduce symptomatology [73]. The remaining study with this aim [67] is a study protocol.

    Two of the completed studies working with a blended stepped care concept did not find a significant superiority of the blended intervention compared with face-to-face [47] or TAU [20]. Haug et al [47] even found a significantly better outcome of the control group (nonblended f2f) in one of the outcome measures. Neither Haug et al nor Braamse et al [20] reported results on cost-saving or cost-effectiveness; however, one study protocol with a stepped care concept mentioned planning to evaluate cost-effectiveness [46].

    Sequential Blended Interventions With F2F, Then Internet

    Two of the five studies with Internet intervention designed as maintenance treatment found a substantially and significantly lower relapse rate compared with access to TAU groups [41,44]. One further study was a case study reporting how iCBT aftercare substantially reduced depression, leading to remission after discharge from inpatient treatment [22]. One study with an empirical correlational design and without a control group showed a significant relationship between the number of iCBT modules accessed and substance abuse reduction in the year following inpatient treatment when controlling for motivation, self-efficacy, and pretreatment substance abuse [56]. One study was a study protocol [76].

    One case study aiming at long-term monitoring of patient progress beyond the acute phase of face-to-face treatment, as well as maintaining the therapeutic relationship in the absence of face-to-face contacts in a remote setting [71], reported continuing therapeutic contacts and a reduction from very severe depression symptoms to remission within 46 weeks.


    Principal Findings

    This study has shown that, in the past few years, a growing number of blended interventions that combine Internet and face-to-face interventions have been developed for common mental health problems. The interventions we found have different concepts and various aims. First results are encouraging and suggest that, compared with stand-alone face-to-face interventions, blended therapy may save clinician time without reducing therapy outcome, can lead to lower dropout rates and/or greater abstinence rates of patients with substance abuse, may help maintain effects of inpatient therapy, and may even increase the effects of psychotherapy, although results are mixed and more research is clearly needed.

    Compared with the field of both Internet-based stand-alone treatment and face-to-face interventions, the field of blended interventions is, however, under development and still small. Most aims of the interventions stated in the studies have not been evaluated rigorously. For instance, only 19 out of 36 completed studies were RCTs, and only eight of them ensured comparability by involving a nonblended intervention control group with the same number of sessions [27,50,57,62,70,75,78,83]. Only eight studies were considered high quality studies, and only six studies involved a control group big enough to detect effect sizes of d=0.35 or less. Also, the cost-effectiveness of blended treatments compared with face-to-face psychotherapy has only been evaluated in three out of the 36 completed studies. However, this issue is focused upon in current large research projects [46,84].

    Several questions remain. For instance, not much is known about the optimal ratio of Internet and face-to-facesessions that would allow costs to be minimized while maintaining or increasing effectiveness. Only one study in our review is moving toward answering this question: In the study by Jacmon et al [79], clients received face-to-face sessions as wished as an adjunct to iCBT. Also, clients were actively invited by therapists to undertake face-to-face contacts if a depression measure indicated that depression continued at clinical levels. Jacmon et al reported that this intervention produced effects at least as large as a completely face-to-face treatment, with an average of about eight fewer individual sessions per client in comparison with a similar nonblended face-to-face treatment in another study. However, the findings of this pilot study cannot be generalized because of its small sample size and lack of a control group. Another open question is which elements of face-to-face therapy can most suitably be delegated to the Internet. Possibly, elements that do not need an intensive dialogue between client and therapist, such as psychoeducation, may most easily be delegated to the Internet [16]. Also, it remains unclear who would benefit most from which relationship between face-to-face and Internet modules. For instance, patients who have difficulty in expressing all of their thoughts and feelings in writing (eg, because of lack of introspection or because they are not conscious of their own role in maintaining their disorder) might need more elements of face-to-face intervention psychotherapy. This might also be the case for patients with severe, chronic, or personality disorders. A helpful tool to set up a personalized blended treatment taking the patients’ characteristics into account might be the recently developed “Fit for Blended Care” instrument [17].

    Moreover, although a number of studies stated that increasing the effectiveness of face-to-face psychotherapy is an aim of the blending of treatments with Internet options, evidence for this hypothesis is limited to four studies with an integrated concept [24,25,32,70]. Such an assumption is, however, supported by a recent meta-analysis by Lindhiem and colleagues [85]. On the basis of 10 RCTs, they found that, in comparison with strictly on-site interventions, psychological interventions were considerably more effective for a range of conditions when behavior changes between face-to-face sessions had been supported by a mobile component, such as short message service (SMS; d=0.27). There is also some evidence that telephone-based interventions may help maintain advances achieved during outpatient treatment or increase effectiveness of face-to-face interventions [86,87]. Nevertheless, more research is clearly needed to conclude whether the effectiveness of psychotherapy can potentially be increased by using blended treatments. Besides the identified advantages of blended interventions, for example, saving clinician time, improving success rates in the reduction of symptomatology, and helping to prevent relapse after face-to-face therapy, these interventions might possibly have limitations compared with face-to-face interventions. For instance, in the study by Marks et al [62], dropout was substantially higher in the mainly Internet-based group compared with the face-to-face control group (43% if mainly computer-guided and 24% if entirely clinician-guided). On the other hand, other studies have reported similar [50,75] or even lower [24] dropout rates of a blended compared with a nonblended intervention. Yet, the questions of blended interventions’ negative effects and potential losses compared with nonblended interventions remain to be answered, just as much as the one about potential negative effects of Internet-based treatments [12].

    An interesting question we encountered is what blended treatments exactly entail. The identified studies in our review that explicitly use the term “blended” are the more recently published ones and describe integrated blended treatments, that is, combined treatments that provide face-to-face and Internet-based elements within the same period. Although there is currently no clear definition, this description might be an implicit understanding of blended interventions that is more limiting than our definition. We consider that blended interventions should also include the combination of the Internet-based interventions arranged before or after the face-to-face treatments following clear rules and procedures, such as within stepped care or aftercare that directly follows the acute phase treatment. In light of the numerous studies describing sequential blended interventions, we decided to use this wider definition.

    Limitations

    This study has some limitations. As within every systematic review, the risk of selection bias when including relevant studies needs to be considered. However, our use of independent ratings by two of the authors worked against this bias. In addition, publication bias needs to be considered. We did not contact authors for additional data or additional studies, which would have automatically limited the number of studies that could be included. Also, we only reviewed bibliographies from included studies, so we possibly missed studies that were cited in papers that we reviewed but did not include. Furthermore, the types of studies we included were heterogeneous (for instance, we included study protocols). It is possible that more narrow inclusion criteria (eg, randomized controlled studies with nonblended active control groups only) would have produced more information about the effectiveness of blended interventions. However, the number of such studies in the field is yet very small, and future studies are needed to explore whether blended treatments can, for example, be superior compared with nonblended treatments with regard to effect sizes or lower costs.

    For further research, it would be of interest to explore effectiveness and cost-effectiveness of blended concepts, especially concerning the optimal balance of face-to-face and Internet interventions. Information on this aspect would help determine which therapy with which theoretical foundation (such as cognitive behavioral therapy or psychoanalysis) is feasible for blended interventions, as well as where the optimal balance of therapy modules lies for individual patients in light of the type and severity of disorder, state of motivation, ability of introspection, and other variables such as age, gender, and computer skills.

    Conclusions

    To conclude, we have found that several studies have shown that blended interventions are feasible and effective compared with no treatment controls. There are many different kinds of blended concepts that, in every phase of treatment, may offer added value concerning either effectiveness or cost-effectiveness. However, to evaluate the actual benefit of blended concepts for mental health care, more RCTs on effectiveness and cost-effectiveness compared with traditional nonblended psychotherapy are required. Thus, more research is needed, especially concerning disorders for which blended interventions are particularly effective, the amount of face-to-face contact that is needed, and the parts of therapy that can be delegated to the Internet.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Previous psychological research combining Internet and face-to-face psychotherapy.

    PDF File (Adobe PDF File), 42KB
    ‎

    Multimedia Appendix 2

    Outcome of previous psychological research combining Internet and face-to-face (f2f) psychotherapy.

    PDF File (Adobe PDF File), 87KB
    ‎

    Multimedia Appendix 3

    Study quality assessment of blended treatments.

    PDF File (Adobe PDF File), 42KB
    1. Andersson G, Topooco N, Havik O, Nordgreen T. Internet-supported versus face-to-face cognitive behavior therapy for depression. Expert Rev Neurother 2016;16(1):55-60. [CrossRef] [Medline]
    2. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One 2015;10(3):e0119895 [FREE Full text] [CrossRef] [Medline]
    3. Mayo-Wilson E, Montgomery P. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst Rev 2013(9):1-48. [CrossRef] [Medline]
    4. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
    5. Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, et al. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One 2014;9(6):e99912 [FREE Full text] [CrossRef] [Medline]
    6. Königbauer J, Letsch J, Ebler P, Ebert DD, Baumeister H. Internet- and mobile-based depression interventions for people with diagnosed depression: a systematic review and meta-analysis. J Affect Disord 2017 Jul 10;223:28-40. [CrossRef] [Medline]
    7. Cuijpers P, van SA, Andersson G. Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med 2008 Apr;31(2):169-177 [FREE Full text] [CrossRef] [Medline]
    8. Ebert DD, Berking M, Thiart H, Riper H, Laferton JA, Cuijpers P, et al. Restoring depleted resources: Efficacy and mechanisms of change of an internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol 2015 Dec;34S:1240-1251. [CrossRef] [Medline]
    9. Berger T, Andersson G. [Internet-based psychotherapies: characteristics and empirical evidence]. Psychother Psychosom Med Psychol 2009;59(3-4):159-66; quiz 167. [CrossRef] [Medline]
    10. Boettcher J, Rozental A, Andersson G, Carlbring P. Side effects in internet-based interventions for social anxiety disorder. Internet Interv 2014 Mar;1(1):3-11. [CrossRef]
    11. Rozental A, Boettcher J, Andersson G, Schmidt B, Carlbring P. Negative effects of internet interventions: a qualitative content analysis of patients' experiences with treatments delivered online. Cogn Behav Ther 2015;44(3):223-236. [CrossRef] [Medline]
    12. Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, et al. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interv 2014 Mar;1(1):12-19. [CrossRef]
    13. Ebert DD, Donkin L, Andersson G, Andrews G, Berger T, Carlbring P, et al. Does internet-based guided-self-help for depression cause harm? an individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials. Psychol Med 2016 Oct;46(13):2679-2693. [CrossRef] [Medline]
    14. Moritz S, Schröder J, Meyer B, Hauschildt M. The more it is needed, the less it is wanted: attitudes toward face-to-face intervention among depressed patients undergoing online treatment. Depress Anxiety 2013 Feb;30(2):157-167. [CrossRef] [Medline]
    15. Kooistra LC, Wiersma JE, Ruwaard J, van Oppen P, Smit F, Lokkerbol J, et al. Blended vs. face-to-face cognitive behavioural treatment for major depression in specialized mental health care: study protocol of a randomized controlled cost-effectiveness trial. BMC Psychiatry 2014 Oct 18;14:290 [FREE Full text] [CrossRef] [Medline]
    16. van der Vaart R, Witting M, Riper H, Kooistra L, Bohlmeijer ST, van Gemert-Pijnen LJ. Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study. BMC Psychiatry 2014 Dec 14;14:355 [FREE Full text] [CrossRef] [Medline]
    17. Wentzel J, van der Vaart R, Bohlmeijer ET, van der Vaart R. Mixing online and face-to-face therapy: how to benefit from blended care in mental health care. JMIR Ment Health 2016 Feb 09;3(1):e9 [FREE Full text] [CrossRef] [Medline]
    18. Cuijpers P, Huibers M, Ebert DD, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? a metaregression analysis. J Affect Disord 2013 Jul;149(1-3):1-13. [CrossRef] [Medline]
    19. Andersson G, Carlbring P, Holmström A, Sparthan E, Furmark T, Nilsson-Ihrfelt E, et al. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol 2006 Aug;74(4):677-686. [CrossRef] [Medline]
    20. Braamse AM, van MB, Visser OJ, Boenink AD, Cuijpers P, Eeltink CE, et al. A randomized clinical trial on the effectiveness of an intervention to treat psychological distress and improve quality of life after autologous stem cell transplantation. Ann Hematol 2016 Jan;95(1):105-114 [FREE Full text] [CrossRef] [Medline]
    21. Braamse AM, van Meijel B, Visser OJ, Boenink AD, Cuijpers P, Eeltink CE, et al. A randomized clinical trial on the effectiveness of an intervention to treat psychological distress and improve quality of life after autologous stem cell transplantation. Ann Hematol 2016 Jan;95(1):105-114 [FREE Full text] [CrossRef] [Medline]
    22. Brakemeier E, Marchner J, Gutgsell S, Engel V, Radtke M, Tuschen-Caffier B, et al. CBASP@home: ein internetbasiertes situationsanalysen- training zur stabilisierung des therapieerfolgs nach stationärer therapie für chronisch depressive patienten. Verhaltenstherapie 2013 Aug;23:190-203. [CrossRef]
    23. Callan JA, Howland RH, Puskar K. Using computers and the Internet for psychiatric nursing intervention. J Psychosoc Nurs Ment Health Serv 2009 Jan;47(1):13-14. [Medline]
    24. Campbell AN, Nunes EV, Matthews AG, Stitzer M, Miele GM, Polsky D, et al. Internet-delivered treatment for substance abuse: a multisite randomized controlled trial. Am J Psychiatry 2014 Jun;171(6):683-690 [FREE Full text] [CrossRef] [Medline]
    25. Cochran G, Stitzer M, Campbell AN, Hu MC, Vandrey R, Nunes EV. Web-based treatment for substance use disorders: differential effects by primary substance. Addict Behav 2015 Jun;45:191-194 [FREE Full text] [CrossRef] [Medline]
    26. van Straten A, Seekles W, van't Veer-Tazelaar NJ, Beekman AT, Cuijpers P. Stepped care for depression in primary care: what should be offered and how? Med J Aust 2010 Jun 07;192(11 Suppl):S36-S39. [Medline]
    27. Campbell AN, Nunes EV, Pavlicova M, Hatch-Maillette M, Hu MC, Bailey GL, et al. Gender-based outcomes and acceptability of a computer-assisted psychosocial intervention for substance use disorders. J Subst Abuse Treat 2015 Jun;53:9-15 [FREE Full text] [CrossRef] [Medline]
    28. Cunningham C, Stitzer M, Campbell AN, Pavlicova M, Hu MC, Nunes EV. Contingency management abstinence incentives: cost and implications for treatment tailoring. J Subst Abuse Treat 2017 Jan;72:134-139. [CrossRef] [Medline]
    29. Tofighi B, Campbell AN, Pavlicova M, Hu MC, Lee JD, Nunes EV. Recent internet use and associations with clinical outcomes among patients entering addiction treatment involved in a web-delivered psychosocial intervention study. J Urban Health 2016 Oct;93(5):871-883. [CrossRef] [Medline]
    30. Campbell AN, Nunes EV, Miele GM, Matthews A, Polsky D, Ghitza UE, et al. Design and methodological considerations of an effectiveness trial of a computer-assisted intervention: an example from the NIDA Clinical Trials Network. Contemp Clin Trials 2012 Mar;33(2):386-395 [FREE Full text] [CrossRef] [Medline]
    31. Campbell AN, Turrigiano E, Moore M, Miele GM, Rieckmann T, Hu MC, et al. Acceptability of a web-based community reinforcement approach for substance use disorders with treatment-seeking American Indians/Alaska Natives. Community Ment Health J 2015 May;51(4):393-403 [FREE Full text] [CrossRef] [Medline]
    32. Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Nuro KF, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry 2008 Jul;165(7):881-888 [FREE Full text] [CrossRef] [Medline]
    33. Olmstead TA, Ostrow CD, Carroll KM. Cost-effectiveness of computer-assisted training in cognitive-behavioral therapy as an adjunct to standard care for addiction. Drug Alcohol Depend 2010 Aug 01;110(3):200-207 [FREE Full text] [CrossRef] [Medline]
    34. Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Rounsaville BJ. Enduring effects of a computer-assisted training program for cognitive behavioral therapy: a 6-month follow-up of CBT4CBT. Drug Alcohol Depend 2009 Feb 01;100(1-2):178-181 [FREE Full text] [CrossRef] [Medline]
    35. Carroll KM, Kiluk BD, Nich C, Babuscio TA, Brewer JA, Potenza MN, et al. Cognitive function and treatment response in a randomized clinical trial of computer-based training in cognitive-behavioral therapy. Subst Use Misuse 2011;46(1):23-34 [FREE Full text] [CrossRef] [Medline]
    36. Sugarman DE, Nich C, Carroll KM. Coping strategy use following computerized cognitive-behavioral therapy for substance use disorders. Psychol Addict Behav 2010 Dec;24(4):689-695 [FREE Full text] [CrossRef] [Medline]
    37. Carroll KM, Kiluk BD, Nich C, Gordon MA, Portnoy GA, Marino DR, et al. Computer-assisted delivery of cognitive-behavioral therapy: efficacy and durability of CBT4CBT among cocaine-dependent individuals maintained on methadone. Am J Psychiatry 2014 Apr;171(4):436-444 [FREE Full text] [CrossRef] [Medline]
    38. Carroll KM, Herman A, DeVito EE, Frankforter TL, Potenza MN, Sofuoglu M. Catehol-o-methyltransferase gene Val158met polymorphism as a potential predictor of response to computer-assisted delivery of cognitive-behavioral therapy among cocaine-dependent individuals: preliminary findings from a randomized controlled trial. Am J Addict 2015 Aug;24(5):443-451 [FREE Full text] [CrossRef] [Medline]
    39. Morie KP, Nich C, Hunkele K, Potenza MN, Carroll KM. Alexithymia level and response to computer-based training in cognitive behavioral therapy among cocaine-dependent methadone maintained individuals. Drug Alcohol Depend 2015 Jul 01;152:157-163 [FREE Full text] [CrossRef] [Medline]
    40. Christensen DR, Landes RD, Jackson L, Marsch LA, Mancino MJ, Chopra MP, et al. Adding an Internet-delivered treatment to an efficacious treatment package for opioid dependence. J Consult Clin Psychol 2014 Dec;82(6):964-972 [FREE Full text] [CrossRef] [Medline]
    41. Ebert D, Tarnowski T, Gollwitzer M, Sieland B, Berking M. A transdiagnostic internet-based maintenance treatment enhances the stability of outcome after inpatient cognitive behavioral therapy: a randomized controlled trial. Psychother Psychosom 2013;82(4):246-256. [CrossRef] [Medline]
    42. Ebert DD, Gollwitzer M, Riper H, Cuijpers P, Baumeister H, Berking M. For whom does it work? moderators of outcome on the effect of a transdiagnostic internet-based maintenance treatment after inpatient psychotherapy: randomized controlled trial. J Med Internet Res 2013 Oct 10;15(10):e191 [FREE Full text] [CrossRef] [Medline]
    43. Ebert DD, Hannig W, Tarnowski T, Sieland B, Götzky B, Berking M. [Web-based rehabilitation aftercare following inpatient psychosomatic treatment]. Rehabilitation (Stuttg) 2013 Jun;52(3):164-172. [CrossRef] [Medline]
    44. Golkaramnay V, Bauer S, Haug S, Wolf M, Kordy H. The exploration of the effectiveness of group therapy through an Internet chat as aftercare: a controlled naturalistic study. Psychother Psychosom 2007;76(4):219-225. [CrossRef] [Medline]
    45. Hantsoo L, Epperson CN, Thase ME, Kim DR. Antepartum depression: treatment with computer-assisted cognitive-behavioral therapy. Am J Psychiatry 2013 Aug;170(8):929-930 [FREE Full text] [CrossRef] [Medline]
    46. Härter M, Heddaeus D, Steinmann M, Schreiber R, Brettschneider C, König HH, et al. [Collaborative and stepped care for depression: Development of a model project within the hamburg network for mental health (psychenet.de)]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2015 Apr;58(4-5):420-429. [CrossRef] [Medline]
    47. Haug T, Nordgreen T, Öst LG, Kvale G, Tangen T, Andersson G, et al. Stepped care versus face-to-face cognitive behavior therapy for panic disorder and social anxiety disorder: predictors and moderators of outcome. Behav Res Ther 2015 Aug;71:76-89. [CrossRef] [Medline]
    48. Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, et al. The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res 2013 Aug 05;15(8):e153 [FREE Full text] [CrossRef] [Medline]
    49. Høifødt RS, Mittner M, Lillevoll K, Katla SK, Kolstrup N, Eisemann M, et al. Predictors of response to web-based cognitive behavioral therapy with high-intensity face-to-face therapist guidance for depression: a bayesian analysis. J Med Internet Res 2015 Sep 02;17(9):e197 [FREE Full text] [CrossRef] [Medline]
    50. Kay-Lambkin F, Baker A, Lewin T, Carr V. Acceptability of a clinician-assisted computerized psychological intervention for comorbid mental health and substance use problems: treatment adherence data from a randomized controlled trial. J Med Internet Res 2011 Jan 27;13(1):e11 [FREE Full text] [CrossRef] [Medline]
    51. Kemmeren LL, van Schaik DJF, Riper H, Kleiboer AM, Bosmans JE, Smit JH. Effectiveness of blended depression treatment for adults in specialised mental healthcare: study protocol for a randomised controlled trial. BMC Psychiatry 2016 Apr 21;16:113 [FREE Full text] [CrossRef] [Medline]
    52. Kolovos S, Kenter RM, Bosmans JE, Beekman AT, Cuijpers P, Kok RN, et al. Economic evaluation of internet-based problem-solving guided self-help treatment in comparison with enhanced usual care for depressed outpatients waiting for face-to-face treatment: A randomized controlled trial. J Affect Disord 2016 Aug;200:284-292. [CrossRef] [Medline]
    53. Kok RN, van Straten A, Beekman AT, Cuijpers P. Short-term effectiveness of web-based guided self-help for phobic outpatients: randomized controlled trial. J Med Internet Res 2014 Sep 29;16(9):e226 [FREE Full text] [CrossRef] [Medline]
    54. Kenwright M, Liness S, Marks I. Reducing demands on clinicians by offering computer-aided self-help for phobia/panic. Feasibility study. Br J Psychiatry 2001 Nov;179:456-459 [FREE Full text] [Medline]
    55. Kiluk BD, Devore KA, Buck MB, Nich C, Frankforter TL, LaPaglia DM, et al. Randomized trial of computerized cognitive behavioral therapy for alcohol use disorders: efficacy as a virtual stand-alone and treatment add-on compared with standard outpatient treatment. Alcohol Clin Exp Res 2016 Sep;40(9):1991-2000. [CrossRef] [Medline]
    56. Klein AA, Slaymaker VJ, Dugosh KL, McKay JR. Computerized continuing care support for alcohol and drug dependence: a preliminary analysis of usage and outcomes. J Subst Abuse Treat 2012 Jan;42(1):25-34. [CrossRef] [Medline]
    57. Kok RN, van SA, Beekman AT, Cuijpers P. Short-term effectiveness of web-based guided self-help for phobic outpatients: randomized controlled trial. J Med Internet Res 2014 Sep 29;16(9):e226 [FREE Full text] [CrossRef] [Medline]
    58. Kooistra LC, Ruwaard J, Wiersma J, van Oppen P, van der Vaart R, van Gemert-Pijnen JE, et al. Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care. Internet Interv 2016 May;4:61-71. [CrossRef]
    59. Kordy H, Wolf M, Aulich K, Bürgy M, Hegerl U, Hüsing J, et al. Internet-delivered disease management for recurrent depression: a multicenter randomized controlled trial. Psychother Psychosom 2016;85(2):91-98. [CrossRef] [Medline]
    60. Krieger T, Meyer B, Sude K, Urech A, Maercker A, Berger T. Evaluating an e-mental health program (“deprexis”) as adjunctive treatment tool in psychotherapy for depression: design of a pragmatic randomized controlled trial. BMC Psychiatry 2014 Oct 08;14:285 [FREE Full text] [CrossRef] [Medline]
    61. Månsson KN, Skagius RE, Gervind E, Dahlin M, Andersson G. Development and initial evaluation of an internet-based support system for face-to-face cognitive behavior therapy: a proof of concept study. J Med Internet Res 2013 Dec 10;15(12):e280 [FREE Full text] [CrossRef] [Medline]
    62. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan;34(1):9-17. [Medline]
    63. Kim SJ, Marsch LA, Acosta MC, Guarino H, Aponte-Melendez Y. Can persons with a history of multiple addiction treatment episodes benefit from technology delivered behavior therapy? A moderating role of treatment history at baseline. Addict Behav 2016 Mar;54:18-23 [FREE Full text] [CrossRef] [Medline]
    64. Nordmo M, Sinding A, Carlbring P, Andersson G, Havik O, Nordgreen T. Internet-delivered cognitive behavioural therapy with and without an initial face-to-face psychoeducation session for social anxiety disorder: A pilot randomized controlled trial. Internet Interv 2015 Nov;2(4):429-436. [CrossRef]
    65. Pier C, Austin DW, Klein B, Mitchell J, Schattner P, Ciechomski L, et al. A controlled trial of internet-based cognitive-behavioural therapy for panic disorder with face-to-face support from a general practitioner or email support from a psychologist. Ment Health Fam Med 2008 Mar;5(1):29-39 [FREE Full text] [Medline]
    66. Shandley K, Austin DW, Klein B, Pier C, Schattner P, Pierce D, et al. Therapist-assisted, internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res 2008 May 19;10(2):e14 [FREE Full text] [CrossRef] [Medline]
    67. Reins JA, Ebert DD, Lehr D, Riper H, Cuijpers P, Berking M. Internet-based treatment of major depression for patients on a waiting list for inpatient psychotherapy: protocol for a multi-centre randomised controlled trial. BMC Psychiatry 2013 Nov 26;13:318 [FREE Full text] [CrossRef] [Medline]
    68. Robertson L, Smith M, Castle D, Tannenbaum D. Using the internet to enhance the treatment of depression. Australas Psychiatry 2006 Dec;14(4):413-417. [CrossRef] [Medline]
    69. Romijn G, Riper H, Kok R, Donker T, Goorden M, Hakkaart van Roijen L, et al. Cost-effectiveness of blended vs. face-to-face cognitive behavioural therapy for severe anxiety disorders: study protocol of a randomized controlled trial. BMC Psychiatry 2015 Dec 12;15:311 [FREE Full text] [CrossRef] [Medline]
    70. Sethi S, Campbell A, Ellis L. The use of computerized self-help packages to treat adolescent depression and anxiety. J Technol Hum Serv 2010 Aug 31;28(3):144-160. [CrossRef]
    71. Tannenbaum D, Spiranovic C. Severe, recurrent depression managed in a remote setting via the internet: an example of remote care using the healthsteps system. In: Taylor CB, editor. How to Practice Evidence-based Psychiatry: Basic Principles and Case Studies. Arlington, TX: American Psychiatric Publishing, Inc; 2011:275-286.
    72. Van Voorhees BW, Ellis JM, Gollan JK, Bell CC, Stuart SS, Fogel J, et al. Development and process evaluation of a primary care internet-based intervention to prevent depression in emerging adults. Prim Care Companion J Clin Psychiatry 2007;9(5):346-355 [FREE Full text] [Medline]
    73. Whitfield G, Williams C, Hinshelwood R, Pashley A, Campsie L. The impact of a novel computerized CBT CD-Rom (overcoming depression) offered to patients referred to clinical psychology. Eur Psychiatry 2007 Mar;22:S248. [CrossRef]
    74. Wilhelmsen M, Lillevoll K, Risør MB, Høifødt R, Johansen ML, Waterloo K, et al. Motivation to persist with internet-based cognitive behavioural treatment using blended care: a qualitative study. BMC Psychiatry 2013 Nov 07;13:296 [FREE Full text] [CrossRef] [Medline]
    75. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, et al. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry 2005 Jun;162(6):1158-1164. [CrossRef] [Medline]
    76. Zwerenz R, Becker J, Knickenberg RJ, Hagen K, Dreier M, Wölfling K, et al. Enhancing inpatient psychotherapeutic treatment with online self-help: study protocol for a randomized controlled trial. Trials 2015 Mar 17;16:98 [FREE Full text] [CrossRef] [Medline]
    77. Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Nuro KF, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry 2008 Jul;165(7):881-888 [FREE Full text] [CrossRef] [Medline]
    78. Marsch LA, Guarino H, Acosta M, Aponte-Melendez Y, Cleland C, Grabinski M, et al. Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. J Subst Abuse Treat 2014 Jan;46(1):43-51 [FREE Full text] [CrossRef] [Medline]
    79. Jacmon J, Malouff J, Taylor N. Treatment of major depression: effectiveness of cognitive behavior therapy with an internet course as a central component. EJAP 2010 Jan 03;5(2):8. [CrossRef]
    80. Kenter RM, van SA, Hobbel SH, Smit F, Bosmans J, Beekman A, et al. Effectiveness and cost effectiveness of guided online treatment for patients with major depressive disorder on a waiting list for psychotherapy: study protocol of a randomized controlled trial. Trials 2013 Dec 01;14:412 [FREE Full text] [CrossRef] [Medline]
    81. Murphy SM, Campbell AN, Ghitza UE, Kyle TL, Bailey GL, Nunes EC, et al. Cost-effectiveness of an internet-delivered treatment for substance abuse: Data from a multisite randomized controlled trial. Drug Alcohol Depend 2016 Apr 01;161:119-126 [FREE Full text] [CrossRef] [Medline]
    82. Cahill J, Barkham M, Hardy G, Rees A, Shapiro DA, Stiles WB, et al. Outcomes of patients completing and not completing cognitive therapy for depression. Br J Clin Psychol 2003 Jun;42(Pt 2):133-143. [CrossRef] [Medline]
    83. Kenter RM, Cuijpers P, Beekman A, van Straten A. Effectiveness of a web-based guided self-help intervention for outpatients with a depressive disorder: short-term results from a randomized controlled trial. J Med Internet Res 2016 Mar 31;18(3):e80 [FREE Full text] [CrossRef] [Medline]
    84. Kleiboer A, Smit J, Bosmans J, Ruwaard J, Andersson G, Topooco N, et al. European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries. Trials 2016 Aug 03;17(1):387 [FREE Full text] [CrossRef] [Medline]
    85. Lindhiem O, Bennett CB, Rosen D, Silk J. Mobile technology boosts the effectiveness of psychotherapy and behavioral interventions: a meta-analysis. Behav Modif 2015 Nov;39(6):785-804 [FREE Full text] [CrossRef] [Medline]
    86. Rose GL, Skelly JM, Badger GJ, Ferraro TA, Helzer JE. Efficacy of automated telephone continuing care following outpatient therapy for alcohol dependence. Addict Behav 2015 Feb;41:223-231 [FREE Full text] [CrossRef] [Medline]
    87. Moore BA, Fazzino T, Barry DT, Fiellin DA, Cutter CJ, Schottenfeld RS, et al. The recovery line: a pilot trial of automated, telephone-based treatment for continued drug use in methadone maintenance. J Subst Abuse Treat 2013 Jul;45(1):63-69 [FREE Full text] [CrossRef] [Medline]

    ‎
    f2f: face-to-face
    RCTs: randomized controlled trials

    Edited by G Eysenbach; submitted 31.08.16; peer-reviewed by J Fogel, R van der Vaart, UD Reips; comments to author 25.10.16; revised version received 06.06.17; accepted 27.07.17; published 15.09.17

    Copyright

    ©Doris Erbe, Hans-Christoph Eichert, Heleen Riper, David Daniel Ebert. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 15.09.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.