Exploring the Role of Persuasive Design in Unguided Internet-Delivered Cognitive Behavioral Therapy for Depression and Anxiety Among Adults: Systematic Review, Meta-analysis, and Meta-regression

doi:10.2196/26939

Review

¹Department of Psychology, University of Regina, Regina, SK, Canada

²PSPNET, University of Regina, Regina, SK, Canada

³Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institutet, Stockholm, Sweden

⁴Department of Psychology, Stockholm University, Stockholm, Sweden

Corresponding Author:

Hugh C McCall, MA

Department of Psychology

University of Regina

3737 Wascana Pkwy

Regina, SK, S4S 0A2

Canada

Phone: 1 306 585 4111

Email: hugh.c.mccall@gmail.com

Background: Internet-delivered cognitive behavioral therapy (ICBT) is an effective treatment that can overcome barriers to mental health care. Various research groups have suggested that unguided ICBT (ie, ICBT without therapist support) and other eHealth interventions can be designed to enhance user engagement and thus outcomes. The persuasive systems design framework captures most design recommendations for eHealth interventions, but there is little empirical evidence that persuasive design is related to clinical outcomes in unguided ICBT.

Objective: This study aims to provide an updated meta-analysis of randomized controlled trials of unguided ICBT for depression and anxiety, describe the frequency with which various persuasive design principles are used in such interventions, and use meta-regression to explore whether a greater number of persuasive design elements predicts efficacy in unguided ICBT for depression and anxiety.

Methods: We conducted a systematic review of 5 databases to identify randomized controlled trials of unguided ICBT for depression and anxiety. We conducted separate random effects meta-analyses and separate meta-regressions for depression and anxiety interventions. Each meta-regression included 2 steps. The first step included, as a predictor, whether each intervention was transdiagnostic. For the meta-regression of ICBT for depression, the first step also included the type of control condition. The number of persuasive design principles identified for each intervention was added as a predictor in the second step to reveal the additional variance in effect sizes explained by persuasive design.

Results: Of the 4471 articles we identified in our search, 46 (1.03%) were eligible for inclusion in our analyses. Our meta-analyses showed effect sizes (Hedges g) ranging from 0.22 to 0.31 for depression interventions, depending on the measures taken to account for bias in the results. We found a mean effect size of 0.45 (95% CI 0.33-0.56) for anxiety interventions, with no evidence that the results were inflated by bias. Included interventions were identified as using between 1 and 13 persuasive design principles, with an average of 4.95 (SD 2.85). The meta-regressions showed that a greater number of persuasive design principles predicted greater efficacy in ICBT for depression (R² change=0.27; B=0.04; P=.02) but not anxiety (R² change=0.05; B=0.03; P=.17).

Conclusions: These findings show wide variability in the use of persuasive design in unguided ICBT for depression and anxiety and provide preliminary support for the proposition that more persuasively designed interventions are more efficacious, at least in the treatment of depression. Further research is needed to clarify the role of persuasive design in ICBT.

J Med Internet Res 2021;23(4):e26939

doi:10.2196/26939

Keywords

ICBT; internet; depression; anxiety; persuasive design; eHealth

Background

Depression and anxiety are highly prevalent and represent the leading and the sixth leading causes of disability worldwide, respectively [1]. Despite the demonstrated efficacy of psychotherapeutic and pharmacological interventions for depression and anxiety [2-4], many people face structural barriers to accessing mental health care (eg, financial barriers, transportation barriers, inconvenience, and limited availability of services) [5,6]. Internet-delivered cognitive behavioral therapy (ICBT) is the most common type of internet intervention and an effective treatment for several common mental health problems, including depression and anxiety [7]. Unlike traditional cognitive behavioral therapy (CBT), ICBT enables users to access treatment materials privately at a time and location that is convenient for them, allowing it to be administered economically on a large scale and circumvent barriers to traditional forms of mental health care [8-10]. ICBT can be therapist guided or unguided. Guidance appears to improve adherence and clinical outcomes [11], but unguided ICBT is economical, highly scalable, and believed by many researchers to have considerable potential for improving public health [12-15].

Since the early 2000s, various research groups have suggested that eHealth interventions such as unguided ICBT can be designed in ways that improve user engagement and thus outcomes. In 2003, Fogg [16] presented the functional triad principle, suggesting that technology can function as a tool, a medium for relaying content, and a social actor to help facilitate behavior change. In 2009, Oinas-Kukkonen and Harjumaa [17] developed the persuasive systems design (PSD) framework, which elaborated on the functional triad and included 28 recommended design principles to produce more persuasive and engaging technological systems. They divided these principles into 4 categories: (1) primary task support principles, which facilitate the completion of the primary tasks of an intervention or other system; (2) dialogue support principles, through which an intervention or other system supports a user to help them enact their target behavior; (3) system credibility support principles, which facilitate a more credible and persuasive intervention or other system; and (4) social support principles, which leverage principles of social psychology to help users of an intervention or other system motivate one another. The 28 principles are described in Multimedia Appendix 1 [17].

Several other research groups have provided their own design recommendations for eHealth interventions. Despite using different terminology, most of these recommendations appear to align closely with the principles included in the PSD framework. Examples include recommendations related to personalization [18-22], tailoring [19,21,22], reminders [19,20], self-monitoring [18,23], liking [19,22,24], and various dialogue support principles [18,19,23]. A few design recommendations are not captured in the PSD framework (eg, time-limited access [20] and greater use of metaphors [22]), but to our knowledge, none of these have been proposed by 2 or more research groups; that is, the PSD framework appears to capture most common recommendations. Various groups’ recommendations and the related PSD framework principles are displayed in Multimedia Appendix 2 [18-24].

In 2012, Kelders et al [25] used the PSD framework to assess whether the persuasive design principles used in 83 eHealth interventions for chronic conditions, lifestyle changes, and mental health predicted adherence. They conducted a meta-regression, finding that a greater number of dialogue support principles predicted greater adherence to eHealth interventions. However, to our knowledge, there is no empirical research demonstrating a relationship between persuasive design and symptom change in eHealth interventions.

Objectives and Hypothesis

This study aims to (1) present a systematic review and meta-analysis of randomized controlled trials of unguided ICBT for depression and anxiety among adults, (2) systematically examine the frequency with which various persuasive design principles are used in such interventions, and (3) use meta-regression to examine the extent to which persuasive design could explain the variability in effect sizes identified through the meta-analysis. Thus, the overarching objective of this study is to review the efficacy, the use of persuasive design, and the relationship between efficacy and persuasive design in unguided ICBT for depression and anxiety. We hypothesized that using a greater number of persuasive design principles would predict greater efficacy among the included studies.

Study Design

This study consisted of a systematic review, 2 meta-analyses, and 2 meta-regressions. The methods used in each phase of the study are described in the following sections. We registered the methodological protocol for this study on PROSPERO on October 24, 2019 (ID: 153466), before commencing the literature search, and kept a log of revisions to the original protocol throughout the course of this research (Multimedia Appendix 3 [26-28]). We followed the guidelines outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement in the preparation of this paper [29].

Systematic Review Methods

Eligibility Criteria

We searched for randomized controlled trials of unguided ICBT interventions for symptoms of depression and/or anxiety among adults that had been published in English in academic journals since 2000. We included trials of ICBT targeting symptoms of any type of depressive or anxiety disorder, as defined in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders [30], with various kinds of control conditions (eg, waitlist, treatment as usual, and active control). Studies involving samples with a mean age of less than 18 years were excluded.

Although we excluded studies in which ICBT was delivered with guidance from a therapist or coach, we did not exclude studies involving diagnostic interviews or contact of a logistical nature between participants and research teams. Interventions that used a CBT model of treatment and were delivered via the internet were considered ICBT interventions regardless of whether the authors of trials identified them as such. We included interventions using third-wave CBT approaches (eg, mindfulness-based CBT and acceptance and commitment therapy) [31] because prior research has not demonstrated significant differences in outcomes between traditional CBT and third-wave approaches [32,33].

Literature Search

On October 29, 2019, we conducted a literature search on MEDLINE, PsycINFO, PubMed, Web of Science, and PsycArticles. To be identified, articles were required to include the words “CBT,” “internet,” “trial,” and “depression” or “anxiety” or one of several similar phrases for each of these terms in their titles, keywords, or abstracts. The search terms are shown in detail in Multimedia Appendix 4. This search was updated on July 2, 2020.

Study Selection

After removing duplicates of studies identified in 2 or more databases, HCM and CRFS independently screened the studies in 3 stages: by title, by abstract, and by full text. Wherever the 2 screeners reached different decisions about whether to retain or exclude a study, that study was included in the next stage of screening. Differences in decisions on the full-text screening were resolved through discussion.

Data Extraction

HCM extracted several types of data from each study: study characteristics (eg, type of control condition and time between pretreatment and posttreatment measures), risk of bias [34], general intervention characteristics (eg, target symptoms and medium of delivery), persuasive design principles [17] as operationalized by Kelders et al [25], and efficacy data. Consistent with the approach of Kelders et al [25], we did not code principles in the system credibility support category of the PSD framework because they were reported very infrequently and would have been challenging to code objectively (eg, a system should have a “competent look and feel” and “provide endorsements from respected sources” [17]). In most cases, we coded persuasive design principles as present or absent based on the descriptions of interventions in the included studies, although we consulted other available sources of information when possible (eg, intervention websites and study protocols). The complete list of data items is provided in Multimedia Appendix 5. The persuasive design principle tunneling, which refers to the sequential presentation of treatment elements in a structured, linear manner, was not counted toward the total number of persuasive design principles in this study. This is because researchers have recently proposed that eHealth interventions can be made more engaging by providing users with greater flexibility and control concerning the modules or features they wish to use [19,22], which contrasts with the principle of tunneling.

Risk of Bias Assessment

We assessed the risk of bias among included studies using the Cochrane risk of bias tool [34]. We did not assess the risk domain blinding participants and personnel because it is not possible for participants to be blind to their conditions in psychotherapy research [35]. Furthermore, we did not assess the risk domain blinding of outcome assessment because all outcome measures were self-report measures, and participants could thus not be blinded. Self-report measures are generally considered equivalent to blind clinical observers in psychotherapy research, and research suggests that they do not result in inflated effect sizes [35].

Meta-analysis Methods

We conducted meta-analyses using Comprehensive Meta-Analysis software (Biostat Inc) [36]. As prior research suggests that ICBT for generalized and social anxiety is more efficacious than ICBT for depression [7,37], we conducted separate meta-analyses of ICBT for anxiety and ICBT for depression. Given the availability of symptom change data for both anxiety and depression, trials of ICBT designed to treat both conditions were included in both meta-analyses. We measured heterogeneity in the effect sizes of the included studies using the I² index and formally tested the degree of heterogeneity using the Q statistic [38]. In each of the 2 meta-analyses, we used a random effects model, used between-groups effect size (Hedges g) as the summary measure, and weighted each study by the inverse of the within-study variance of the primary outcome measure plus the between-study variance. Several studies evaluated 2 unguided ICBT interventions; in such cases, we treated the evaluation of each intervention as a separate study, except we divided the control group sample size by 2, such that each control group participant was included only once in the analyses [39]. We evaluated the risk of publication bias using funnel plots and accounted for publication bias using the trim and fill technique [40]. We explored the influence of study-level bias on outcomes by repeating the meta-analyses without studies deemed to be at high risk on one or more dimensions of the Cochrane tool for assessing risk of bias [34].

Meta-regression Methods

We conducted 2 meta-regressions using Comprehensive Meta-Analysis [36]—one for depression interventions and one for anxiety interventions—to determine the degree to which persuasive design principles could explain variance in effect sizes among studies. Paralleling the approach taken to the meta-analyses, we included trials of ICBT designed to treat both depression and anxiety in both meta-regressions, given the availability of symptom change data for both conditions. We also weighted each study by the inverse of the within-study variance of the primary outcome measure plus the between-study variance, as in the meta-analyses.

We used 3 predictor variables. Our main predictor of interest was the total number of persuasive design principles identified for each intervention. We were unable to include the number of persuasive design principles in each category of the PSD framework as separate predictors, as Kelders et al [25] did, because of the risk of overfitting, given the limited number of included studies. We also input a binary variable reflecting whether each intervention was transdiagnostic (ie, designed to treat symptoms of both depression and anxiety). We did this to account for the possibility that unguided ICBT focused on treating a narrower range of symptoms (ie, anxiety or depression) may be more efficacious for treating those symptoms than transdiagnostic unguided ICBT designed to treat a broader range of symptoms (ie, both depression and anxiety). Our final predictor was a binary variable reflecting whether each study used a control condition with active elements (eg, psychoeducation and mood monitoring) because a previous meta-analysis of unguided ICBT found a large mean effect size among studies using passive control conditions and a small mean effect size among studies using active control conditions [41]. However, the control condition type was not included as a predictor in the meta-regression of ICBT for anxiety because there were insufficient studies to justify an additional predictor variable (eg, because of the risk of overfitting), following most recommendations concerning acceptable subjects per variable ratios in linear regression analyses [26].

We conducted each meta-regression in 2 steps. The first step included transdiagnostic status and, for the meta-regression of ICBT for depression, the control condition type. In both meta-regressions, the number of persuasive design principles identified was then added in the second step. This 2-step approach was used to reveal the amount of additional variance persuasive design explained in the second step after accounting for the other variables in the first step.

We conducted 5 assumption tests at each step of each meta-regression. First, we examined Pearson r correlations and scatterplots to test the assumption of linearity of the relationship between each continuous predictor variable and Hedges g [42]. Second, we checked Cook distance values to identify any outlier studies that had unduly large influences on the results [43]. Third, we inspected the distribution of studentized residuals using a histogram to ensure that the residuals were normally distributed [42]. Fourth, we inspected scatterplots plotting studentized residuals against predicted values to test the assumption of homoscedasticity [42]. Finally, we examined variance inflation factors to check for multicollinearity [42].

Systematic Review Results

Study Selection

Between the original and updated literature searches, we identified 4471 articles, 39 of which were found eligible for analysis. Having found another 7 eligible articles through a hand search, we included a total of 46 articles. The flow of studies through the study selection process is shown in Figure 1. Separate flowcharts for the original and updated literature searches are shown in Multimedia Appendices 6 and 7, respectively. The 2 screeners (HCM and CRFS) made the same screening decision (ie, retain or remove) for 81.66% (2066/2530) of articles during the title screening, 86.39% (1035/1198) of articles during the abstract screening, and 81.9% (276/337) of articles during the full-text screening.

Study Characteristics

The 46 eligible studies included 16,632 participants, excluding participants assigned to experimental groups irrelevant to this study (eg, guided ICBT groups). Studies were most often published in or after 2017 (24/46, 52%); included samples drawn from the general population (26/46, 57%), clinical populations (14/46, 30%), or both (6/46, 13%); and most often used waitlist control conditions without active elements (28/46, 61%). The characteristics of each study are presented in Table 1.

Table 1. Study characteristics.

Category and study		Intervention	Participant, n^a	Duration in weeks^b,c	Control condition	Recruitment population
						Clinical	Nonclinical
ICBT^d for depression
	Berger et al, 2011 [44]	Deprexis	51	10	Waitlist		✓
	Bücker et al, 2019 [45]	MOOD	125	6	Care as usual	✓	✓
	Clarke et al, 2002 [46]	ODIN^e	299	4	Health information website	✓
	Clarke et al, 2005 [47]	ODIN	175	5	Health information website	✓
	Clarke et al, 2009 [48]	—^f	160	16	Health information website	✓
	Dahne et al, 2019 [49]	¡Aptívate!	33	8	Care as usual	✓	✓
	Dahne et al, 2019 [49]	iCouch CBT	20	8	Care as usual	✓	✓
	Dahne et al, 2019 [50]	Moodivate	33	8	Care as usual	✓
	Dahne et al, 2019 [50]	MoodKit	28	8	Care as usual	✓
	de Graaf et al, 2009 [51]	Colour Your Life	203	13.05	Care as usual		✓
	Farrer et al, 2011 [52]	MoodGym and Bluepages	73	6	Care as usual	✓
	Gräfe et al, 2020 [53]	Deprexis	3805	12	Brochure and care as usual	✓
	Hur et al, 2018 [54]	Todac Todac^g	34	3	Mood charting app	✓	✓
	Lintvedt et al, 2013 [55]	MoodGym and Bluepages	163	8	Waitlist		✓
	Löbner et al, 2018 [56]	MoodGym (German adapted, version III)	647	6	Care as usual	✓
	Lüdtke et al, 2018 [57]	Be good to yourself	88	4	Waitlist	✓	✓
	Lüdtke et al, 2018 [58]	—	132	4	Care as usual	✓
	McDermott and Dozois, 2019 [59]	MoodGym	302	8	Attentional control		✓
	Meyer et al, 2009 [60]	Deprexis	396	9	Waitlist		✓
	Meyer et al, 2015 [61]	Deprexis	163	13.05	Waitlist	✓	✓
	Mira et al, 2017 [62]	Sonreír es Divertido^h	80	12	Waitlist		✓
	Mohr et al, 2013 [63]	moodManager	68	6	Waitlist	✓
	Montero-Marin et al, 2016 [64]	Smiling is Fun	124	13.05	Improved treatment as usual	✓
	Moritz et al, 2012 [65]	Deprexis	210	8	Waitlist		✓
	Morris et al, 2015 [66]	Panoply	166	3	Web-based expressive writing		✓
	Noguchi et al, 2017 [67]	—	651	5	Waitlist		✓
	Schure et al, 2019 [68]	Thrive	343	8	Depression information website		✓
	Silverstone et al, 2017 [69]	MoodGym	109	12	Care as usual	✓
	Spek et al, 2007 [70]	—	202	10	Waitlist		✓
ICBT for depression and anxiety
	Bakker et al, 2018 [71]	MoodKit	120	4.29	Waitlist		✓
	Bakker et al, 2018 [71]	MoodMission	114	4.29	Waitlist		✓
	Kleiboer et al, 2015 [72]	Allesondercontroleⁱ	213	6	Waitlist and web-based information		✓
	Moberg et al, 2019 [73]	Pacifica	500	4.35	Waitlist		✓
	Powell et al, 2013 [74]	MoodGym	3070	6	Waitlist		✓
	Proudfoot et al, 2013 [75]	myCompass	459	8	Waitlist		✓
	Shirotsuki et al, 2017 [76]	—	48	6	Mood monitoring		✓
	Twomey et al, 2014 [77]	MoodGym	66	4.57	Waitlist	✓
ICBT for anxiety
	Berger et al, 2017 [78]	Velibra	139	9	Waitlist	✓
	Boettcher et al, 2018 [79]	Challenger	139	7	Waitlist		✓
	Boettcher et al, 2018 [79]	—	139	7	Waitlist		✓
	Botella et al, 2010 [80]	Talk to Me	91	8.7	Waitlist		✓
	Ciuca et al, 2018 [81]	PAXPD^j	75	12	Waitlist	✓	✓
	Donker et al, 2019 [82]	0Phobia	193	3	Waitlist		✓
	Ivanova et al, 2016 [83]	Ångesthjälpen^k	102	10	Waitlist		✓
	Kenardy et al, 2003 [84]	—	83	6	Waitlist		✓
	Lin et al, 2020 [85]	—	26	8	Waitlist		✓
	McCall et al, 2018 [86]	Overcome Social Anxiety	101	17.4	Waitlist		✓
	Oh et al, 2020 [87]	Todaki	41	4	Book on panic disorder	✓
	Powell et al, 2020 [88]	E-couch	2116	6	Waitlist		✓
	Titov et al, 2008 [89]	Shyness	64	10	Waitlist		✓

^aFor the purpose of this table, n was calculated as the number of participants assigned to the intervention identified in each row plus the number of participants assigned to the control condition (ie, excluding participants assigned to use other interventions).

^bStudy duration expressed in days was divided by 7. Study duration expressed in months was multiplied by 4.35 (the average number of weeks in a month during a 365-day year).

^cSome studies reported data from multiple posttreatment time points; for such studies, the duration, as shown in this table, is the number of weeks between pretreatment and whichever posttreatment time point was selected for use in the analyses reported in this study.

^dICBT: internet-delivered cognitive behavioral therapy.

^eODIN: Overcoming Depression on the Internet.

^fData were not reported.

^gTodac Todac translates to “Tap Tap.”

^hSonreír es Divertido translates to “Smiling is Fun.”

ⁱAllesondercontrole translates to “all is under control.”

^jPAXPD: PAXonline Program for Panic Disorder.

^kÅngesthjälpen translates to “The Anxiety Help.”

Risk of Bias

We evaluated the risk of bias among included studies using 5 of the 7 domains in the Cochrane risk of bias tool [34]. Of the 46 included studies, 14 (30%) were identified to be at high risk of bias in at least one domain, whereas only 4 (9%) were found to be at low risk of bias in all domains assessed. Most studies (28/46, 61%) were found to be at low or unclear risk in each domain. The risk of bias identified in each study is presented in Table 2.

Table 2. Risk of bias in included studies.

Category and study			Random sequence generation		Allocation concealment		Incomplete outcome data (attrition bias)		Selective reporting		Other bias
ICBT^a for depression
	Berger et al, 2011 [44]	Low		Unclear		Low		Unclear		Low
	Bücker et al, 2019 [45]	Low		Low		Low		Unclear		Low
	Clarke et al, 2002 [46]	Low		Low		Low		Unclear		Low
	Clarke et al, 2005 [47]	Low		Low		High		Unclear		Low
	Clarke et al, 2009 [48]	Unclear		Unclear		Low		Unclear		Unclear
	Dahne et al, 2019 [49]	Unclear		Unclear		Low		Unclear		Low
	Dahne et al, 2019 [50]	Unclear		Unclear		Unclear		Unclear		Low
	de Graaf et al, 2009 [51]	Low		Low		Low		Low		Low
	Farrer et al, 2011 [52]	Unclear		Low		Low		Unclear		Low
	Gräfe et al, 2020 [53]	Low		Unclear		Low		Low		Low
	Hur et al, 2018 [54]	Low		Low		Low		Unclear		Low
	Lintvedt et al, 2013 [55]	Low		Unclear		Low		Unclear		Low
	Löbner et al, 2018 [56]	Low		Low		Low		Unclear		Low
	Lüdtke et al, 2018 [57]	Unclear		Low		Low		Unclear		Low
	Lüdtke et al, 2018 [58]	High		Low		Low		Unclear		Low
	McDermott and Dozois, 2019 [59]	Unclear		Unclear		Low		Unclear		Unclear
	Meyer et al, 2009 [60]	Low		High		High		Unclear		Low
	Meyer et al, 2015 [61]	Low		Low		Low		Unclear		Low
	Mira et al, 2017 [62]	Low		Low		Low		Unclear		Low
	Mohr et al, 2013 [63]	Low		Low		Low		Unclear		Low
	Montero-Marin et al, 2016 [64]	Low		Low		Low		Low		Low
	Moritz et al, 2012 [65]	Unclear		Unclear		Low		Unclear		Low
	Morris et al, 2015 [66]	Unclear		Unclear		Low		Unclear		Low
	Noguchi et al, 2017 [67]	Low		Low		Low		Unclear		Low
	Schure et al, 2019 [68]	Unclear		Unclear		Low		Low		Low
	Silverstone et al, 2017 [69]	High		High		High		Unclear		Low
	Spek et al, 2007 [70]	Unclear		Low		Low		Unclear		Low
ICBT for depression and anxiety
	Bakker et al, 2018 [71]	High		High		High		Unclear		Low
	Kleiboer et al, 2015 [72]	Low		Low		High		Unclear		Unclear
	Moberg et al, 2019 [73]	Unclear		Unclear		High		Unclear		Low
	Powell et al, 2013 [74]	Low		Low		Low		Low		Low
	Proudfoot et al, 2013 [75]	Low		Low		High		Unclear		Low
	Shirotsuki et al, 2017 [76]	Unclear		Low		Low		Unclear		Low
	Twomey et al, 2014 [77]	Low		High		High		Unclear		Low
ICBT for anxiety
	Berger et al, 2017 [78]	Low		Low		Low		Unclear		Low
	Boettcher et al, 2018 [79]	Low		Low		Low		Unclear		Low
	Botella et al, 2010 [80]	Unclear		Unclear		High		Unclear		Unclear
	Ciuca et al, 2018 [81]	Low		Low		Low		Unclear		Low
	Donker et al, 2019 [82]	Low		Low		Low		Low		Low
	Ivanova et al, 2016 [83]	Low		Low		High		Low		Low
	Kenardy et al, 2003 [84]	Unclear		Unclear		Low		Unclear		Unclear
	Lin et al, 2020 [85]	Low		Unclear		High		Unclear		Unclear
	McCall et al, 2018 [86]	Low		High		Low		Unclear		Low
	Oh et al, 2020 [87]	Unclear		Unclear		Low		Low		Low
	Powell et al, 2020 [88]	Low		Low		High		Low		Low
	Titov et al, 2008 [89]	Low		Unclear		Low		Unclear		Low

^aICBT: internet-delivered cognitive behavioral therapy.

Intervention Characteristics

In total, 37 unguided ICBT interventions were evaluated in the 46 included studies. Of these 37 interventions, 15 (41%) were designed to treat depression exclusively and 9 (24%) were designed to treat depression and anxiety or stress. Other interventions were designed to treat social anxiety (6/37, 16%), panic (2/37, 5%), fear of public speaking (1/37, 3%), generalized anxiety (1/37, 3%), acrophobia (1/37, 3%), or symptoms of multiple anxiety disorders (2/37, 5%). Most interventions (23/37, 62%) were described as traditional CBT interventions, but many interventions (9/37, 24%) were described as being strongly influenced by elements of third-wave CBT (eg, mindfulness) or other therapeutic approaches (eg, positive psychology), and several interventions were based on behavioral activation (2/37, 5%), cognitive therapy (2/37, 5%), or problem-solving therapy (1/37, 3%). Half of the interventions (19/37, 51%) were delivered via a web browser, but many interventions were delivered via a mobile app (11/37, 30%) or both a browser and an app (5/37, 14%). Of the 37 interventions, it was unclear how 2 (5%) interventions were delivered. The characteristics of each intervention are presented in Table 3.

Table 3. Intervention characteristics.

Study	Name of the intervention	Target symptoms	Theoretical approach	Composition	Delivery medium	Number of persuasive design principles identified
Dahne et al, 2019 [49]	¡Aptívate!	Depression	Behavioral activation	Unclear	Mobile app	8
Donker et al, 2019 [82]	0Phobia	Acrophobia	CBT^a	6 modules	Mobile app	6
Kleiboer et al, 2015 [72]	Allesondercontrole^b	Depression and anxiety	Problem-solving therapy	5 lessons	Web browser	1
Ivanova et al, 2016 [83]	Ångesthjälpen^c	Panic and social anxiety	Acceptance and commitment therapy	8 modules	App, browser, and CD	4
Lüdtke et al, 2018 [57]	Be Good to Yourself	Depression	CBT with third-wave elements	4 modules	Mobile app	4
Boettcher et al, 2018 [79]	Challenger	Social anxiety	CBT	N/A^d	Mobile app	13
de Graaf et al, 2009 [51]	Colour Your Life	Depression	CBT	9 modules	Web browser	2
Berger et al, 2011 [44]; Gräfe et al, 2020 [53]; Meyer et al, 2009 [60]; Meyer et al, 2015 [61]; and Moritz et al, 2012 [65]	Deprexis	Depression	CBT and other approaches	12 modules	Web browser	5
Powell et al, 2020 [88]	E-couch	Social anxiety	CBT	6 modules	App and browser	2
Dahne et al, 2019 [49]	iCouch CBT	Depression and anxiety	CBT	Unclear	Mobile app	1
Bücker et al, 2019 [45]	MOOD	Depression	CBT with third-wave elements	9 modules	Web browser	5
Farrer et al, 2011 [52]; Lintvedt et al, 2013 [55]; Löbner et al, 2018 [56]; McDermott and Dozois, 2019 [59]; and Twomey et al, 2014 [77]	MoodGym^e	Depression and anxiety	CBT and other approaches	5 modules	Web browser	1
Dahne et al, 2019 [50]	Moodivate	Depression	Behavioral activation	7 modules	Mobile app	8
Bakker et al, 2018 [71] and Dahne et al, 2019 [50]	MoodKit	Depression and anxiety	CBT	4 features	Mobile app	5
Mohr et al, 2013 [63]	moodManager	Depression	CBT	18 lessons	Web browser	4
Bakker et al, 2018 [71]	MoodMission	Depression and anxiety	CBT	N/A	Mobile app	4
Proudfoot et al, 2013 [75]	myCompass	Depression, anxiety, and stress	CBT and other approaches	12 modules	App and browser	6
McCall et al, 2018 [86]	Overcome Social Anxiety	Social anxiety	CBT	7 modules	Web browser	8
Clarke et al, 2002 [46] and Clarke et al, 2005 [47]	ODIN^f	Depression	Cognitive therapy	7 modules	Web browser	5
Moberg et al, 2019 [73]	Pacifica	Depression, anxiety, and stress	CBT and other approaches	Unclear	Mobile app	8
Morris et al, 2015 [66]	Panoply	Depression	Cognitive therapy	N/A	Web browser	8
Ciuca et al, 2018 [81]	PAXPD^g	Panic disorder	CBT	16 modules	Web browser	3
Titov et al, 2008 [89]	Shyness	Social anxiety	CBT	6 lessons	Web browser	8
Mira et al, 2017 [62] and Montero-Marin et al, 2016 [64]	Sonreír es Divertido^h	Depression	CBT and other approaches	10 modules	Web browser	5
Botella et al, 2010 [80]	Talk to Me	Fear of public speaking	CBT	Unclear	Web browser	6
Schure et al, 2019 [68]	Thrive	Depression	CBT	3 modules	App and browser	3
Hur et al, 2018 [54]	Todac Todacⁱ	Depression	CBT	3 modules	Mobile app	7
Oh et al, 2020 [87]	Todaki	Panic	CBT	4 modes	Mobile app	7
Berger et al, 2017 [78]	Velibra	Various anxiety disorders	CBT and other approaches	6 sessions	Web browser	5
Boettcher et al, 2018 [79]	Not reported	Social anxiety disorder	CBT	9 modules	Unclear	2
Clarke et al, 2009 [48]	Not reported	Depression	CBT	4 sections	Web browser	8
Kenardy et al, 2003 [84]	Not reported	Anxiety	CBT	6 sessions	Web browser	3
Lin et al, 2020 [85]	Not reported	Social anxiety	CBT	8 modules	Web browser	10
Lüdtke et al, 2018 [58]	Not reported	Depression	CBT	1 module	App and browser	3
Noguchi et al, 2017 [67]	Not reported	Depression and stress	CBT	Unclear	Web browser	1
Shirotsuki et al, 2017 [76]	Not reported	Depression- and anxiety-related symptoms	CBT	6 modules	e-learning system and guidebook	2
Spek et al, 2007 [70]	Not reported	Depression	CBT	8 modules	Web browser	2

^aCBT: cognitive behavioral therapy.

^bAllesondercontrole translates to “all is under control.”

^cÅngesthjälpen translates to “The Anxiety Help.”

^dN/A: not applicable.

^eBluepages was offered as a complement to MoodGym in studies by Farrer et al [52] and Lintvedt et al [55] but was omitted from this table (and all analyses) because it is a psychoeducation package and not an internet-delivered cognitive behavioral therapy intervention.

^fODIN: Overcoming Depression on the Internet.

^gPAXPD: PAXonline Program for Panic Disorder.

^hSonreír es Divertido translates to “Smiling is Fun.”

ⁱTodac Todac translates to “Tap Tap.”

Persuasive Design

On average, interventions included 4.95 (SD 2.85) persuasive design principles (excluding tunneling). The total number of persuasive design elements ranged from 1 to 13. Principles in the primary task support category were the most common (mean 2.86, SD 1.32), followed by principles in the dialogue support category (mean 1.27, SD 1.19) and social support category (mean 0.81, SD 1.60). The number of interventions in which each persuasive design principle was identified is presented in Table 4.

Table 4. Persuasive design principles identified.

Persuasive design principle			Brief description^a		Interventions used, n (%)
Primary task support
	Reduction	Divides target behavior into simple steps		35 (95)
	Tunneling	Delivers content in a step-by-step format		29 (78)
	Tailoring	Provides content adapted to user group		2 (5)
	Personalization	Provides content that is adapted to one user		18 (49)
	Self-monitoring	Provides ability to monitor progress or status		20 (54)
	Simulation	Provides ability to observe relevant behavior		6 (16)
	Rehearsal	Provides ability to rehearse a behavior		25 (68)
Dialogue support
	Praise	Offers praise to participant		8 (22)
	Rewards	Offers reward to participant		5 (14)
	Reminders	Provides reminders		13 (35)
	Suggestion	Provides suggestions		15 (41)
	Similarity	Is designed to look familiar		0 (0)
	Liking	Is visually designed to be attractive		1 (3)
	Social role	Acts as if it has a social role		5 (14)
Social support
	Social learning	Facilitates learning from other users		7 (19)
	Social comparison	Facilitates comparison with other users		5 (14)
	Normative influence	Provides normative information on target behavior		2 (5)
	Social facilitation	Facilitates awareness of others using intervention		5 (14)
	Cooperation	Stimulates users to cooperate		8 (22)
	Competition	Stimulates users to compete		0 (0)
	Recognition	Shows users who adopted target behavior		3 (8)

^aThese descriptions were adapted from the operational definitions provided by Kelders et al [25].

Meta-analysis Results

Meta-analysis of Unguided ICBT for Depression

We conducted a meta-analysis of 37 comparisons across 34 trials of unguided ICBT for depression. There was statistically significant heterogeneity in Hedges g among the studies (Q=89.85, df=36; P<.001). An I² statistic of 59.93 indicated that a moderate proportion of variability was attributable to true heterogeneity rather than sampling error [90,91]. The weighted mean between-subjects effect size was small to moderate (Hedges g=0.31; SE 0.04; 95% CI 0.24-0.38). The forest plot for this meta-analysis is shown in Figure 2. Weighted mean effect sizes after excluding studies deemed to be at high risk of bias and after adjusting for publication bias using the trim and fill technique [40] are presented in Table 5.

Figure 2. Meta-analysis of unguided internet-delivered cognitive behavioral therapy for depression.

Table 5. Summary statistics of meta-analyses with and without bias corrections.

Meta-analysis			Hedges g (95% CI)
Meta-analysis of ICBT^a for depression
	All studies of ICBT for depression	0.31 (0.24-0.38)
	All studies with trim and fill adjustment	0.23 (0.16-0.31)
	Studies with high risk of bias excluded	0.28 (0.20-0.36)
	Studies with high risk of bias excluded, with trim and fill adjustment	0.22 (0.14-0.31)
Meta-analysisof ICBT for anxiety
	All studies of ICBT for anxiety	0.45 (0.33-0.56)
	All studies with trim and fill adjustment	0.45 (0.33-0.56)
	Studies with high risk of bias excluded	0.54 (0.29-0.79)
	Studies with high risk of bias excluded, with trim and fill adjustment	0.54 (0.29-0.79)

^aICBT: internet-delivered cognitive behavioral therapy.

Meta-analysis of Unguided ICBT for Anxiety

We included 19 studies that reported 21 comparisons in a meta-analysis of unguided ICBT for anxiety. The results indicated statistically significant heterogeneity of Hedges g among studies (Q=68.47, df=20; P<.001). The corresponding I² statistic of 70.79 suggested that a substantial proportion of the variability represented true heterogeneity [90,91]. The weighted mean between-subjects effect size was moderate (Hedges g=0.45; SE 0.06; 95% CI 0.33-0.56). A forest plot is shown in Figure 3. Additional weighted mean effect sizes accounting for publication- and study-level bias are presented in Table 5.

Figure 3. Meta-analysis of unguided internet-delivered cognitive behavioral therapy for anxiety.

Meta-regression Results

Meta-regression of Unguided ICBT for Depression

The meta-regression of ICBT for depression, like the meta-analysis of ICBT for depression, included 34 studies reporting 37 comparisons. We used 3 predictors in this meta-regression: the total number of persuasive design principles (mean 3.90, SD 2.33), whether each intervention was designed to treat symptoms of both depression and anxiety (19/37, 51%) or only depression (18/37, 49%), and whether each study used an active control condition (13/37, 35%) or a passive control condition (24/37, 65%). The results for both steps of the meta-regression are presented in Table 6. With the possible exception of very minor heteroscedasicity of residuals at one or both steps, all assumptions were met, as detailed in Multimedia Appendix 8.

Table 6. Meta-regression of unguided internet-delivered cognitive behavioral therapy for depression.

Step and variable		Model							Predictors
		Model summary			R²		R² change		B^a		SE		95% CI		P value		Variance inflation factor
		Q (df)	P value
Step 1		1.05 (2)	.59	0.0		N/A^b
	Constant							0.28		0.07		0.15 to 0.41		<.001		3.03
	Active control condition							−0.02		0.08		−0.19 to 0.14		.79		1.11
	Transdiagnostic intervention							0.07		0.08		−0.09 to 0.23		.40		1.11
Step 2		6.74 (3)	.08	0.27		0.27
	Constant							0.10		0.10		−0.09 to 0.29		.32		8.63
	Active control condition							−0.01		0.07		−0.16 to 0.13		.85		1.14
	Transdiagnostic intervention							0.13		0.08		−0.02 to 0.28		.09		1.36
	Persuasive design principles							0.04		0.02		0.01 to 0.07		.02		1.22

^aUnstandardized β coefficient.

^bN/A: not applicable.

Meta-regression of Unguided ICBT for Anxiety

Similar to the meta-analysis of ICBT for anxiety, the meta-regression of ICBT for anxiety included 19 studies reporting 21 comparisons. We used 2 predictors: the total number of persuasive design principles (mean 5.05, SD 3.17) and whether each intervention was designed to treat symptoms of both depression and anxiety (8/21, 38%) or only anxiety (13/21, 62%). The results for both steps of the meta-regression are presented in Table 7. The assumption of normality of residuals may not have been met fully at both steps, although the residuals roughly approximated normal distributions. The assumption of homoscedasticity of the residuals was violated in step 1. The assumption tests for this meta-regression are detailed in Multimedia Appendix 9.

Table 7. Meta-regression of unguided internet-delivered cognitive behavioral therapy for anxiety.

Step and variable		Model					Predictors
		Model summary		R²	R² change		B^a		SE		95% CI		P value		Variance inflation factor
		Q (df)	P value
Step 1		4.80 (1)	.03	0.0	N/A^b
	Constant					0.57		0.08		0.41 to 0.73		<.001		1.77
	Transdiagnostic intervention					−0.27		0.12		−0.51 to −0.03		.03		1.00
Step 2		7.55 (2)	.02	0.05	0.05
	Constant					0.42		0.13		0.18 to 0.67		<.001		5.12
	Transdiagnostic intervention					−0.23		0.12		−0.46 to −0.00		.049		1.07
	Persuasive design principles					0.03		0.02		−0.01 to 0.06		.17		1.07

^aUnstandardized β coefficient.

^bN/A: not applicable.

Principal Findings

Recent years have witnessed a proliferation of randomized trials of eHealth interventions, including many trials of unguided ICBT for depression and anxiety. Indeed, most of the studies included in this review were published in or after 2017. There was considerable diversity in the design of both studies (eg, study duration and type of control condition) and interventions (eg, mode of delivery and use of persuasive design principles).

The results of the meta-analysis of unguided ICBT for depression were consistent with the results of previous meta-analyses. We reported 4 mean effect sizes (Hedges g) for unguided ICBT for depression, ranging from 0.22 to 0.31, based on the corrections we made for publication bias and study-level bias. Previous meta-analyses of unguided ICBT for depression have found comparable mean effect sizes (Hedges g or Cohen d) ranging from 0.24 to 0.36 [12,92-95]. Our meta-analysis of unguided ICBT for anxiety yielded a mean effect size of 0.45. There was no evidence of publication bias, and the mean effect size was greater (Hedges g=0.54) after excluding studies found to be at a high risk of bias. Several previous meta-analyses of ICBT for symptoms of anxiety disorders found effect sizes between 0.70 and 1.12 [41,96-98]; however, all these meta-analyses included trials of guided ICBT interventions, which likely explains the greater mean effect sizes, at least in part. We are aware of only 1 meta-analysis that has included a subgroup analysis of unguided ICBT for anxiety—social anxiety, specifically—finding mean effect sizes (Hedges g) of 0.78 and 0.19 for studies using passive and active control conditions, respectively [41]. It is worth noting that our review included many transdiagnostic interventions designed to treat symptoms of both depression and anxiety. The meta-regressions showed that these interventions were significantly less efficacious for treating anxiety symptoms compared with interventions designed to treat anxiety symptoms only; however, their efficacy in treating depression did not significantly differ from interventions designed to treat symptoms of depression only.

We identified wide variability in the use of persuasive design in unguided ICBT for depression and anxiety, with several interventions using only 1 persuasive design principle and others using as many as 13. The intervention identified as having the greatest number of persuasive design principles (ie, 13), called Challenger, was specifically designed to be engaging, with many features inspired by the literature on gamification [79,99]. The mean number of persuasive design principles identified across interventions (4.95, excluding the principle of tunneling) was comparable with the mean of 5.4 principles identified by Kelders et al [25] among mental health interventions in their review. The mean number of persuasive design principles identified in the primary task support (mean 2.86, SD 1.32; excluding tunneling), dialogue support (mean 1.27, SD 1.19), and social support (mean 0.81, SD 1.60) categories were also roughly comparable with the corresponding means identified among mental health interventions by Kelders et al [25] (2.6, 1.6, and 1.3, respectively).

Persuasive design was a significant predictor of effect size in the meta-regression of ICBT for depression. The unstandardized β coefficient (B) of 0.04 suggested that for each additional persuasive design principle an intervention uses, one could predict the effect size (Hedges g) for that intervention to increase by 0.04, compared with a control condition in a randomized trial. However, meta-regression is an inherently observational procedure [100], and the results therefore could not show whether persuasive design caused certain ICBT interventions for depression to be more efficacious than others. Persuasive design did not predict efficacy in the meta-regression of ICBT for anxiety. However, it is worth noting that the meta-regression of ICBT for anxiety included far fewer studies than the meta-regression of ICBT for depression and had limited statistical power to identify an effect. Indeed, persuasive design had an unstandardized β coefficient of 0.03 in the meta-regression of ICBT for anxiety, which—although not statistically significant—was comparable in magnitude with that of the meta-regression of ICBT for depression. The results of the meta-regression of unguided ICBT for anxiety should be interpreted cautiously because assumption tests showed that certain assumptions were unmet. Nonetheless, our results suggest that persuasive design is more closely related to outcomes in interventions for depression than anxiety. Given that persuasive design is purported to motivate engagement in treatment [17] and that lack of motivation is a hallmark of depression, it is possible that persuasive design is particularly important in ICBT for depression.

Overall, our findings support the hypothesis that persuasive design predicts efficacy in unguided ICBT, at least in the treatment of depression. Our findings also support the validity of the PSD framework [17] by showing that it is meaningfully related to treatment outcomes. Although the results do not demonstrate the importance of any specific persuasive design principles, they support the growing body of theory and data suggesting, broadly, that persuasive design matters in eHealth [18-24]. These findings are encouraging and timely. ICBT has become well established over the last two decades, having now been evaluated in hundreds of trials [101] and currently being funded by many governments around the world [102]. It is clear that ICBT is effective, and a natural next step in ICBT research will be to explore possible avenues for making it more effective. Our findings suggest that enhanced persuasive design may be one such avenue. Notably, because ICBT is highly scalable, particularly when it is unguided, even slight increases in effectiveness can have substantial and wide-reaching implications for public health.

Limitations

This study had several limitations. First, a considerable amount of data was unreported; in particular, it is likely that many interventions used persuasive design principles that were not described in the included studies. Second, although we were able to identify the principles in the PSD framework as present or absent, we did not have access to the interventions themselves, and we were unable to evaluate how effectively persuasive design principles were implemented. Third, we were unable to show, through our meta-regressions, whether specific persuasive design principles predicted efficacy. Finally, only 1 researcher was involved in data extraction; a second extractor would have helped reduce the risk of error, inconsistency, or bias.

Future Directions

Further research will be required to clarify the role of persuasive design in unguided ICBT and other eHealth interventions. First, dismantling studies comparing versions of interventions with and without certain persuasive design principles could evaluate the utility of specific principles. Factorial randomized trials of this kind would allow researchers to efficiently evaluate multiple persuasive design principles in a single study. Second, it would be helpful to explore how intervention users experience persuasive design, which could perhaps be achieved through qualitative research or the development of a self-report questionnaire assessing user experiences of persuasive design. Third, the literature would benefit from a more detailed description of persuasive design in unguided ICBT interventions based on a careful review of the interventions themselves (ie, rather than this study’s review of descriptions of interventions from randomized trials). Finally, further research will be required to test our finding that persuasive design predicts efficacy in unguided ICBT for depression but not for anxiety.

Conclusions

The literature on ICBT and other eHealth interventions is evolving rapidly. This review has provided an updated meta-analysis of unguided ICBT for depression and anxiety, generally finding smaller effect sizes for depression than for anxiety. It has also documented the wide variability in the use of persuasive design in unguided ICBT and demonstrated through a meta-regression that persuasive design predicts efficacy in unguided ICBT for depression. Persuasive design is a promising avenue for further optimization of eHealth interventions, including ICBT, and an area of research that is worth investigating further.

Acknowledgments

The authors would like to thank Dr Gordon Asmundson, Dr Donald Sharpe, Dr Swati Mehta, and Dr Janine Olthuis for providing valuable feedback on this research. This research was made possible by scholarship funding provided to HCM by the Social Sciences and Humanities Research Council of Canada. This research was also supported by PSPNET, which is led by HDH and funded by the Canadian Government’s Ministry of Public Safety and Emergency Preparedness. HDH holds funding from the Canadian Institutes of Health Research, Saskatchewan Health Research Foundation, Saskatchewan Centre for Patient-Oriented Research, and Craig Neilson Foundation. CRFS is currently employed through funding from AFA Försäkring.

Authors' Contributions

HCM and HDH formulated the idea for this study and developed the search terms. HCM conducted the literature search. HCM and CRFS conducted eligibility screening of the identified articles. HCM extracted and analyzed the data and wrote the first draft of the manuscript with support from HDH. All authors contributed to the revision of the manuscript.

Conflicts of Interest

None declared.

‎

Multimedia Appendix 1

The persuasive systems design framework.

DOCX File , 33 KB

‎

Multimedia Appendix 2

Recommendations for making eHealth interventions more engaging and related persuasive design principles.

DOCX File , 39 KB

‎

Multimedia Appendix 3

Log of revisions and clarifications to the original methodological protocol.

DOCX File , 34 KB

‎

Multimedia Appendix 4

Search terms.

DOCX File , 30 KB

‎

Multimedia Appendix 5

Data items.

DOCX File , 31 KB

‎

Multimedia Appendix 6

Flow of studies in the original literature search (October 29, 2019).

PNG File , 32 KB

‎

Multimedia Appendix 7

Flow of studies in the revised literature search (July 2, 2020).

PNG File , 31 KB

‎

Multimedia Appendix 8

Assumption tests for meta-regression of unguided internet-delivered cognitive behavioral therapy for depression.

DOCX File , 103 KB

‎

Multimedia Appendix 9

Assumption tests for meta-regression of unguided internet-delivered cognitive behavioral therapy for anxiety.

DOCX File , 94 KB

Depression and other common mental disorders: global health estimates. World Health Organization. 2017. URL: https://www.who.int/mental_health/management/depression/prevalence_global_health_estimates/en/ [accessed 2021-04-22]
Cuijpers P. Four decades of outcome research on psychotherapies for adult depression: an overview of a series of meta-analyses. Can Psychol / Psychologie Canadienne 2017;58(1):7-19. [CrossRef]
Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry 2013 Jul;58(7):376-385. [CrossRef] [Medline]
Otte C. Cognitive behavioral therapy in anxiety disorders: current state of the evidence. Dialogues Clin Neurosci 2012 Jan;13(4):413-421. [CrossRef]
Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med 2013 Aug 09;44(6):1303-1317. [CrossRef]
Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med 2011 Aug;41(8):1751-1761 [FREE Full text] [CrossRef] [Medline]
Andersson G, Carlbring P, Titov N, Lindefors N. Internet interventions for adults with anxiety and mood disorders: a narrative umbrella review of recent meta-analyses. Can J Psychiatry 2019 Jul 16;64(7):465-470 [FREE Full text] [CrossRef] [Medline]
Christensen H, Hickie IB. E-mental health: a new era in delivery of mental health services. Med J Aust 2010 Jun 07;192(S11):S2-S3. [CrossRef] [Medline]
Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, Bohman B, et al. Guided and unguided self-help for social anxiety disorder: randomised controlled trial. Br J Psychiatry 2009 Nov;195(5):440-447. [CrossRef] [Medline]
Richards D, Enrique A, Palacios J, Duffy D. Internet-delivered cognitive behaviour therapy. In: Cognitive Behavioral Therapy and Clinical Applications. Online: IntechOpen; 2017.
Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions — a systematic review. Internet Interv 2014 Oct;1(4):205-215. [CrossRef]
Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009 Dec;38(4):196-205. [CrossRef] [Medline]
Gerhards SA, de Graaf LE, Jacobs LE, Severens JL, Huibers MJ, Arntz A, et al. Economic evaluation of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry 2010 Apr;196(4):310-318 [FREE Full text] [CrossRef] [Medline]
Lorenzo-Luaces L, Johns E, Keefe JR. The generalizability of randomized controlled trials of self-guided internet-based cognitive behavioral therapy for depressive symptoms: systematic review and meta-regression analysis. J Med Internet Res 2018 Nov 09;20(11):e10113 [FREE Full text] [CrossRef] [Medline]
Andersson G, Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry 2014 Feb;13(1):4-11 [FREE Full text] [CrossRef] [Medline]
Fogg BJ. Persuasive technology. Ubiquity 2002 Dec 01;2002(December):2. [CrossRef]
Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst 2009;24:-. [CrossRef]
Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012 Mar;18(2):137-144. [CrossRef] [Medline]
Fleming TM, de Beurs D, Khazaal Y, Gaggioli A, Riva G, Botella C, et al. Maximizing the impact of e-therapy and serious gaming: time for a paradigm shift. Front Psychiatry 2016;7:65 [FREE Full text] [CrossRef] [Medline]
Helgadóttir FD, Menzies RG, Onslow M, Packman A, O'Brian S. Online CBT I: bridging the gap between eliza and modern online CBT treatment packages. Behav Change 2012 Feb 22;26(4):245-253. [CrossRef]
Schubart JR, Stuckey HL, Ganeshamoorthy A, Sciamanna CN. Chronic health conditions and internet behavioral interventions: a review of factors to enhance user engagement. Comput Inform Nurs 2011 Feb;29(2):81-92. [CrossRef] [Medline]
Ludden GD, van Rompay TJ, Kelders SM, van Gemert-Pijnen JE. How to increase reach and adherence of web-based interventions: a design research viewpoint. J Med Internet Res 2015 Jul 10;17(7):e172 [FREE Full text] [CrossRef] [Medline]
Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011 Jan;13(1):e2 [FREE Full text] [CrossRef] [Medline]
Neilsen AS, Wilson RL. Combining e-mental health intervention development with human computer interaction (HCI) design to enhance technology-facilitated recovery for people with depression and/or anxiety conditions: an integrative literature review. Int J Ment Health Nurs 2019 Feb 22;28(1):22-39. [CrossRef] [Medline]
Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012 Nov 14;14(6):e152 [FREE Full text] [CrossRef] [Medline]
Austin PC, Steyerberg EW. The number of subjects per variable required in linear regression analyses. J Clin Epidemiol 2015 Jun;68(6):627-636 [FREE Full text] [CrossRef] [Medline]
Cuijpers P, Weitz E, Cristea IA, Twisk J. Pre-post effect sizes should be avoided in meta-analyses. Epidemiol Psychiatr Sci 2017 Aug;26(4):364-368 [FREE Full text] [CrossRef] [Medline]
Hoyt WT, Del Re AC. Effect size calculation in meta-analyses of psychotherapy outcome research. Psychother Res 2018 May;28(3):379-388. [CrossRef] [Medline]
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009 Jul 21;6(7):e1000100 [FREE Full text] [CrossRef] [Medline]
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: Dsm-5. 5th Ed. Washington, D.C: American Psychiatric Association; 2013:1-970.
Hayes SC. Acceptance and commitment therapy, relational frame theory, and the third wave of behavioral and cognitive therapies. Behavior Therapy 2004;35(4):639-665. [CrossRef]
Hunot V, Moore T, Caldwell D, Furukawa T, Davies P, Jones H, et al. 'Third wave' cognitive and behavioural therapies versus other psychological therapies for depression. Cochrane Database Syst Rev 2013 Oct 18(10):CD008704. [CrossRef] [Medline]
Hofmann SG, Sawyer AT, Fang A. The empirical status of the "new wave" of cognitive behavioral therapy. Psychiatr Clin North Am 2010 Sep;33(3):701-710 [FREE Full text] [CrossRef] [Medline]
Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Br Med J 2011 Oct 18;343(oct18 2):5928 [FREE Full text] [CrossRef] [Medline]
Munder T, Barth J. Cochrane's risk of bias tool in the context of psychotherapy outcome research. Psychother Res 2018 May 09;28(3):347-355. [CrossRef] [Medline]
Comprehensive Meta-Analysis Software (CMA). URL: https://www.meta-analysis.com/ [accessed 2020-11-15]
Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010 Oct;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
Huedo-Medina TB, Sánchez-Meca J, Marín-Martínez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I² index? Psychol Methods 2006;11(2):193-206. [CrossRef]
Borenstein M, Hedges LV, Higgins JP, Rothstein H. Regression in Meta-Analysis. 2017. URL: https://www.meta-analysis.com/downloads/MRManual.pdf [accessed 2020-12-31]
Duval S, Tweedie R. A nonparametric “Trim and Fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 2000 Mar;95(449):89-98. [CrossRef]
Kampmann IL, Emmelkamp PM, Morina N. Meta-analysis of technology-assisted interventions for social anxiety disorder. J Anxiety Disord 2016 Dec;42:71-84. [CrossRef] [Medline]
Hair Jr JF, Black WC, Babin BJ, Anderson RE. Multivariate Data Analysis. 7. Ed. Harlow: Pearson; 2014:1-7.
Viechtbauer W, Cheung MW. Outlier and influence diagnostics for meta-analysis. Res Synth Methods 2010 Apr 04;1(2):112-125. [CrossRef] [Medline]
Berger T, Hämmerli K, Gubser N, Andersson G, Caspar F. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther 2011 Dec;40(4):251-266. [CrossRef] [Medline]
Bücker L, Schnakenberg P, Karyotaki E, Moritz S, Westermann S. Diminishing effects after recurrent use of self-guided internet-based interventions in depression: randomized controlled trial. J Med Internet Res 2019 Oct 02;21(10):e14240 [FREE Full text] [CrossRef] [Medline]
Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleher C, et al. Overcoming depression on the internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res 2002 Dec;4(3):E14 [FREE Full text] [CrossRef] [Medline]
Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005 Jun;7(2):e16 [FREE Full text] [CrossRef] [Medline]
Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009 May;38(4):222-234 [FREE Full text] [CrossRef] [Medline]
Dahne J, Collado A, Lejuez CW, Risco CM, Diaz VA, Coles L, et al. Pilot randomized controlled trial of a Spanish-language Behavioral Activation mobile app (¡Aptívate!) for the treatment of depressive symptoms among united states Latinx adults with limited English proficiency. J Affect Disord 2019 May 01;250:210-217. [CrossRef] [Medline]
Dahne J, Lejuez C, Diaz VA, Player MS, Kustanowitz J, Felton JW, et al. Pilot randomized trial of a self-help behavioral activation mobile app for utilization in primary care. Behav Ther 2019 Jul;50(4):817-827 [FREE Full text] [CrossRef] [Medline]
de Graaf LE, Gerhards SA, Arntz A, Riper H, Metsemakers JF, Evers SM, et al. Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry 2009 Jul;195(1):73-80 [FREE Full text] [CrossRef] [Medline]
Farrer L, Christensen H, Griffiths KM, Mackinnon A. Internet-based CBT for depression with and without telephone tracking in a national helpline: randomised controlled trial. PLoS One 2011 Nov;6(11):e28099 [FREE Full text] [CrossRef] [Medline]
Gräfe V, Moritz S, Greiner W. Correction to: health economic evaluation of an internet intervention for depression (deprexis), a randomized controlled trial. Health Econ Rev 2020 Jul 30;10(1):24 [FREE Full text] [CrossRef] [Medline]
Hur JW, Kim B, Park D, Choi S. A scenario-based cognitive behavioral therapy mobile app to reduce dysfunctional beliefs in individuals with depression: a randomized controlled trial. Telemed J E Health 2018 Dec;24(9):710-716. [CrossRef] [Medline]
Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin Psychol Psychother 2013 Sep 02;20(1):10-27. [CrossRef] [Medline]
Löbner M, Pabst A, Stein J, Dorow M, Matschinger H, Luppa M, et al. Computerized cognitive behavior therapy for patients with mild to moderately severe depression in primary care: A pragmatic cluster randomized controlled trial (@ktiv). J Affect Disord 2018 Oct 01;238:317-326. [CrossRef] [Medline]
Lüdtke T, Pult LK, Schröder J, Moritz S, Bücker L. A randomized controlled trial on a smartphone self-help application (Be Good to Yourself) to reduce depressive symptoms. Psychiatry Res 2018 Nov;269:753-762. [CrossRef] [Medline]
Lüdtke T, Westermann S, Pult LK, Schneider BC, Pfuhl G, Moritz S. Evaluation of a brief unguided psychological online intervention for depression: a controlled trial including exploratory moderator analyses. Internet Interv 2018 Sep;13:73-81. [CrossRef]
McDermott R, Dozois DJ. A randomized controlled trial of internet-delivered CBT and attention bias modification for early intervention of depression. J Exp Psychopathol 2019 Apr 18;10(2):204380871984250. [CrossRef]
Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res 2009 May 11;11(2):e15 [FREE Full text] [CrossRef] [Medline]
Meyer B, Bierbrodt J, Schröder J, Berger T, Beevers CG, Weiss M, et al. Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial. Internet Interv 2015 Mar;2(1):48-59. [CrossRef]
Mira A, Bretón-López J, García-Palacios A, Quero S, Baños RM, Botella C. An internet-based program for depressive symptoms using human and automated support: a randomized controlled trial. Neuropsychiatric Dis Treat 2017 Mar;Volume 13:987-1006. [CrossRef]
Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One 2013 Aug;8(8):e70086 [FREE Full text] [CrossRef] [Medline]
Montero-Marín J, Araya R, Pérez-Yus MC, Mayoral F, Gili M, Botella C, et al. An internet-based intervention for depression in primary care in spain: a randomized controlled trial. J Med Internet Res 2016;18(8):e231 [FREE Full text] [CrossRef] [Medline]
Moritz S, Schilling L, Hauschildt M, Schröder J, Treszl A. A randomized controlled trial of internet-based therapy in depression. Behav Res Ther 2012 Aug;50(7-8):513-521. [CrossRef] [Medline]
Morris RR, Schueller SM, Picard RW. Efficacy of a Web-based, crowdsourced peer-to-peer cognitive reappraisal platform for depression: randomized controlled trial. J Med Internet Res 2015 Mar;17(3):e72 [FREE Full text] [CrossRef] [Medline]
Noguchi R, Sekizawa Y, So M, Yamaguchi S, Shimizu E. Effects of five-minute internet-based cognitive behavioral therapy and simplified emotion-focused mindfulness on depressive symptoms: a randomized controlled trial. BMC Psychiatry 2017 Dec 04;17(1):85 [FREE Full text] [CrossRef] [Medline]
Schure MB, Lindow JC, Greist JH, Nakonezny PA, Bailey SJ, Bryan WL, et al. Use of a fully automated internet-based cognitive behavior therapy intervention in a community population of adults with depression symptoms: randomized controlled trial. J Med Internet Res 2019 Nov 18;21(11):e14754 [FREE Full text] [CrossRef] [Medline]
Silverstone PH, Rittenbach K, Suen VY, Moretzsohn A, Cribben I, Bercov M, et al. Depression outcomes in adults attending family practice were not improved by screening, stepped-care, or online CBT during a 12-week study when compared to controls in a randomized trial. Front Psychiatry 2017;8:32 [FREE Full text] [CrossRef] [Medline]
Spek V, Nyklícek I, Smits N, Cuijpers P, Riper H, Keyzer J, et al. Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychol Med 2007 Dec;37(12):1797-1806. [CrossRef] [Medline]
Bakker D, Kazantzis N, Rickwood D, Rickard N. A randomized controlled trial of three smartphone apps for enhancing public mental health. Behav Res Ther 2018 Oct;109:75-83. [CrossRef] [Medline]
Kleiboer A, Donker T, Seekles W, van SA, Riper H, Cuijpers P. A randomized controlled trial on the role of support in internet-based problem solving therapy for depression and anxiety. Behav Res Ther 2015 Sep;72:63-71. [CrossRef] [Medline]
Moberg C, Niles A, Beermann D. Guided self-help works: randomized waitlist controlled trial of Pacifica, a mobile app integrating cognitive behavioral therapy and mindfulness for stress, anxiety, and depression. J Med Internet Res 2019 Jun 08;21(6):e12556 [FREE Full text] [CrossRef] [Medline]
Powell J, Hamborg T, Stallard N, Burls A, McSorley J, Bennett K, et al. Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in the general population: randomized controlled trial. J Med Internet Res 2013;15(1):e2 [FREE Full text] [CrossRef] [Medline]
Proudfoot J, Clarke J, Birch M, Whitton AE, Parker G, Manicavasagar V, et al. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 2013;13:312 [FREE Full text] [CrossRef] [Medline]
Shirotsuki K, Nonaka Y, Abe K, Adachi S, Adachi S, Kuboki T, et al. The effect for Japanese workers of a self-help computerized cognitive behaviour therapy program with a supplement soft drink. Biopsychosoc Med 2017 Sep 19;11(1):23 [FREE Full text] [CrossRef] [Medline]
Twomey C, O'Reilly G, Byrne M, Bury M, White A, Kissane S, et al. A randomized controlled trial of the computerized CBT programme, MoodGYM, for public mental health service users waiting for interventions. Br J Clin Psychol 2014 Nov;53(4):433-450. [CrossRef] [Medline]
Berger T, Urech A, Krieger T, Stolz T, Schulz A, Vincent A, et al. Effects of a transdiagnostic unguided internet intervention ('velibra') for anxiety disorders in primary care: results of a randomized controlled trial. Psychol Med 2017 Jan;47(1):67-80. [CrossRef] [Medline]
Boettcher J, Magnusson K, Marklund A, Berglund E, Blomdahl R, Braun U, et al. Adding a smartphone app to internet-based self-help for social anxiety: a randomized controlled trial. Comput Hum Behav 2018 Oct;87(6):98-108. [CrossRef]
Botella C, Gallego MJ, Garcia-Palacios A, Guillen V, Baños RM, Quero S, et al. An internet-based self-help treatment for fear of public speaking: a controlled trial. Cyberpsychol Behav Soc Netw 2010 Aug;13(4):407-421. [CrossRef] [Medline]
Ciuca AM, Berger T, Crişan LG, Miclea M. Internet-based treatment for panic disorder: a three-arm randomized controlled trial comparing guided (via real-time video sessions) with unguided self-help treatment and a waitlist control. PAXPD study results. J Anxiety Disord 2018 May;56:43-55. [CrossRef] [Medline]
Donker T, Cornelisz I, van Klaveren C, van Straten A, Carlbring P, Cuijpers P, et al. Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: a randomized clinical trial. JAMA Psychiatry 2019 Jul 01;76(7):682-690 [FREE Full text] [CrossRef] [Medline]
Ivanova E, Lindner P, Ly KH, Dahlin M, Vernmark K, Andersson G, et al. Guided and unguided acceptance and commitment therapy for social anxiety disorder and/or panic disorder provided via the internet and a smartphone application: a randomized controlled trial. J Anxiety Disord 2016 Dec;44:27-35. [CrossRef] [Medline]
Kenardy J, McCafferty K, Rosa V. Internet-delivered indicated prevention for anxiety disorders: a randomized controlled trial. Behav Cognit Psychother 2003;31(3):279-289. [CrossRef]
Lin LY, ` K, Kishimoto T, Rodriguez M, Qian M, Yang Y, et al. An internet-based intervention for individuals with social anxiety and different levels of Taijin Kyofusho in China. J Cross Cult Psychol 2020 May 21;51(5):387-402. [CrossRef]
McCall HC, Richardson CG, Helgadottir FD, Chen FS. Evaluating a web-based social anxiety intervention among university students: randomized controlled trial. J Med Internet Res 2018 Dec 21;20(3):e91 [FREE Full text] [CrossRef] [Medline]
Oh J, Jang S, Kim H, Kim J. Efficacy of mobile app-based interactive cognitive behavioral therapy using a chatbot for panic disorder. Int J Med Inform 2020 Aug;140:104171. [CrossRef] [Medline]
Powell J, Williams V, Atherton H, Bennett K, Yang Y, Davoudianfar M, et al. Effectiveness and cost-effectiveness of a self-guided internet intervention for social anxiety symptoms in a general population sample: randomized controlled trial. J Med Internet Res 2020 Jan 10;22(1):e16804 [FREE Full text] [CrossRef] [Medline]
Titov N, Andrews G, Choi I, Schwencke G, Mahoney A. Shyness 3: randomized controlled trial of guided versus unguided internet-based CBT for social phobia. Aust N Z J Psychiatry 2008 Dec;42(12):1030-1040. [CrossRef] [Medline]
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Br Med J 2003 Sep 6;327(7414):557-560 [FREE Full text] [CrossRef] [Medline]
Higgins J, Thomas J. Cochrane Handbook for Systematic Reviews of Interventions. URL: https://training.cochrane.org/handbook [accessed 2021-01-06]
Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry 2017 Apr 01;74(4):351-359. [CrossRef] [Medline]
Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
Wright JH, Owen J, Richards D, Eells T, Richardson T, Brown G, et al. Computer-assisted cognitive-behavior therapy for depression: a systematic review and meta-analysis. J Clin Psychiatry 2019 Mar 19;80(2):19 [FREE Full text] [CrossRef] [Medline]
Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
Olthuis JV, Watt M, Bailey K, Hayden J, Stewart S. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev 2015 Mar 05(3):CD011565. [CrossRef] [Medline]
Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered treatment for generalized anxiety disorder: a systematic review and meta-analysis. Internet Interv 2015 Sep;2(3):272-282. [CrossRef]
Miloff A, Marklund A, Carlbring P. The challenger app for social anxiety disorder: new advances in mobile psychological treatment. Internet Interv 2015 Nov;2(4):382-391. [CrossRef]
Baker WL, White CM, Cappelleri J, Kluger J, Coleman C, Health Outcomes‚ Policy‚Economics (HOPE) Collaborative Group. Understanding heterogeneity in meta-analysis: the role of meta-regression. Int J Clin Pract 2009 Oct;63(10):1426-1434. [CrossRef] [Medline]
Andersson G, Titov N, Dear BF, Rozental A, Carlbring P. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry 2019 Jan 02;18(1):20-28. [CrossRef] [Medline]
Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, et al. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv 2018 Sep;13:108-115 [FREE Full text] [CrossRef] [Medline]

‎

CBT: cognitive behavioral therapy

ICBT: internet-delivered cognitive behavioral therapy

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PSD: persuasive systems design

Edited by G Eysenbach; submitted 23.01.21; peer-reviewed by A Pencer; comments to author 12.02.21; revised version received 14.02.21; accepted 11.04.21; published 29.04.21

©Hugh C McCall, Heather D Hadjistavropoulos, Christopher Richard Francis Sundström. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.04.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

This paper is in the following e-collection/theme issue:

Exploring the Role of Persuasive Design in Unguided Internet-Delivered Cognitive Behavioral Therapy for Depression and Anxiety Among Adults: Systematic Review, Meta-analysis, and Meta-regression