This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297) Mobile Health (mhealth) (2939) mHealth for Wellness, Behavior Change and Prevention (2871) Depression and Mood Disorders; Suicide Prevention (1363) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (491) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1399) mHealth for Treatment Adherence (397) Games and Gamification for Health (650)

Published on in Vol 23, No 9 (2021): September

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25922, first published .
Facilitator Contact, Discussion Boards, and Virtual Badges as Adherence Enhancements to a Web-Based, Self-guided, Positive Psychological Intervention for Depression: Randomized Controlled Trial

Facilitator Contact, Discussion Boards, and Virtual Badges as Adherence Enhancements to a Web-Based, Self-guided, Positive Psychological Intervention for Depression: Randomized Controlled Trial

Facilitator Contact, Discussion Boards, and Virtual Badges as Adherence Enhancements to a Web-Based, Self-guided, Positive Psychological Intervention for Depression: Randomized Controlled Trial

Authors of this article:

Judith Tedlie Moskowitz1 Author Orcid Image ;   Elizabeth L Addington1 Author Orcid Image ;   Eva Shiu2 Author Orcid Image ;   Sarah M Bassett1 Author Orcid Image ;   Stephanie Schuette3 Author Orcid Image ;   Ian Kwok4 Author Orcid Image ;   Melanie E Freedman4 Author Orcid Image ;   Yan Leykin5 Author Orcid Image ;   Laura R Saslow6 Author Orcid Image ;   Michael A Cohn7 Author Orcid Image ;   Elaine O Cheung1 Author Orcid Image
Article Authors Cited by (15) Tweetations (3) Metrics

    Original Paper

    1Department of Medical Social Sciences, Osher Center for Integrative Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, United States

    2Department of Obstetrics and Gynecology, Biological Sciences Division, University of Chicago, Chicago, IL, United States

    3Department of Psychology and Neuroscience, Duke University, Durham, NC, United States

    4Department of Psychiatry and Behavioral Sciences, Osher Center for Integrative Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, United States

    5Department of Psychology, Palo Alto University, Palo Alto, CA, United States

    6Department of Health Behavior and Biological Sciences, University of Michigan School of Nursing, Ann Arbor, MI, United States

    7Osher Center for Integrative Medicine, University of California San Francisco, San Francisco, CA, United States

    Corresponding Author:

    Judith Tedlie Moskowitz, MPH, PhD

    Department of Medical Social Sciences

    Osher Center for Integrative Medicine

    Northwestern University Feinberg School of Medicine

    625 N Michigan Ave

    Suite 2700

    Chicago, IL, 60611

    United States

    Phone: 1 3125037712

    Email: judith.moskowitz@northwestern.edu


    Background: Adherence to self-guided interventions tends to be very low, especially in people with depression. Prior studies have demonstrated that enhancements may increase adherence, but little is known about the efficacy of various enhancements in comparison to, or in combination with, one another.

    Objective: The aim of our study is to test whether 3 enhancements—facilitator contact (FC), an online discussion board, and virtual badges (VB)—alone, or in combination, improve adherence to a self-guided, web-based intervention for depression. We also examined whether age, gender, race, ethnicity, comfort with technology, or baseline depression predicted adherence or moderated the effects that each enhancement had on adherence.

    Methods: Participants were recruited through web-based sources and, after completing at least 4 out of 7 daily emotion reports, were sequentially assigned to 1 of 9 conditions—the intervention alone; the intervention plus 1, 2, or all 3 enhancements; or an emotion reporting control condition. The intervention was a positive psychological program consisting of 8 skills that specifically targeted positive emotions, and it was delivered over 5 weeks in a self-guided, web-based format. We operationalized adherence as the number of skills accessed.

    Results: A total of 602 participants were enrolled in this study. Participants accessed, on average, 5.61 (SD 2.76) of 8 skills. The total number of enhancements participants received (0-3) did not predict the number of skills accessed. Participants who were assigned to the VB+FC condition accessed significantly more skills than those in the intervention only conditions. Furthermore, participants in arms that received the combination of both the VB and FC enhancements (VB+FC and VB+FC+online discussion board) accessed a greater number of skills relative to the number of skills accessed by participants who received either VB or FC without the other. Moderation analyses revealed that the receipt of VB (vs no VB) predicted higher adherence among participants with moderately severe depression at baseline.

    Conclusions: The results suggested that the VB+FC combination significantly increased the number of skills accessed in a self-guided, web-based intervention for elevated depression. We have provided suggestions for refinements to these enhancements, which may further improve adherence.

    Trial Registration: ClinicalTrials.gov NCT02861755; http://clinicaltrials.gov/ct2/show/NCT02861755

    J Med Internet Res 2021;23(9):e25922

    doi:10.2196/25922

    Keywords

    mHealth; adherence; depression; discussion board; gamification; positive psychological intervention; mobile phone 


    Despite the fact that up to 20% of people in the United States will experience depression over the course of their lives [1], more than half will remain undiagnosed and untreated [2] for a variety of reasons including treatment cost and geographic inaccessibility [3,4]. Web-based self-help or self-guided (eHealth) programs can overcome potential barriers associated with conventional modes of treatment and may be particularly useful for individuals who might not otherwise seek out conventional face-to-face therapy [5]. However, eHealth interventions for depression also come with significant challenges. Retention in these web-based, self-guided interventions is low compared with other types of trials [6]. For example, previous studies reported retention rates as low as 43% posttreatment, with even lower rates with longer-term follow-up [7]. A recent review of retention in studies of smartphone-delivered interventions for mental health concluded that researchers can expect attrition of up to one-third of participants who enroll in the study [8]. In addition, adherence to intervention content—operationalized as the number of log-ins, duration of web exposure, or number of modules or exercises completed—is as low as 50% for web-based programs [7], and complete adherence to all components of smartphone-delivered interventions for depression was achieved by an average of only 34% of participants [8]. Predictors of poor adherence in web-based depression trials include higher baseline levels of depression [7], male gender [9,10], and lower levels of education [9]. Older age is sometimes found to be associated with better adherence [9,10] and sometimes poorer adherence [7]. Those with less frequent internet use at baseline (weekly to monthly compared with daily) may be more adherent to self-guided programs [10].

    Social enhancements such as discussion boards that encourage users to generate and share content with each other can improve engagement [11-14]. Opportunities to earn points and badges can also increase motivation by providing virtual rewards to participants in the form of positive feedback [15-17]. More conventional enhancements, such as reminders via postcards or brief phone calls, can also increase participation [18,19], although other studies have found that similar enhancements such as personalization, interactivity, reminders, and text messages were not associated with better intervention adherence [20]. The discrepancy in these results suggests that specific automated enhancements, or combinations of automated and human-supported enhancements, may be necessary to maximize adherence to self-guided programs.

    In this study, we tested the following three enhancements to a self-guided web-based intervention for people experiencing depressive symptoms: brief facilitator contact (FC), an online discussion board (ODB), and virtual badges (VB). These features were developed in previous phases of the program [21,22] and were selected based on research suggesting that personal contact, interactive components, and incentivizing participation increased engagement in web-based programs [7,23-29].

    One enhancement was brief facilitator phone contact. Contact from study staff, a mental health professional, or another type of coach can provide a sense of obligation or accountability on the part of the participant and increase adherence to intervention content [29]. In eHealth interventions, this social presence, or sense that there is another human behind the intervention who is aware of the participant’s engagement and may provide reminders or prompts, significantly improves adherence [30,31]. FC may increase user engagement by building rapport and connection to the intervention, providing feedback, or supporting meaningful use by tailoring the content or application of the material to participants’ needs [32].

    The second enhancement we tested was an ODB. Asynchronous discussion boards where participants can reflect on their experiences of learning the skills presented in the intervention and interact with other participants may provide a sense of peer support that improves the efficacy of the intervention. Although web-based support groups alone have not been particularly effective [33], from a collaborative learning perspective, the addition of a forum that allows participants to interact asynchronously with other participants and intervention content may help facilitate understanding and retention of the content and increase engagement with the targeted behaviors [34].

    VB are a common form of gamification that increases engagement with an intervention [35] through the inclusion of playful, enjoyable, or competitive elements that boost adherence to a designated activity [36]. Although VB have been criticized for their emphasis on extrinsic motivation at the expense of intrinsic [35], there is evidence that gamified interventions are more engaging than those that do not contain such elements [37].

    We hypothesize that the enhancements would increase adherence, operationalized as the number of sessions accessed, and aim to explore whether the combination of enhancements had a significantly greater impact than any one alone. We also examine whether age, gender, race or ethnicity, comfort with technology, or baseline depression predict adherence or moderate the effects of each enhancement on adherence. Finally, we explore enjoyment ratings as predictors of use or engagement with each enhancement.


    Participants

    Detailed methods are described in the study by Cheung et al [22]. The study was approved by the institutional review board and preregistered through ClinicalTrials.gov (trial number: NCT02861755). All participants were recruited on the web through platforms such as ResearchMatch, Craigslist, and Reddit. Eligibility was determined using a web-based screener. To be eligible for the study, participants had to (1) have at least mild levels of depression, as indicated by a Patient Health Questionnaire-8 (PHQ-8) depression score of ≥5 [38]; (2) be aged ≥18 years; (3) have daily access to the internet; (4) own a mobile phone; (5) live in the United States; and (6) be able to read and write in English.

    Procedures

    Baseline and Randomization

    After obtaining web-based consent, participants completed a baseline questionnaire and a 7-day run-in period in which they were asked to complete brief daily assessments of emotion. If participants completed the baseline assessment and at least 4 of 7 days of the run-in period, they were allocated to one of the nine study arms. We stratified participant allocation to a condition based on gender and the level of depressive symptom severity (PHQ-8 score: 5-9 [mild]; 10-14 [moderate]; 15-19 [moderately severe]; >20 [severe]) to ensure sufficient numbers of each group within each condition, and participants were sequentially assigned to the study arm after stratification.

    The nine study arms varied according to the type and number of adherence enhancements. The nine conditions are as follows: (1) intervention alone, (2) intervention+FC, (3) intervention+ODB, (4) intervention+VB, (5) intervention+FC+ODB, (6) intervention+FC+VB, (7) intervention+ODB+VB, (8) intervention+FC+ODB+VB, and (9) the emotion reporting control condition.

    Enhancements

    In the FC conditions, the study staff called participants once per week to check their progress through the course. Specifically, following a prescribed set of questions, the staff member asked about the participant’s experience learning that week’s skills, using the home practice, and completing the daily emotion surveys; they probed for questions about the skills, accomplishments, barriers, and any difficulties with the technology. If the participant could not be reached by phone, the staff member sent an email with the same content in place of the call and encouraged the participant to reply if they had questions. Staff members were explicitly trained not to provide supportive counseling or skill training, but instead focused on discussing progress, challenges, and technology issues.

    Participants who were assigned to the ODB had access to a virtual discussion board where they could participate anonymously using pseudonyms. The study staff provided prompts to seed the discussion board and were notified when a participant posted something. Posts were reviewed for inappropriate content and signs of extreme distress or suicidality.

    In the VB condition, participants could earn flower badges that they could arrange in a garden plot. They received badges for completing activities such as creating a profile, reading a lesson, or logging in to the website for 7 consecutive days.

    MARIGOLD Intervention

    Conditions 1-8 received the MARIGOLD program, a positive psychological intervention that consisted of eight skills that specifically targeted positive emotions: noticing positive events, capitalizing, gratitude, behavioral activation, mindfulness, positive reappraisal, personal strengths, and acts of kindness. Additional details of the positive emotion skills intervention can be found elsewhere [21,22,39-43]. The skills were delivered over the course of 5 weeks in a self-guided, web-based format, and each week had associated home practice activities. Participants did not have access to all the skills at once; instead, 1 to 3 skills were released in each of the 5 weeks. Participants in conditions 1 to 8 also completed the same daily emotion reporting as the control condition (described in the following section).

    Emotion Reporting Control

    Participants in arm 9 were asked to log in to the website once per day for 49 days (the length of the other eight conditions) to report their emotions. Participants in the emotion reporting control arm completed approximately 20 of the 49 days of daily emotion reporting during the study period (mean 19.90, SD 18.01; range 0-50). They did not receive any of the intervention content and thus did not have adherence data as we have defined it; participants in the control condition will therefore not be included in the present analyses (we have included greater detail on the rates of retention for participants in the emotion reporting control arm and the comparison of daily emotion reporting in the emotion reporting control arm versus the other eight conditions in the supplemental analyses; Multimedia Appendix 1).

    Measures

    Adherence and Enhancement Ratings

    The primary adherence outcome was operationalized as the number of skills (out of eight) accessed (see supplemental analyses in Multimedia Appendix 1 for analyses with other operationalizations of adherence [number of pages viewed and home practice completed] and study retention. Predictors of alternative operationalizations of adherence and of retention were essentially the same). In addition, we explored the evaluations of the enhancements, measured both quantitatively and qualitatively. Upon completion of the 5-week MARIGOLD study, participants completed postcourse surveys in which they provided feedback about the intervention content and enhancements. Participants were asked to rate whether they enjoyed each enhancement they were assigned to receive (ie, FC, ODB, and VB) on a slider scale from 0 (definitely not) to 100 (definitely yes). Participants also asked open-ended questions regarding the benefits, drawbacks, and impact of each of the enhancements they received.

    Predictors of Adherence and Enhancement Ratings

    We also examined demographics, baseline depression, and comfort with technology as predictors of adherence and enjoyment of enhancements. Gender was assessed as male, female, or other, or prefer not to answer. Participants were asked whether they considered themselves Hispanic or Latino or Latina and how they self-identified their race (Black, White, Asian, Native American, Pacific Islander, and mixed or other). Age was calculated based on the date of birth. Depression was assessed using the PHQ-8 [38]. Finally, comfort with technology was assessed with six items that tapped a participant’s overall frequency of and confidence with technology use (eg, “I am confident I can navigate websites”) with response options ranging from 0 (not at all confident) to 10 (totally confident).

    Analyses

    Baseline Predictors of Adherence and Engagement With Enhancements

    We first conducted an overdispersed (or quasi-) Poisson regression [44,45] to examine whether the number of enhancements received predicted the number of skills accessed. We then conducted the analyses predicting the number of skills accessed from the intervention arm, with the intervention only condition as the reference category.

    We then conducted overdispersed Poisson regressions predicting the number of skills accessed from age, gender (1=female; male=0), race or ethnicity (with indicator variables to represent each race or ethnicity category), comfort with technology, and baseline depressive symptom severity (categorized as mild, moderate, moderately severe, and severe).

    To further examine whether adherence to the study differed as a function of the combination of enhancements received, we conducted an additional overdispersed Poisson regression predicting each adherence outcome as a function of whether the participant received each enhancement (FC, ODB, and VB), the FCODB, FCVB, and ODBVB two-way interactions and the FCODBVB three-way interaction. The two-way and three-way interactions allowed us to explore whether receiving a specific combination of enhancements had a significantly greater impact than receiving any one enhancement alone.

    Moderators of Adherence

    We examined whether the effect of each enhancement on adherence was moderated by the following characteristics: age, gender (female vs male), ethnicity (Hispanic vs non-Hispanic), race (Asian, Black or African American, White, other race [Pacific Islander, Native American, and mixed or other]), comfort with technology, and baseline depressive symptom severity (mild, moderate, moderately severe, and severe). To examine this, overdispersed Poisson regressions predict adherence by whether the participants received each enhancement, with dummy variables representing each enhancement, the moderator of interest, and the two-way interactions between the moderator and each enhancement type (FC×moderator, ODB×moderator, and VB×moderator). Owing to power limitations, we did not test all 4 moderators in the same model or explored interactions among the moderators and combinations of enhancements.

    Power Analysis

    A sensitivity power analysis using the G*Power 3.1.9.6 software [46] revealed that the sample size for this study (N=539 participants across the eight intervention arms) was sufficient to detect a small effect size with adequate power. The minimum effect size necessary for this study was w=0.12, assuming an α significance criterion of P=.05 and power=0.80.


    Screening, Enrollment, and Randomization

    Of the 1805 respondents who completed the screener, 1037 (57.45%) were deemed eligible and entered the run-in period. Of these, 58.05% (602/1037) completed at least 4 of 7 days of the emotion reports for the run-in and were sequentially assigned to one of the nine study conditions. Figure 1 shows the CONSORT (Consolidated Standards of Reporting Trials) diagram. There were 485 eligibility forms that were deemed spam. Most of these cases were identified because they enrolled in quick succession on the same day and had suspicious repeating patterns of first or last names. We sent additional screener questions to these cases and none completed them and therefore did not proceed to enrollment.

    ‎
    Figure 1. CONSORT (Consolidated Standards of Reporting Trials) diagram. FU: follow-up.
    View this figure

    Baseline Characteristics

    Across the nine conditions, there were no differences in baseline demographics, depression, or comfort with technology (Table 1). The average age of the participants was 38 years (range 18-80 years). Three-fourths of the sample were female (422/573, 73.6%), three-fourths were White (421/573, 73.4%), and 43% (244/568) were college graduates. Compared with those who did not complete at least 4 of the 7 days of the run-in period, participants who completed the run-in were older (mean 37.95, SD 13.68 vs mean 36.03, SD 12.77; P=.04), more likely to be female (467/633, 73.7% vs 210/323, 65%; P=.008), and had lower baseline depressive symptom severity (PHQ-8: mean 13.85, SD 5.18 vs mean 15.34, SD 5.11); P<.001; Table S1 in Multimedia Appendix 1).

    Table 1. Sample demographics and baseline variables by condition.
    Baseline variablesTotal sample (N=602)ConditionP valuea


    		Intervention alone (n=73)FCb (n=71)ODBc (n=69)VBd (n=67)FC+ODB (n=66)FC+VB (n=66)ODB+VB (n=64)FC+ODB +VB (n=63)Control (n=63)
    Age (years), mean (SD)37.8 (13.5)39.1 (13.4)37.8 (14.5)38.9 (14.1)38.4 (15.5)38.0 (12.6)38.4 (14.2)36.1 (12.1)38.6 (12.8)34.7 (12.1).67
    Gender, n (%).85

    		Male146 (26)18 (25)17 (25)18 (26)16 (25)16 (25)17 (27)15 (25)13 (25)16 (25)

    		Female422 (74)51 (72)48 (72)49 (72)49 (75)47 (75)46 (73)46 (75)39 (75)47 (75)
    Baseline PHQ-8e, n (%).99

    		Mild139 (23)17 (23)16 (23)16 (23)16 (24)16 (24)16 (24)14 (22)14 (22)14 (22)

    		Moderate193 (32)23 (32)23 (32)21 (30)21 (31)21 (32)21 (32)21 (33)21 (33)21 (33)

    		Moderately severe175 (29)21 (29)21 (30)21 (30)19 (28)19 (29)19 (29)19 (30)18 (29)18 (29)

    		Severe95 (16)12 (16)11 (16)11 (16)11 (16)10 (15)10 (15)10 (16)10 (16)10 (16)
    Ethnicity, n (%).89

    		Hispanic100 (17)13 (18)8 (12)11 (16)10 (15)13 (21)11 (18)12 (20)12 (23)10 (16)

    		Non-Hispanic473 (83)58 (82)59 (88)57 (84)55 (85)50 (79)52 (83)49 (80)40 (77)53 (84)
    Race, n (%)

    		Black93 (16)8 (11)7 (10)15 (22)9 (14)13 (21)12 (19)10 (16)9 (17)10 (16).62

    		White421 (74)(73)(73)(74)(79)(68)(73)(67)(79)(75).86

    		Asian49 (9)9 (13)11 (16)4 (6)4 (6)4 (6)6 (10)5 (8)3 (6)3 (5).26

    		Native American27 (5)2 (3)5 (8)4 (6)4 (6)3 (5)4 (6)2 (3)0 (0)3 (5).71

    		Pacific Islander2 (1)0 (0)0 (0)1 (2)0 (0)0 (0)1 (2)0 (0)0 (0)0 (0).56

    		Mixed or other43 (8)4 (6)5 (8)5 (7)7 (11)5 (8)2 (3)8 (13)1 (2)6 (10).38
    Education, n (%).33

    		<High school4 (1)0 (0)0 (0)1 (2)0 (0)2 (3)0 (0)1 (2)0 (0)0 (0)

    		High school40 (7)4 (6)2 (3)10 (15)6 (9)5 (8)7 (11)2 (3)2 (4)2 (3)

    		Some college170 (30)23 (32)21 (32)15 (22)23 (35)16 (26)18 (29)23 (38)16 (31)15 (25)

    		College graduate244 (43)23 (32)34 (52)32 (47)26 (40)27 (44)28 (44)21 (34)21 (40)32 (53)

    		Professional degree98 (17)18 (25)8 (12)9 (13)9 (14)12 (19)8 (13)12 (20)12 (23)10 (17)

    		Other12 (2)3 (4)1 (2)1 (2)1 (2)0 (0)2 (3)2 (3)1 (2)1 (2)
    Income (US $), n (%).24

    		<10,00048 (8)10 (14)4 (6)8 (12)3 (5)8 (13)6 (10)5 (8)2 (4)2 (3)

    		10,000 to 19,99965 (11.5)11 (16)6 (9)9 (13)11 (17)10 (16)4 (7)4 (7)4 (8)6 (10)

    		20,000 to 29,99992 (16)11 (15)10 (15)13 (19)12 (19)11 (18)8 (13)7 (12)10 (19)10 (15)

    		30,000 to 49,999102 (18)10 (14)15 (23)16 (24)13 (20)9 (15)10 (16)10 (16)10 (19)9 (15)

    		50,000 to 74,999105 (19)10 (14)13 (20)11 (16)10(15)7 (11)13 (21)9 (15)13 (25)19 (31)

    		75,000 to 99,99972 (13)10 (14)6 (9)5 (8)7 (11)7 (11)10 (16)17 (28)3 (6)7 (12)

    		100,000 to 199,99968 (12)7 (10)10 (15)4 (6)9 (14)7 (11)8 (13)9 (15)7 (14)7 (12)

    		>200,00014 (2)1 (1)2 (3)1 (2)0 (0)3 (5)3 (5)0 (0)3 (6)1 (2)
    Comfort with technology, mean (SD)9.2 (1.3)9.1 (1.6)9.3 (1.2)9.2 (1.2)9.1 (1.5)9.2 (1.2)9.0 (1.3)9.2 (1.3)9.1 (1.7)9.5 (0.7).57

    aP values were based on analysis of variance (continuous) and chi-square tests (categorical outcomes).

    bFC: facilitator contact.

    cODB: online discussion board.

    dVB: virtual badges.

    ePHQ-8: Patient Health Questionnaire-8.

    Adherence

    Participants accessed, on average, 5.61 (SD 2.76) of 8 skills. The total number of enhancements that participants received (0-3) did not predict the number of skills accessed (1 enhancement vs 0: P=.63; 2 enhancements vs 0: P=.25; 3 enhancements vs 0: P=.19). We conducted an analysis predicting the number of skills accessed as a function of whether the participant received each enhancement and their two- and three-way interactions. There were no significant main effects for the FC (P=.24) or ODB (P=.36) enhancements. There was a significant main effect of receiving the VB enhancement, such that participants in the conditions that received VB (VB, VB+FC, VB+ODB, and VB+FC+ODB) had a greater number of skills (mean 5.90, 95% CI 5.52-6.31) relative to participants in groups that did not receive the VB enhancement (intervention only, FC, ODB, and FC+ODB; mean 5.30; range 4.96-5.67; χ21=5.0; P=.03). This main effect, however, appeared to be driven by a significant VB by FC two-way interaction predicting the number of skills accessed (χ21=5.5; P=.02) such that participants in arms that received the combination of both the VB and FC enhancements together (VB+FC and VB+FC+ODB) accessed a greater number of skills (mean 6.42; range 5.87-7.03) than participants who received either enhancement without the other (VB without FC: mean 5.42; range 4.92-5.98; P=.01; FC without VB: mean 5.15; range 4.67-5.68; P=.001). Furthermore, participants who received either VB or FC enhancement without the other did not differ in the number of skills accessed compared with participants who received neither enhancement (neither VB or FC: mean 5.45; range 4.98-5.98; P=.40 to P=.93). No other two-way or three-way interactions emerged with statistical significance (FC by ODB interaction: P=.17; ODB by VB interaction: P=.44; and FC by ODB by VB interaction: P=.94). Taken together, these findings suggest that receiving the VB and FC enhancements in combination increased the number of skills accessed, but receiving either the VB or FC enhancement without the other did not (Figure 2).

    ‎
    Figure 2. Number of skills accessed (n=8) as a function of condition. Error bars denote 95% CIs. FC: facilitator contact; ODB: online discussion board; VB: virtual badges.
    View this figure

    The patterns of results were the same for other indicators of adherence (eg, pages viewed or home practice completed) and retention in the study. The details of these analyses are provided in Multimedia Appendix 1.

    Moderators of Adherence

    We examined age, gender, race, ethnicity, comfort with technology, and baseline depression as predictors of adherence and as moderators of the effects of each individual enhancement on adherence. As seen in Table 2, none of these baseline characteristics significantly predicted participants’ adherence to the intervention content (P=.19 to P=.87).

    Table 2. Overdispersed Poisson regression predicting adherence from baseline predictors.
    PredictorNumber of skills accessed (n=8)

    		Odds ratio (95% CI)P value
    Age (years)1.00 (0.99-1.00).79
    Gender

    		Female1.08 (0.96-1.21).20

    		Male (reference)—a—
    Ethnicity

    		Hispanic1.04 (0.91-1.19).60

    		Non-Hispanic (reference)——
    Race

    		Asian0.94 (0.78-1.13).50

    		Black or African American1.09 (0.96-1.23).19

    		Other raceb0.97 (0.82-1.15).70

    		White (reference)——
    Comfort with technology1.01 (0.97-1.04).78
    Baseline PHQ-8c

    		Moderate1.05 (0.92-1.19).47

    		Moderately severe0.99 (0.86-1.13).87

    		Severe1.04 (0.90-1.22).58

    		Mild (reference)——

    aNot available (indicates reference value).

    bOther race: the other race category constitutes those who identified as Pacific Islander, Native American, mixed, or other.

    cPHQ-8: Patient Health Questionnaire-8.

    Next, we explored whether the enhancements had a greater effect on adherence depending on demographics, comfort with technology, or baseline depression. There was a significant baseline depressive symptom severity by VB two-way interaction in predicting the number of skills accessed, χ23=8.6 (P=.04). Specifically, among participants with moderately severe depression (scores between 15 and 19 on the PHQ-8), receiving VB enhancement was associated with better adherence compared with those who did not receiving VB enhancement (B=1.09; P=.01). Two trends in the data that did not reach conventional levels of statistical significance are worth noting: for those in the mild (5-9) and moderate (10-14) range in the PHQ, there was also a positive association between the VB group and adherence (mild: B=1.00; P=.06; moderate: B=0.80; P=.10). At the highest level of depressive symptoms (scores >20); however, the association was in the opposite direction, with those who received the VB enhancement showing lower adherence compared with those who did not receive the VB enhancement (B=−1.21; P=.08). Given that the effects among participants in 3 of the 4 depression groups (mild, moderate, and severe but not moderately severe) did not reach statistical significance, caution is warranted in drawing conclusions based on these data.

    Exploratory Analyses on Engagement With Enhancements

    To better understand the responses to the three enhancements, we further explored the level of engagement and enjoyment ratings for each one.

    Analysis of FC

    For FC (n=266), study staff were able to contact 45.5% (121/266) of the participants in the FC enhancement conditions by phone at least once, and the average number of weeks that staff were able to reach participants by phone was 1 (SD 1.42; range 0-6). Facilitators were able to reach participants who were older, female, and more comfortable with technology and those who had baseline depression scores in the moderate and moderately severe range more frequently than those who were younger, male, less comfortable with technology, and had baseline depression scores in the mild range (P=.001 to P=.04; Table 3). The number of weeks that participants could be reached by phone was positively correlated with the number of skills accessed (Spearman r206=0.32; P<.001).

    Table 3. Overdispersed Poisson regressions predicting engagement with the virtual badges, facilitator contact, and ODBa enhancements from baseline predictors.
    PredictorNumber of virtual badgesFacilitator contact phone contactODB posts

    		ORb (95% CI)P valueOR (95% CI)P valueOR (95% CI)P value
    Age (years)1.01 (1.00-1.02).051.06 (1.03-1.08)<.0011.01 (0.99-1.02).29
    Gender

    		Female1.23 (0.88-1.71).244.08 (1.44-11.58).0080.61 (0.40-0.92).02

    		Male (reference)—c—————
    Ethnicity

    		Hispanic1.60 (1.10-2.32).011.52 (0.55-4.22).421.19 (0.67-2.13).55

    		Non-Hispanic——————
    Race

    		Asian0.84 (0.48-1.46).530.86 (0.16-4.68).861.21 (0.63-2.34).56

    		Black or African American1.01 (0.70-1.47).961.56 (0.66-3.70).311.05 (0.62-1.77).86

    		Other raced0.56 (0.33-0.95).060.44 (0.10-2.03).290.49 (0.21-1.14).10

    		White (reference)——————
    Comfort with technology1.05 (0.95-1.16).361.44 (1.02-2.06).041.00 (0.87-1.15).99
    Baseline PHQ-8e

    		Moderate1.25 (0.84-1.85).274.88 (1.37-17.34).010.77 (0.46-1.29).32

    		Moderately severe1.46 (0.99-2.17).065.34 (1.47-19.38).010.85 (0.51-1.43).54

    		Severe1.06 (0.65-1.72).813.10 (0.67-14.33).150.59 (0.29-1.19).14

    		Mild (reference)——————

    aODB: online discussion board.

    bOR: odds ratio.

    cNot available (indicates reference value).

    dOther race: the other race category constitutes those identified as Pacific Islander, Native American, mixed, or other.

    ePHQ-8: Patient Health Questionnaire-8.

    Participants rated the facilitator phone calls as moderately enjoyable (mean 47.20, SD 32.64; range 0-100), and the qualitative feedback supported a wide range of responses such as “I liked hearing from a real person, because then I knew someone was paying attention to what I was doing in the course” versus “It got irritating to answer the same questions and it was frustrating and embarrassing because I hadn't done the skills.” Enjoyment ratings of FC were significantly correlated with staff contact by phone (Spearman r107=0.27; P=.006) and the FC enjoyment ratings were positively associated with the number of skills accessed (Spearman r98=0.25; P=.01).

    Analysis of the ODB

    Among participants assigned to receive the ODB enhancement (n=262), 25.2% (66/262) posted on the discussion board at least once, and slightly less than half 48.5% (127/262) liked or commented on a post at least once. Male participants posted a greater number of times on the ODB than female participants (P=.02). The number of times that participants posted on the ODB was positively correlated with the number of skills accessed (Spearman r198=0.49; P<.001).

    Enjoyment ratings for the ODB were also in the midrange (mean 45.53, SD 30.69; range 0-100), although the ratings varied greatly. Some participants appreciated connecting with other participants through the discussion board: “It was motivational to see others progress and be able to communicate with others.” However, for others, the ODB fell short of expectations: “There doesn’t seem to be much activity there.” Still others were in the middle ground and neither liked or disliked it: “I was able to connect with other people in the program but it wasn’t very personal, in the sense that I would just comment a few blurbs here and there but didn’t feel like I was doing much.” Those who rated the ODB as more enjoyable had more posts on the board (Spearman r101=0.29; P=.003). However, participants’ ODB enjoyment ratings were not associated with the number of skills accessed in the program (Spearman r91=0.10; P=.34).

    Analysis of VB

    Among participants who were assigned to receive a VB enhancement (n=260), the number of badges earned ranged from 9 to 192, with a mean of 17.29 (SD 16.48). All participants assigned to receive a VB enhancement earned at least one badge. As seen in Table 3, older participants and Hispanic participants earned a greater number of VB than younger and non-Hispanic participants (P=.01 to P=.05). None of the other baseline characteristics significantly predicted the total number of VB (all P=.06 to P=.96).

    For enjoyment ratings, participants rated the VB as moderately enjoyable (mean 50.90, SD 33.17; range 0-100). However, there was wide variability in enjoyment ratings, and the qualitative feedback for the VB reflected this range: “It is a visual point system that is cute and fun. It seems silly but I felt like I accomplished something when I got each flower.” Others did not appreciate the whimsical garden plot calling it childish and a waste of time. Although the number of badges earned and enjoyment ratings of the VB enhancement were significantly correlated (Spearman r115=.30; P=.001), participants’ VB enjoyment ratings did not predict the number of skills accessed (Spearman r109=0.14; P=.16).


    Principal Findings

    Web-based, self-guided interventions hold significant promise for people with elevated depressive symptoms. Attaining acceptable levels of adherence to these programs is critical to effectiveness, yet this has proven to be a challenge. We developed and tested 3 enhancements that we hypothesized would, either alone or in combination, improve adherence to a self-guided positive psychological intervention for people with elevated depression. The enhancements were an ODB, VB, and FC. Participants in the eight conditions that received the intervention content also received 1, 2, or all 3 or no enhancements. Our results suggest that the combination of VB and FC is especially impactful and points to areas for future focus to improve adherence across the three enhancements.

    For the ODB enhancement, participants had access to an online forum where they could anonymously post questions, share their experiences, and offer encouragement to other participants. Previous findings regarding the efficacy of discussion boards for improving adherence to web-based programs have been mixed. For example, one study found that an internet discussion board or support group resulted in lower adherence to the self-guided program compared with the self-guided program alone [47], although participants who were assigned to receive the discussion board had a greater reduction in depression at 6 months.

    The engagement on the discussion board was quite low. Only a quarter of the participants posted on the board at least once, and qualitative feedback indicated that this low level of activity further discouraged participants from making use of the ODB. Therefore, it is perhaps not surprising that there was no indication that the ODB increased adherence to the program, either alone or in combination with other enhancements. Simple availability without engagement is not sufficient for an ODB to increase adherence to the intervention content. There are a number of ways that we could have increased engagement in the ODB, such as requiring participants to post as part of the home practice assignments or making it easier to navigate to the discussion board to lower the barriers to engagement.

    VB is a form of gamification and is intended to increase adherence to the intervention content by providing rewards for completing activities on the platform, for example, reaching a goal for the number of log-ins or consecutive homework completions. Participants in the arm that received both VB and FC completed significantly more sessions than did those in the no-enhancement arm (Figure 2). Furthermore, participants in arms that received VB and FC together (VB+FC; VB+FC+ODB) accessed, on average, one more skill (mean 6.42) compared with participants who received either VB or FC without the other (mean 5.42; FC alone, VB alone, ODB+VB, and ODB+FC). Participants were of a mixed opinion on whether the VB were enjoyable, with 33.9% (39/115) giving the VB low enjoyment ratings (below 31 on a 0-100 scale) and 33% (38/115) providing high enjoyment ratings (between 70 and 100). Previous studies document similar challenges to the successful use of gamification elements, such as a lack of clarity on which behaviors are being rewarded or confusion over how progress was being illustrated [48-50]. Future work to make the badges more interpretable, enjoyable, and clearly linked to target behaviors will help to increase the impact of the VB on adherence to the intervention.

    Our FC enhancement consisted of staff contacting participants by phone once per week for a brief check-in regarding any challenges they were experiencing while completing that week’s skills and answering questions or concerns they had. We included the FC component based on studies showing that contact with a person improves adherence to self-guided, web-based interventions [18,23,27,51,52]. Although receipt of the FC alone did not significantly improve adherence to the program, when combined with VB, there was an improvement such that those participants who received both FC and VB completed more of the intervention sessions compared with those who did not. The average number of times a facilitator was able to reach a participant was one, and fewer than half of the participants were ever reached by a facilitator by phone. The data indicate that the more participants enjoyed the FC, the more skills they completed, and as with VB, there was a wide range of enjoyment ratings for FC, leaving a lot of room for improvement. One easy change that may increase participants’ enjoyment of the FC condition, and as a result improve the impact of FC on adherence, is to vary the questions that are asked each week and word them so they do not come across as judgmental. Qualitative feedback indicated that participants found these questions repetitive and, especially if they had not completed the home practice that week, somewhat shaming. Furthermore, some participants may prefer email over phone contact, and providing an email contact option could increase the impact of the enhancement. Other ways to improve the impact of the FC condition could include improving the supportive accountability [29], such as setting clear expectations and goals for engagement with the platform, and tailored performance monitoring that is supportive rather than shaming. In addition, clarifying the expertise of the facilitators and increasing trust could enhance the impact of FC [29].

    Aside from the content of the enhancement, a simple count of the number of enhancements the participant was assigned to (up to 3) was not predictive of adherence. On the basis of our data, it is not necessarily better. However, the combination of VB and FC increased adherence compared with either enhancement alone. It may be that the FC helps encourage people to engage with the platform, with the phone call providing a reminder and the facilitator providing someone to be accountable. Once participants are on the platform, the VB reinforce that engagement and provide some extrinsic motivation to remain engaged [29].

    We explored potential moderators of the effect of each enhancement and found that depression influenced whether VB enhancement was associated with adherence. Specifically, among those with moderately severe baseline depression (PHQ-8 scores between 15 and 19), those who received the VB enhancement demonstrated better adherence than those who were not assigned to a VB condition. For those in the mild and moderate range of the PHQ-8 (scores between 5 and 14), the direction of the effect was the same but did not reach statistical significance. For participants with severe baseline depression (>20), the effect was in the opposite direction, such that VB were associated with (nonsignificantly) poorer adherence. Although this interaction should be interpreted with caution given the marginal significance and the very small number of participants in this group (those with the highest levels of depression who received the VB), this pattern was also apparent in our analyses with the proportion of intervention completed as an outcome (the number of pages viewed as a proportion of the total possible number of pages; Multimedia Appendix 1). It may be that for people with severe clinical depression, the badges were off-putting, and extra thought should be put into this form of gamification for people with the highest levels of depression. Of note, other studies of web-based depression interventions tend to exclude people who have scores in the severe range [10,20] so would not have picked up this potential demotivating effect of VB for people with the highest levels of depression.

    Limitations

    There are several limitations to consider in this study. First, the sample was recruited from all web-based sources and likely differed from clinic-based samples or those referred by medical providers. The sample was predominantly made up of non-Hispanic White individuals and females, which significantly limits the generalizability of the findings. Furthermore, participants needed to complete 4 of 7 daily emotion reports as part of a run-in to be assigned to a condition in which only 58.05% (602/1037) of the participants were able to accomplish. Although run-in periods are often used in clinical trials [53], by including only those who demonstrated adherence to the study protocol, we may have artificially enriched our sample with participants who were more likely to be adherent to the intervention. Comparison of those who completed the run-in to those who did not indicate that those who completed were older, more likely to be female, and less depressed than those who consented but did not complete the run-in. We also ran a large number of statistical tests and did not control for multiple comparisons. However, our analyses were hypothesis-driven and appropriate for this stage of intervention development. Finally, we tested the enhancements in the context of a positive psychological intervention, and the content may have influenced adherence, although we do not have a way to test this possibility in this study.

    These findings lead to several suggestions for future self-guided interventions and potential adherence enhancements. First, the program and enhancements were feasible even for people with very high levels of depression. Unlike most previous studies of web-based programs for people with depression, we included participants with the highest levels of depression. The findings demonstrate that people with severe levels of depression can also engage in programs like this, and their inclusion in the study sample suggested important differences in terms of response to enhancements. Second, the combination of VB and FC was associated with better adherence than either enhancement alone. Researchers should go beyond single enhancements to improve adherence and consider including combinations of enhancements that may be more effective than any one enhancement alone. Furthermore, it may be helpful to offer participants a choice of which enhancements they want to access. Providing participant control over which enhancements they receive from the outset could increase feelings of engagement and investment in the program. Finally, as the technology for intervention delivery progresses, there are new possibilities for better-engaging participants. Future work may consider more immersive approaches such as virtual reality [54] and approaches such as SMS text messages [55] or chatbots [56] that foster a sense of social connection and may encourage stronger engagement with the intervention content.

    Self-guided web-based interventions hold great promise to help people living with depression, and creative approaches to better engage participants will provide greater benefit to more people as the programs are more widely disseminated. Despite low levels of engagement with the FC enhancement, the results of our randomized trial suggest that the combination of VB and FC may be especially effective at improving adherence to the intervention, which in turn, may increase the impact of the intervention on well-being and bode well for future programs that incorporate enhancements such as these.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Supplemental analyses.

    DOCX File , 41 KB
    ‎
    Multimedia Appendix 2

    CONSORT-eHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 406 KB
    1. Kessler RC, Bromet EJ. The epidemiology of depression across cultures. Annu Rev Public Health 2013 Mar 18;34(1):119-138 [FREE Full text] [CrossRef] [Medline]
    2. Ghio L, Gotelli S, Marcenaro M, Amore M, Natta W. Duration of untreated illness and outcomes in unipolar depression: a systematic review and meta-analysis. J Affect Disord 2014 Jan;152-154:45-51. [CrossRef] [Medline]
    3. Brenes GA, Danhauer SC, Lyles MF, Hogan PE, Miller ME. Barriers to mental health treatment in rural older adults. Am J Geriatr Psychiatry 2015 Nov;23(11):1172-1178 [FREE Full text] [CrossRef] [Medline]
    4. Hollon SD, Muñoz RF, Barlow DH, Beardslee WR, Bell CC, Bernal G, et al. Psychosocial intervention development for the prevention and treatment of depression: promoting innovation and increasing access. Biolog Psychiat 2002 Sep;52(6):610-630. [CrossRef]
    5. Wallin E, Maathz P, Parling T, Hursti T. Self-stigma and the intention to seek psychological help online compared to face-to-face. J Clin Psychol 2018 Jul 08;74(7):1207-1218. [CrossRef] [Medline]
    6. Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    7. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009 Apr 24;11(2):e13 [FREE Full text] [CrossRef] [Medline]
    8. Linardon J, Fuller-Tyszkiewicz M. Attrition and adherence in smartphone-delivered interventions for mental health problems: A systematic and meta-analytic review. J Consult Clin Psychol 2020 Jan;88(1):1-13. [CrossRef] [Medline]
    9. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’ meta-analysis. Psychol. Med 2015 Apr 17;45(13):2717-2726. [CrossRef]
    10. Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, et al. The association between adherence and outcome in an Internet intervention for depression. J Affect Disord 2018 Mar 15;229:443-449. [CrossRef] [Medline]
    11. Horgan A, McCarthy G, Sweeney J. An evaluation of an online peer support forum for university students with depressive symptoms. Arch Psychiatr Nurs 2013 Apr;27(2):84-89. [CrossRef] [Medline]
    12. Griffiths KM, Calear AL, Banfield M. Systematic review on Internet Support Groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms? J Med Internet Res 2009 Sep 30;11(3):e40 [FREE Full text] [CrossRef] [Medline]
    13. Ellis L, Campbell A, O'Dea B. Comparative randomized trial of an online cognitive-behavioral therapy program and an online support group for depression and anxiety. J CyberTher Rehab 2011;4(4):461-467 [FREE Full text]
    14. Maher CA, Lewis LK, Ferrar K, Marshall S, De Bourdeaudhuij I, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res 2014 Feb 14;16(2):e40 [FREE Full text] [CrossRef] [Medline]
    15. Dias LP, Barbosa JL, Vianna HD. Gamification and serious games in depression care: A systematic mapping study. Telemat Informat 2018 Apr;35(1):213-224. [CrossRef]
    16. Deterding S, Sicart M, Nacke L, O'Hara K, Dixon D. Gamification. using game-design elements in non-gaming contexts. In: Proceddings of the Extended Abstracts on Human Factors in Computing Systems. 2011 Presented at: CHI '11: CHI Conference on Human Factors in Computing Systems; May 7 - 12, 2011; Vancouver BC Canada p. 2425-2428. [CrossRef]
    17. Brown M, O'Neill N, van Woerden H, Eslambolchilar P, Jones M, John A. Gamification and adherence to web-based mental health interventions: a systematic review. JMIR Ment Health 2016 Aug 24;3(3):e39 [FREE Full text] [CrossRef] [Medline]
    18. Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005 Jun 21;7(2):e16 [FREE Full text] [CrossRef] [Medline]
    19. Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med 2016 Dec 8;23(6):776-794. [CrossRef] [Medline]
    20. Kelders SM, Bohlmeijer ET, Pots WT, van Gemert-Pijnen JE. Comparing human and automated support for depression: Fractional factorial randomized controlled trial. Behav Res Ther 2015 Sep;72:72-80. [CrossRef] [Medline]
    21. Addington EL, Cheung EO, Bassett SM, Kwok I, Schuette SA, Shiu E, et al. The MARIGOLD study: Feasibility and enhancement of an online intervention to improve emotion regulation in people with elevated depressive symptoms. J Affect Disord 2019 Oct 01;257:352-364 [FREE Full text] [CrossRef] [Medline]
    22. Cheung E, Addington EL, Bassett SM, Schuette SA, Shiu EW, Cohn MA, et al. A self-paced, web-based, positive emotion skills intervention for reducing symptoms of depression: protocol for development and pilot testing of MARIGOLD. JMIR Res Protoc 2018 Jun 05;7(6):e10494 [FREE Full text] [CrossRef] [Medline]
    23. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009 Dec 15;38(4):196-205. [CrossRef] [Medline]
    24. Christensen H. Delivering interventions for depression by using the internet: randomised controlled trial. Br Med J 2004 Jan 31;328(7434):265. [CrossRef]
    25. Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. J Med Internet Res 2002 Feb 15;4(1):e3 [FREE Full text] [CrossRef] [Medline]
    26. Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009 May 13;38(4):222-234 [FREE Full text] [CrossRef] [Medline]
    27. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med 2003 Feb 14;33(2):217-227. [CrossRef] [Medline]
    28. van den Berg S, Shapiro DA, Bickerstaffe D, Cavanagh K. Computerized cognitive-behaviour therapy for anxiety and depression: a practical solution to the shortage of trained therapists. J Psychiatr Ment Health Nurs 2004 Oct;11(5):508-513. [CrossRef] [Medline]
    29. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res 2011 Mar 10;13(1):e30 [FREE Full text] [CrossRef] [Medline]
    30. Tate DF, Jackvony EH, Wing RR. A randomized trial comparing human e-mail counseling, computer-automated tailored counseling, and no counseling in an Internet weight loss program. Arch Intern Med 2006 Aug 14;166(15):1620-1625. [CrossRef] [Medline]
    31. Fry JP, Neff RA. Periodic prompts and reminders in health promotion and health behavior interventions: systematic review. J Med Internet Res 2009 May 14;11(2):e16 [FREE Full text] [CrossRef] [Medline]
    32. Jonathan GK, Pivaral L, Ben-Zeev D. Augmenting mHealth with human support: Notes from community care of people with serious mental illnesses. Psychiatr Rehabil J 2017 Sep;40(3):336-338 [FREE Full text] [CrossRef] [Medline]
    33. Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry 2013 Jul;35(4):332-338 [FREE Full text] [CrossRef] [Medline]
    34. Francescato D, Mebane M, Porcelli R, Attanasio C, Pulino M. Developing professional skills and social capital through computer supported collaborative learning in university contexts. Int J Hum Comp Stud 2007 Feb;65(2):140-152. [CrossRef]
    35. Cheng VW, Davenport T, Johnson D, Vella K, Hickie IB. Gamification in apps and technologies for improving mental health and well-being: systematic review. JMIR Ment Health 2019 Jun 26;6(6):e13717 [FREE Full text] [CrossRef] [Medline]
    36. Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform 2017 Jul;71:31-48 [FREE Full text] [CrossRef] [Medline]
    37. Kelders SM, Sommers-Spijkerman M, Goldberg J. Investigating the direct impact of a gamified versus nongamified well-being intervention: an exploratory experiment. J Med Internet Res 2018 Jul 26;20(7):e247 [FREE Full text] [CrossRef] [Medline]
    38. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009 Apr;114(1-3):163-173. [CrossRef] [Medline]
    39. Cohn MA, Pietrucha ME, Saslow LR, Hult JR, Moskowitz JT. An online positive affect skills intervention reduces depression in adults with type 2 diabetes. J Posit Psychol 2014 Jan 01;9(6):523-534 [FREE Full text] [CrossRef] [Medline]
    40. Dowling GA, Merrilees J, Mastick J, Chang VY, Hubbard E, Moskowitz JT. Life enhancing activities for family caregivers of people with frontotemporal dementia. Alzheimer Dis Assoc Disord 2014;28(2):175-181. [CrossRef] [Medline]
    41. Moskowitz J, Carrico A, Cohn M, Duncan L, Bussolari C, Layous K, et al. Randomized controlled trial of a positive affect intervention to reduce stress in people newly diagnosed with HIV; protocol and design for the IRISS study. Open Access J Clin Trials 2014 Sep;2014(6):85-100. [CrossRef]
    42. Moskowitz JT, Carrico AW, Duncan LG, Cohn MA, Cheung EO, Batchelder A, et al. Randomized controlled trial of a positive affect intervention for people newly diagnosed with HIV. J Consult Clin Psychol 2017 May;85(5):409-423 [FREE Full text] [CrossRef] [Medline]
    43. Moskowitz JT, Cheung EO, Snowberg KE, Verstaen A, Merrilees J, Salsman JM, et al. Randomized controlled trial of a facilitated online positive emotion regulation intervention for dementia caregivers. Health Psychol 2019 May;38(5):391-402 [FREE Full text] [CrossRef] [Medline]
    44. Coxe S, West SG, Aiken LS. The analysis of count data: a gentle introduction to poisson regression and its alternatives. J Pers Assess 2009 Mar 17;91(2):121-136. [CrossRef] [Medline]
    45. Gardner W, Mulvey EP, Shaw EC. Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychol Bull 1995 Nov;118(3):392-404. [CrossRef] [Medline]
    46. Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007 May;39(2):175-191. [CrossRef] [Medline]
    47. Griffiths KM, Mackinnon AJ, Crisp DA, Christensen H, Bennett K, Farrer L. The effectiveness of an online support group for members of the community with depression: a randomised controlled trial. PLoS One 2012 Dec 28;7(12):e53244 [FREE Full text] [CrossRef] [Medline]
    48. Zuckerman O, Gal-Oz A. Deconstructing gamification: evaluating the effectiveness of continuous measurement, virtual rewards, and social comparison for promoting physical activity. Pers Ubiquit Comput 2014 Jul 5;18(7):1705-1719. [CrossRef]
    49. Beltran A, O'Connor T, Hughes S, Baranowski J, Nicklas TA, Thompson D, et al. Alpha test of a videogame to increase children's vegetable consumption. Games Health J 2012 Jun;1(3):219-222 [FREE Full text] [CrossRef] [Medline]
    50. Rapp A. A qualitative investigation of gamification: Motivational factors in online gamified services and applications, in Gamification: Concepts, Methodologies, Tools, and Applications. In: Gamification: Concepts, Methodologies, Tools, and Applications. Pennsylvania, United States: IGI Global; 2015:32-48.
    51. Sander L, Paganini S, Lin J, Schlicker S, Ebert DD, Buntrock C, et al. Effectiveness and cost-effectiveness of a guided internet- and mobile-based intervention for the indicated prevention of major depression in patients with chronic back pain-study protocol of the PROD-BP multicenter pragmatic RCT. BMC Psychiatry 2017 Jan 21;17(1):36 [FREE Full text] [CrossRef] [Medline]
    52. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry 2017 Apr 01;74(4):351-359. [CrossRef] [Medline]
    53. Laursen DR, Paludan-Müller AS, Hróbjartsson A. Randomized clinical trials with run-in periods: frequency, characteristics and reporting. Clin Epidemiol 2019 Feb;Volume 11:169-184. [CrossRef]
    54. Maples-Keller JL, Bunnell BE, Kim S, Rothbaum BO. The Use of Virtual Reality Technology in the Treatment of Anxiety and Other Psychiatric Disorders. Harv Rev Psychiatry 2017;25(3):103-113. [CrossRef]
    55. Rathbone AL, Prescott J. The use of mobile apps and SMS messaging as physical and mental health interventions: systematic review. J Med Internet Res 2017 Aug 24;19(8):e295 [FREE Full text] [CrossRef] [Medline]
    56. Greer S, Ramo D, Chang Y, Fu M, Moskowitz J, Haritatos J. Use of the chatbot "Vivibot" to deliver positive psychology skills and promote well-being among young people after cancer treatment: randomized controlled feasibility trial. JMIR Mhealth Uhealth 2019 Oct 31;7(10):e15018 [FREE Full text] [CrossRef] [Medline]

    ‎
    CONSORT: Consolidated Standards of Reporting Trials
    FC: facilitator contact
    ODB: online discussion board
    PHQ-8: Patient Health Questionnaire-8
    VB: virtual badges

    Edited by R Kukafka; submitted 20.11.20; peer-reviewed by A Parks, S Schueller, M Nahum, A Bedendo; comments to author 26.01.21; revised version received 16.04.21; accepted 24.05.21; published 22.09.21

    Copyright

    ©Judith Tedlie Moskowitz, Elizabeth L Addington, Eva Shiu, Sarah M Bassett, Stephanie Schuette, Ian Kwok, Melanie E Freedman, Yan Leykin, Laura R Saslow, Michael A Cohn, Elaine O Cheung. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 22.09.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.