Preliminary Efficacy of a Digital Intervention for Adolescent Depression: Randomized Controlled Trial

Introduction

Background

Depression is a pervasive mental health disorder affecting approximately 4 million adolescents in the United States, and this number is expected to grow annually [1-4]. Depression can be lifelong, often emerging in adolescence and increasing the risk of poor physical health, social and relational problems, academic and employment difficulties, and reduced overall well-being [5-7]. Despite a growing need for effective treatment options, adolescents access [3,6] and adhere to [8,9] mental health treatment at alarmingly low rates owing to a number of factors. Stigma and a general difficulty in identifying symptoms of a mental illness inhibit adolescents’ willingness or awareness of seeking treatment [10,11]. When they do seek help, adolescents face months-long waitlists [12]. Other external forces drive the unmet clinical needs of adolescents experiencing depression, including costs [2,13], a widening divide in the accessibility of treatment [14,15], and an outstripped supply of mental health professionals [14,16]. Barriers in access to mental health treatment and depression incidence rates in adolescents have been exacerbated by the COVID-19 pandemic, necessitating action and new solutions [17-21].

Digital health programs, accessible through smartphones and computers, offer potential avenues for delivering treatment options for mental health conditions, such as depression [22-29]. However, it is essential to acknowledge their limitations, which include the absence of evidence-based protocols and clinical validation [24,25,27]. Digital therapeutics (DTx), which are evidence-based software programs designed to prevent, treat, or manage health conditions, serve to address these limitations [30]. Smartphone-based DTx are particularly well suited for adolescents given their technological literacy, ubiquitous cell phone use, and privacy preferences [22,29,31,32]. DTx are also easily scalable, can serve as adjunct to standard care, improve treatment fidelity [33-35], are cost-effective [36]. These technologies have been shown to be preliminarily effective in treating adolescents with a variety of mental health concerns including attention-deficit/hyperactivity disorder [37], sleep disorders, eating disorders, and anxiety [38]. Ultimately, DTx fill critical gaps in non-DTx digital treatments and can help patients receive effective, data-driven treatment with the added benefit of home access at any time [35,39,40].

Cognitive behavioral therapy (CBT) is a widely recognized and recommended treatment for depression and is considered the standard of care for adolescents [41-43]. Behavioral activation (BA), an important component of CBT [44-47], involves engaging in value-based behaviors that are rewarding or elicit a sense of mastery to help reduce depression symptoms using a combination of motivational strategies, reward seeking, and natural reinforcers, alongside reducing maladaptive and avoidant behaviors. BA has been shown to be an effective stand-alone treatment for adolescent depression [44-48]. Digital CBT applications have demonstrated promise in treating depression [23,26,49,50], and a BA-focused digital therapeutic may represent an effective and accessible treatment option for adolescents.

Limitations such as low user engagement and insufficient existing clinical validation (eg, randomized controlled trials [RCTs]) may limit the widespread adoption of DTx [24,25,27,49]. To our knowledge, there are no DTx for depression tailored specifically for adolescents, including BA-based DTx, and limited evidence exists regarding the efficacy of a smartphone-based DTx for adolescent depression [51-53].

Objectives

This study evaluates the efficacy of Spark, a 5-week self-guided mobile app intervention based on validated BA treatment protocols for adolescents with depression [44,45]. Adolescents aged 13 to 21 years were recruited nationwide into an internet-based RCT comparing Spark with an active control. Our main goal was to assess the preliminary efficacy of Spark in treating moderate to severe depression symptoms as an adjunct to usual care by measuring group differences in (1) changes in depression symptoms over time, including whether each group had a MCID in depression symptoms; (2) the proportion of participants who achieved remission; and (3) the proportion of participants who demonstrated treatment response. As this RCT was conducted between November 2020 and September 2021, another goal was to provide mental health resources to adolescents experiencing symptoms of depression during the COVID-19 pandemic.

Methods

Study Design

This open-label, partial crossover RCT compared Spark with an app with psychoeducational content (control app) in adolescents aged 13 to 21 years with self-identified depression symptoms. The study was conducted in 2 phases: an evaluation of feasibility (phase 1) and a preliminary evaluation of efficacy (phase 2). The results from phase 1 (Spark version 2.0) have been reported elsewhere [54]. Here, we focus on the results from phase 2, which combined data collected using Spark version 2.1 and version 2.2 (refer to the Spark section for more information on the versions). Updates to the Spark app to version 2.2 were deployed approximately halfway during the study; version 2.1 was used between September 2020 and March 2021, and version 2.2 was used between June 2021 and September 2021. The study design, randomization, control app, and outcome measures were the same across both versions. There was an a priori plan to combine the data from the 2 Spark versions for analysis.

Ethical Considerations

The study was approved by the Institutional Review Board of the Western Copernicus Group (20201686) under a nonsignificant risk investigational device exemption and overseen by an independent data and safety monitoring board. The trial was also registered on ClinicalTrials.gov (NCT04524598).

Participants

Participants were recruited nationwide via internet-based advertising and word of mouth. The following inclusion criteria were applied in phase 2: (1) aged 13 to 21 years, (2) self-reported symptoms of depression, (3) residing in the United States for the duration of the 5-week study, (4) under the care of a US-based primary care or licensed mental health care provider and willing and able to provide the name and contact information of the provider during consent appointment, (5) English fluency and literacy of adolescents and consenting legal guardians if aged <18 years, (6) access to a smartphone (iPhone 5s or later or running Android 4.4 KitKat or later) and regular internet access, (7) willingness to provide informed e-consent or assent and have legal guardian willing to provide informed e-consent if aged >18 years, and (8) stable for at least 2 months on any treatment (including medication or psychotherapy) for a mental health disorder.

Participants were excluded if they self-reported any of the following: (1) self-reported lifetime suicide attempt or active self-harm or active suicidal ideation with intent; (2) diagnosed by a clinician with bipolar disorder, substance use disorder, or any psychotic disorder including schizophrenia; (3) incapable of understanding or completing study procedures and digital intervention as determined by the participant, patient or legal guardian, health care provider, or clinical research team, and (4) previously participated in the user testing or clinical testing of the Spark app.

The legal guardians provided consent for participants aged <18 years. Adolescent participants provided consent unless they were under 18. If they were under 18, their legal guardians provided consent and they provided assent.

Procedure

During a virtual consent and enrollment session over video conferencing, participants and their legal guardians (if aged <18 y) were provided with study details, provided electronic informed consent or assent, were assessed for eligibility, and completed web-based baseline questionnaires. Participants downloaded an app on their smartphone and were provided with a safety plan template with instructions [55] to complete it on their own as a personal resource. Participants were randomized 1:1 to the Spark or control arm using a block randomization approach ranging in multiples of 2 from 6 to 12 [56].

All participants had access to their assigned app for the 5-week intervention period. During the intervention period, participants completed an in-app Patient Health Questionnaire-8 (PHQ-8) [57], a self-report questionnaire to assess symptoms of depression, and an internally developed symptom check questionnaire weekly, which they could complete anytime over a 7-day period. Legal guardians completed a weekly symptom check about their child on the web. Following the 5-week intervention period, app access was restricted, and participants and legal guardians completed web-based postintervention questionnaires. Participants randomized to the control arm were also offered access to the Spark app after the 5-week intervention period and completed postintervention questionnaires on the web again following this second intervention period (partial crossover data not reported here). Baseline and postintervention measures assessed participant characteristics, concurrent treatments, depression and anxiety symptoms, resilience, app feedback, and impacts of COVID-19 (refer to Table 1 for schedule and full descriptions) [57-63].

Interventions

Both the Spark and control programs were divided into 5 modules recommended to be completed at a pace of 1 module per week over the 5-week intervention period, but the participants were able to progress at their own pace. Content for a given week was not expected to take >60 minutes to complete. All participants were prompted to complete a weekly PHQ-8 and a clinical concerns questionnaire on the mobile app. Automated app notification reminders to complete these questionnaires were sent. If users had not opened the app in 3 days, an automated app notification encouraging participants to use the app was sent. Automated reminder notifications were sent 7 days before the end of the intervention period to remind participants that the intervention period would be ending in 7 days. Crisis resources were available for participants to access anytime in each app.

Spark (Versions 2.1 and 2.2)

The Spark app is based on CBT, which implements BA [44,45]. A character called “Limbot” is used as a therapeutic guide to encourage users to complete activities and model examples of activities for users. In the app, participants read text, answered questions, inputted text, and completed interactive activities. Participants were encouraged to schedule activities to be completed outside of the app and reflect on the impact on their mood. Tasks in the mobile app progress in a linear fashion, that is, each task must be completed to progress to the next task.

Version 2.1 of Spark included a 5-level program focused on providing psychoeducational content and delivering the BA model of depression by teaching 2 core skills (mood activity logging and activity scheduling). Version 2.2 was also divided into 5 levels and expanded on the content of version 2.1, adding problem-solving, mindfulness, and relapse prevention content. Version 2.2 also included a reward system and animations for completing certain activities, some design changes to the user interface, and an increased number of in-app notifications. The 5 levels of Spark version 2.1 were as follows:

• Level 1 (Start Your Journey): Program introduction and learning about the BA model of depression
• Level 2 (Making Choices): Mood tracking and up and down activities
• Level 3 (Solving Problems): Learn about activity scheduling and complete 3 activations,
• Level 4 (Staying Active): Complete 4 activations and
• Level 5 (Journey’s End): Complete 5 activations.

Version 2.2 expanded on version 2.1 with the addition of new features, UI elements, and content. The 5 levels of Spark version 2.2 were as follows:

• Level 1: Onboarding and Introduction to BA (no differences)
• Level 2: Mood tracking (no differences)
• Level 3: Mindfulness and Activity Scheduling (addition of 2 psychoeducational tasks teaching and reinforcing mindfulness skills)
• Level 4: Problem-Solving and Activity Scheduling (addition of 2 psychoeducational tasks teaching and reinforcing problem-solving skills), and
• Level 5: Relapse Prevention and Activity Scheduling (addition of 6 psychoeducational tasks teaching relapse prevention skills).

Psychoeducational Control

The control app contained 5 modules of age-appropriate psychoeducational content related to the neurobiology of depression and did not contain any active CBT or BA components. In this app, participants read the text on screen related to the brain and behavior, the adolescent brain, depression in the brain, neurobiological factors influencing depression, and personality. The same version of the control app was used for the entire study. The 5 levels of the control app included Lesson 1: Understanding Behavior, Lesson 2: Exploring the Brain, Lesson 3: Mastering Messengers, Lesson 4: Riding the Wave, and Lesson 5: People and Personality.

Safety Monitoring

A rigorous safety monitoring and classification procedure was followed. Potential safety-related events (ie, clinical concerns), which included worsening or persistently high depressive symptoms, any imminent risk events, suicidal ideation, hospitalizations, injury, illness, nonsuicidal self-injury, and direct or indirect indications of abuse, were identified in the following ways: (1) any concerning information provided to study staff during the onboarding session, in email, text, or phone correspondence during participation; (2) symptom deterioration based on weekly PHQ-8 scores was defined as (a) PHQ-8 score that was ≥15 and ≥5 points higher than the baseline score or (b) PHQ-8 scores ≥20 for 2 weeks in a row during the intervention, and (3) during the 5 week intervention period, all participants and their legal guardians (for those aged >18 y) were asked to complete a weekly internally developed symptom check questionnaire that asked them to report any negative symptoms or side effects that they experienced over the past week, to rate how negative each experience was on a scale of 0 (not at all) to 4 (extremely), and whether they believe each experience was caused by the app; (4) for participants randomized into the Spark arm, there were opportunities within the app that allowed for freeform text input. This freeform text was reviewed daily by study staff.

Data and incoming correspondence were reviewed daily and any identified potential clinical concerns were recorded by the study staff and verified by the study investigators. All logged clinical concerns were reviewed daily by a study investigator who determined whether the clinical concern required escalation to the study clinician (Dr Raph Rose), an independent licensed clinical psychologist not otherwise associated with the study or study sponsor, for clinical input or follow-up with the participant. The study clinician would then determine whether the participant was safe and eligible to continue with the study based on the information provided or based on contact with the participant or legal guardian. If a clinical concern related to suicidality was endorsed during the onboarding session, a study investigator administered the Ask Suicide-Screening Questions Toolkit [64] to the participant to determine the imminent risk level, recommend emergency resources if required, and escalate to the study clinician for follow-up.

Participants were withdrawn from the study if clinical concerns met the following criteria: (1) if the study clinician determined that the participant was no longer eligible to continue with the study, (2) if the clinician could not monitor safety because of not being able to reach the participant or other listed contacts, and (3) if the clinician could not monitor safety because participants did not complete the weekly symptom check questionnaire for 2 consecutive weeks. In any of these circumstances, the participant was informed, withdrawn from the study, and sent a list of mental health resources via email.

Safety Classification

Safety classification was based on the following definitions set forth by the US Food and Drug Administration (FDA) and FDA-recognized consensus standards, ISO 14155:2020 Clinical Investigation of Medical Devices for Human Subjects-Good Clinical Practice [65-67]. After study completion, a clinician who was not otherwise involved in the study (JF) reviewed all clinical concern data and provided preliminary event classifications. Any events deemed to be potential adverse events (AEs) by this clinician were sent to the study clinician (Dr Rose) for external classification. Dr Rose was blinded to identifying participant information, participant ID, and JF’s ratings to make final determinations. Information was presented to Dr Rose in a manner that concealed the group condition. Nonetheless, it is possible that Dr Rose could discern the group identification for certain classifications based on the content provided by a participant. To reduce the potential for bias in classification, if Dr Rose provided a stricter classification rating than JF, that rating was maintained.

An AE was defined as an “untoward medical occurrence, unintended disease or injury, or untoward clinical signs (including abnormal laboratory findings) in subjects, users or other persons, whether or not related to the investigational medical device [67] and whether anticipated or unanticipated.”

An adverse device effect (ADE) was defined as an AE “related to the use of an investigational medical device” [67]. This includes any AE “resulting from insufficiencies or inadequacies in the instructions for use, the deployment, the implantation, the installation, the operation, or any malfunction of the investigational medical device.” This also includes “any event that is a result of a user error or intentional misuse” [67]. For this study, ADEs could have occurred in either the Spark or control arms.

A serious AE (SAE) or serious ADE was defined as an AE or ADE that met more than one of the following criteria: Resulted in fatality, posed a life-threatening risk or immediate risk of death at the time of occurrence, led to persistent or significant disability or incapacity, required prolonged inpatient hospitalization, represented an important medical event, as determined by appropriate medical judgment, that could jeopardize the participant’s well-being, or where medical or surgical intervention might be necessary to prevent one of the aforementioned outcomes. This did not include planned hospitalization for a preexisting condition [65].

Unanticipated ADEs (UADEs), as defined in the FDA regulation 21 Code of Federal Regulation 812.3 [65], also referred to as “unanticipated problems,” included any serious adverse effect on health or safety or any life-threatening problem or death caused by, or associated with, a device, if that effect, problem, or death was not previously identified in nature, severity, or degree of incidence in the investigational plan or application; or any other unanticipated serious problem associated with a device that relates to the rights, safety, or welfare of participants.

Outcomes and Statistical Analysis

An a priori statistical analysis plan was restricted to participants with moderate to severe symptoms at baseline (PHQ-8≥10; moderate-to-severe cohort), unless otherwise specified. The α level was set to P=.05, and false discovery rate (FDR) correction for multiple comparisons was applied to the specified analyses.

Depression Symptoms: PHQ-8

The primary outcome was a group difference in the change in depression symptoms from baseline to 5 weeks, as measured by the PHQ-8. A modified intention-to-treat (mITT) approach was used for primary analyses by including all participants randomized within the moderate-to-severe cohort. A per protocol (PP) analysis included data from participants in this cohort who had completed all weekly PHQ-8 questionnaires. Post hoc mITT and PP analyses were also conducted for participants with a baseline PHQ-8≥5 (mild-to-severe cohort) to evaluate the efficacy of Spark in participants with mild to severe symptoms of depression. Statistical analyses were performed using R version 4.1.2 (R Foundation for Statistical Computing) by an independent external statistician (LC) [68].

Missing Data Analysis

Little’s test [69] was used to determine whether group differences existed in the proportion of missing PHQ-8 data across weeks for both moderate-to-severe and mild-to-severe cohorts. For any resulting significant results, the effects of known factors on missing data including Spark version, treatment group, week, baseline PHQ-8 severity, and age group were evaluated with follow-up χ² tests to determine whether data could be missing at random [70].

Multiple Imputation Procedure

Multiple imputation was implemented on missing data from participants who had completed at least the baseline PHQ-8 assessment. Information on Spark version (2.1 and 2.2), treatment group, baseline PHQ-8 score, age group, week, and individual PHQ-8 item was included to impute 100 data sets with the Pan [71] method using the R mice package version 3.14.0 [68,72,73]. Missing PHQ-8 item–level scores and assessment completion days from baseline were imputed.

Statistical Analysis

A linear mixed-effects model (LMM) analysis was implemented on an averaged imputed data set to evaluate the main effects of study arm (Spark and control) and week (0-5), and the study arm×week interaction using the R lme4 package (version 1.1.28 [74]). Four models were implemented in total, including the mITT and PP analyses for both the moderate-to-severe and mild-to-severe cohorts. In each model, group and week were entered as fixed factors. Spark version and assessment completion days from baseline were included as fixed factors to control for the effects of app version and differences in time between the completion of successive weekly assessments. PHQ-8 item–level scores and participants were included as random factors for the intercept, and the time from the baseline assessment was included as a random factor for the slope of participants. FDR correction was applied to the 4 P values for the study arm×week interaction effects. The effect size of the interaction (Cohen f²) was computed using pseudo R² (f²=R²/{1−R²}) [75,76] with the R MuMIn package (version 1.46.0 [77]). Dfs were estimated using the Satterthwaite method provided in the R lmerTest package (version 3.1-3) [78,79]. Follow-up analyses were performed to evaluate the effect of week within each group.

We also evaluated whether there was an average minimal clinically important difference (MCID) in symptom severity within each group, defined as a ≥5 point average decrease in PHQ-8 score between baseline and postintervention [80].

To assess the robustness of the study arm×week interaction, a generalized LMM (GLMM) with multiple imputation with the same model specification as the LMM analysis was implemented on item-level PHQ-8 data across the 100 imputed data sets using a 2-level Pan method [71], and the resulting 4 P values were FDR adjusted.

Group differences in remission rates, defined as PHQ-8 score <5 at postintervention [81], and treatment response rates, defined as a 50% reduction in PHQ-8 score between baseline and postintervention [50], were tested using χ² tests, and the resulting 4 P values were FDR adjusted.

Secondary Clinical Outcomes

Secondary outcomes included anxiety symptoms (Generalized Anxiety Disorder Scale [59]), legal guardian–rated depression symptoms of their child (Mood and Feelings Questionnaire 61]), and participant-rated and legal guardian–rated (of their child) global functioning (Patient-Reported Outcomes Measurement Information System [60]) assessed at baseline and postintervention. Means, SDs, and 95% CIs for the average difference between baseline and postintervention were computed for these outcomes. A questionnaire was developed internally to assess the positive and negative effects of COVID-19, and the proportion of participants in each group that endorsed each item was computed.

Safety

The total numbers of AE, ADE, SAEs, and UADE were computed per group for all randomized participants.

Other Outcomes

Other outcomes measured included program adherence, engagement, and acceptability. Program adherence was assessed as the proportion of participants completing all sessions by postintervention and percent completion per module. Engagement for both study arms was assessed using the User Engagement Scale-short form [63], minutes spent in the app per week, and total app sessions. Usability was assessed using the system usability scale [62]. App acceptability was assessed with internally developed self-reported and legal guardian–reported ratings of the app with a 10-point Likert scale, asking how much the app improved mood or symptoms of depression and how enjoyable it was. Means, SDs, and mean differences between study arms with 95% CIs were calculated for these measures.

Results

Participants

A total of 168 adolescents consented to participate in this study between November 2020 and June 2021. The postintervention and follow-up data were collected between January 2021 and September 2021, when planned data collection had completed. Eight participants did not meet the inclusion criteria. A total of 160 patients were randomized (80 in the Spark arm; refer to Figure 1 for the CONSORT (Consolidated Standards of Reporting Trials) diagram; refer to Table 2 for the characteristics). In total, 41 participants received Spark version 2.1 and 39 participants received Spark version 2.2. Data from participants with a baseline PHQ-8 score ≥10 (moderate-to-severe cohort; Table 2) were included in all planned analyses, including mITT (n=121) and PP (n=86). Post hoc analyses included data from participants with a baseline PHQ-8 score≥5 (mild-to-severe cohort; Table 2) for mITT (n=153) and PP (n=109). Three participants skipped a question on the PHQ-8 assessment and were included in the PP analyses. The Spark and control arms did not differ significantly on any of the baseline demographic characteristics (all P>.05).

‎

Treatment Outcomes: PHQ-8

Weekly means, SD, and 95% CIs for baseline to postintervention change scores per group; remission and treatment response rates per group are reported in Table 3.

Moderate-to-Severe Cohort

Modified Intention to Treat (n=121)

The LMM revealed a nonsignificant study arm x week interaction (t_127.66=−1.911; P=.06; FDR adjusted P=.06; f²=0.0012; power: 0.468-0.494; Figure 2A). There was a nonsignificant effect of study arm (t_177.54=.573; P=.57; f²=0.00034; power: 0.069-0.079) and a significant effect of week (t_1977.06=−4.395; P<.001; f²=0.00215; power: 0.985-0.993). There was a significant effect of week in the Spark arm (t_1759.08=−3.508; P<.001; change score=−5.08, 95% CI −6.72 to −3.42) and control arm (t_561.98=−3.244; P=.001; change score=−3.51, 95% CI −5.09 to −1.93). The GLMM also produced a nonsignificant study arm x week interaction (F_1,24186.41=3.223; P=.07; FDR adjusted P=.07).

At the end of the intervention period, the Spark arm showed significantly higher remission rates compared with the control arm (Spark, 11/63, 17% and control, 2/58, 3%; χ²₁=6.183; P=.01; FDR adjusted P=.03). Treatment response rates between the study arms (Spark, 15/63, 24% and control, 8/58, 14%) were not statistically significant (χ²₁=1.968; P=.07; FDR adjusted P=.16).

‎

Per Protocol (n=86)

The LMM revealed a significant study arm x week interaction (t_91.59=−2.546; P=.01; FDR adjusted P=.02; f²=0.0107; power: 0.699-0.699; Figure 2B). There was a nonsignificant effect of study arm (t_82.43=−0.072; P=.94; f²=.00031; power: 0.041-0.056) and a significant effect of week (t₁₅₁₁=−4.876; P<.001; f²=0.00425; power: 0.995-0.997). There was a significant effect of week in the Spark arm (t_1183.54=−4.242; P<.001; change score=−5.82, 95% CI −7.54 to −4.10) and the control arm (t_532.3=−3.482; P<.001; change score=−3.13, 95% CI −4.49 to −1.76). The GLMM also produced a significant study arm x week interaction (F_1,6.569e+08=6.624; P=.01; FDR adjusted P=.01).

Mild-Severe Cohort

Modified Intention to Treat (n=153)

The LMM revealed a significant study arm x week interaction (t_160.20=−3.054; P=.003; FDR adjusted P=.005; f²=0.00036; power: 0.846-0.884; Figure 2C). There was a nonsignificant effect of study arm (t_149.68=1.272; P=.21; f²=.00157; power: 0.233-0.275) and a significant effect of week (t_2539.94=−3.06; P=.002; f²=0.00058; power: 0.861-0.868). There was a significant effect of week in the Spark arm (t_1697.40=−3.32; P<.001; change score=−4.85, 95% CI −6.59 to −3.09) but not in the control arm (t_1177.40=−1.838; P=.07; change score=−2.86, 95% CI −4.43 to −1.30). The GLMM also produced a significant study arm x week interaction (F_1,19992.155=8.21; P=.004; FDR adjusted P=.008).

At the end of the intervention period, the Spark arm showed significantly higher remission (Spark, 20/74, 27% and control, 9/79, 11%; χ²₁=6.1; P=.01; FDR adjusted P=.03) and treatment response (Spark, 22/74, 30% and control,12/79, 15%; χ²₁=4.7; P=.03; FDR adjusted P=.04) rates compared with the control arm.

Per Protocol (n=109)

The LMM revealed a significant study arm x week interaction (t_115.09=−3.603; P<.001; FDR adjusted P=.002; f²=0.009498; power: 0.941-0.949; Figure 2D). There was a nonsignificant effect of study arm (t_105.69=0.544; P=.59; f²=.00006; power: 0.078-0.086) and a significant effect of week (t_1955.64=−3.6; P<.001; f²=0.00169; power: 0.936~0.939). There was a significant effect of week in the Spark arm (t_1178.37=−4.049; P<.001; change score=−5.57, 95% CI −7.04 to −4.09) and the control arm (t_1011.58=−2.231; P=.03; change scores=−2.39, 95% CI −3.60 to −1.19). The GLMM also produced a significant study arm x week interaction (F_1,1.026e+09=13.205; P<.001; FDR adjusted P<.001).

Notably, all analyses that showed an uncorrected α threshold of P<.05 remained significant following FDR correction, and study arm×week interaction effects were consistent across LMM and GLMM analyses.

Little’s test was significant for moderate-to-severe (χ²₃₄=72.5; P<.001) and mild-to-severe cohorts (χ²₃₉=106.94; P<.001), suggesting that data were not missing completely at random. Missing data patterns were associated with observed variables such as treatment group, week, and age group (Multimedia Appendix 1), suggesting that data are missing at random. No analysis showed a significant effect of Spark app version (significance P>.05).

Secondary Clinical Outcomes

Means, SDs, and 95% CIs of the difference between baseline and postintervention scores for all secondary participant-reported and legal guardian–reported measures are reported for each group in Table 4. The results for the effects of COVID-19 in the moderate-to-severe cohort are presented in Table 5. Notably, 49.2% (59/120) of the participants reported reduced access to mental health resources during the pandemic at baseline.

Safety Outcomes

There were no SAEs, ADEs, or UADEs in either group. There were a total of 2 AEs in the control arm and 4 in the Spark arm. Both AEs in the control arm were clinical deteriorations in PHQ-8 scores for 2 consecutive weeks. In the Spark arm, there was 1 clinical deterioration in the PHQ-8 score for 2 consecutive weeks, 2 instances of participant-reported suicidal ideation, and 1 instance of a legal guardian reporting that the participant was experiencing nonsuicidal self-injury ideation.

Adherence, Acceptability, and Engagement

In the Spark arm, 87% (55/63) of participants completed module 1, 79% (50/63) completed module 2, 64% (40/64) completed module 3, 48% (30/63) completed module 4, and 40% (25/63) completed module 5. App acceptability metrics are reported in Table 6. Engagement with the app as defined as the average number of sessions are reported in Table 7.

Discussion

Principal Findings

This RCT showed that, in planned and exploratory analyses, Spark was efficacious in treating mild to severe symptoms of depression, including a clinically meaningful reduction in depression symptoms and a significantly larger reduction in depression symptoms as well as higher remission and treatment response rates at postintervention compared with an active control. Spark did not negatively impact participant safety, as demonstrated by having no ADEs reported A mITT analysis, restricted to participants with moderate to severe symptoms at baseline, showed no group differences in symptom change from baseline to postintervention. The treatment response and remission rates for the Spark arm were consistent with those found in other digital CBT apps [50], and the Spark arm showed significantly higher remission rates compared with the control arm. A follow-up PP analysis restricted to participants who consistently engaged with their assigned app per the study protocol revealed that the Spark arm showed a significantly larger decrease in depression symptoms compared with the control arm. This finding supports the idea that engagement and adherence are important drivers of treatment outcomes [82,83]. Indeed, engagement with and adherence to Spark were also high, compared with other digital therapeutic studies, and higher than most popular wellness apps [82-86]. Participants also rated Spark as moderately enjoyable, usable, and successful in improving mood and depression symptoms [87]. Although out of scope for this study, future research should directly investigate engagement in clinical outcomes, especially in the context of specific therapeutic components of the intervention, such as mood logs and BAs, and how they may mediate symptom change.

Notably, exploratory analyses including a larger cohort of participants with mild to severe symptoms at baseline showed significant group differences in symptom reduction, remission rate, and treatment response rate. The Spark arm within this cohort also showed an MCID within a 95% CI. Statistical tests were robust to correction for multiple comparisons and suggest that Spark may be an effective treatment for a wide range of depression severity, including those with mild symptoms, consistent with a recent meta-analysis of digital CBT [25]. As mild symptoms are often overlooked and, when untreated, can worsen or have long-term consequences [88], self-guided DTx may be a particularly attractive treatment option for this severity group.

The results also suggest that Spark may be promising as an adjunct treatment and part of a stepped care model for depression treatment in adolescents [89]. Participants were able to continue their usual care while enrolled in the study, and more than three quarters (130/160, 81.3%) of the participants reported having been diagnosed with depression and about half were receiving some form of concurrent treatment for depression (medication, 85/160, 53.1% or psychotherapy, 72/160, 45%). Adjunct digital treatments can reduce the time and cost of standard care [84,90], and as a self-contained treatment option, Spark can be easily integrated into a wide range of treatment plans.

These results are also promising in light of issues with access to mental health care for adolescents, many of which were magnified during the COVID-19 pandemic along with an increase in prevalence rates [1,17-19,21]. Approximately 48.8% (59/121) of the participants identified reduced access to mental health resources during the pandemic, underscoring the treatment potential of self-guided evidence-based DTx, such as Spark, which can be made available immediately, widely, and equitably, when other forms of care are limited.

Limitations

The study was not statistically powered to detect group differences in PHQ-8 symptom change; therefore, these results should be replicated in a fully powered sample. Powered analyses are also warranted to directly investigate the impacts of engagement, baseline symptom severity, and concurrent treatments on outcomes. Although an active control was used, the open-label design of the study is a limitation, as participants were not blinded to treatment assignment and the primary outcome was a self-report PHQ-8; blinded studies with clinician-reported outcomes will provide stronger evidence of efficacy. The exploratory results including adolescents with mild to severe symptoms at baseline, although promising, should be interpreted with caution and replicated. Similarly, although our PP analysis indicates that engagement with Spark may drive symptom improvement, these observations may be owing to survivorship bias. The eligibility criteria for this study were intentionally designed to allow adolescents who were seeking resources during the pandemic to participate and did not require stability on concurrent treatment, an assessment for treatment resistance, or stratification of study arms based on concurrent treatment, which may have introduced noise into the data.

Generalizability

Spark was designed as a brief (5 weeks), adjunct to standard of care intervention to reduce symptoms of depression in adolescents (aged 13-21 years as defined by the FDA for medical devices). As such, the results should be interpreted for an adolescent sample; we would not anticipate these results to generalize to the broader adult and child population. We recruited participants directly (ie, through social media advertising rather than through a health care provider), which could have created a self-selection bias. To evaluate the generalizability of these results in a naturalistic setting, future studies should evaluate the efficacy of Spark as a treatment for depression in patients who are introduced to Spark by a health care provider. Future studies should evaluate long-term outcomes and future product developments (eg, evergreen features).

Conclusions

DTx such as Spark could help overcome known barriers to access to care [14,16,91,92] by providing an immediate treatment option and serve as an effective adjunct form of care. DTx can also serve to reduce stigma around mental health treatment, as interventions can be delivered privately, and can improve treatment adherence by providing an engaging experience. This study provides preliminary evidence that a brief, self-guided, CBT-based DTx can be used to effectively and safely treat mild to severe depression symptoms in adolescents.