Review
Abstract
Background: Digital mental health interventions (DMHIs) are increasingly recognized as potential solutions for adolescent mental health, particularly in low- and middle-income countries (LMICs). The United Nations’ Sustainable Development Goals and universal health coverage are instrumental tools for achieving mental health for all. Within this context, understanding the design, evaluation, as well as the barriers and facilitators impacting adolescent engagement with mental health care through DMHIs is essential.
Objective: This scoping review aims to provide insights into the current landscape of DMHIs for adolescents in LMICs.
Methods: The Joanna Briggs Institute scoping review methodology was used, following the recommendations of the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews). Our search strategy incorporated 3 key concepts: population "adolescents," concept "digital mental health interventions," and context "LMICs." We adapted this strategy for various databases, including ACM Digital Library, APA PsycINFO, Cochrane Library, Google Scholar (including gray literature), IEEE Xplore, ProQuest, PubMed (NLM), ScienceDirect, Scopus, and Web of Science. The articles were screened against a specific eligibility criterion from January 2019 to March 2024.
Results: We analyzed 20 papers focusing on DMHIs for various mental health conditions among adolescents, such as depression, well-being, anxiety, stigma, self-harm, and suicide ideation. These interventions were delivered in diverse formats, including group delivery and self-guided interventions, with support from mental health professionals or involving lay professionals. The study designs and evaluation encompassed a range of methodologies, including randomized controlled trials, mixed methods studies, and feasibility studies.
Conclusions: While there have been notable advancements in DMHIs for adolescents in LMICs, the research base remains limited. Significant knowledge gaps persist regarding the long-term clinical benefits, the maturity and readiness of LMIC digital infrastructure, cultural appropriateness, and cost-effectiveness across the heterogeneous LMIC settings. Addressing these gaps necessitates large-scale, co-designed, and culturally sensitive DMHI trials. Future work should address this.
doi:10.2196/51376
Keywords
Introduction
Background
The World Health Organization (WHO) defines mental health as “a state of mental well-being that enables people to cope with the stresses of life, to realize their abilities, to learn well and work well, and to contribute to their communities. Mental health is an integral component of health and well-being and is more than the absence of mental disorder” []. The United Nations General Assembly has underscored the key role of universal health coverage (UHC)—where everyone can access the health services they need without financial hardship—in achieving health for all []. Indeed, UHC is central to the health-related Sustainable Development Goals (SDGs), namely target 3.4: “by 2030, reduce by one-third premature mortality from noncommunicable diseases through prevention and treatment and promote mental health and well-being” []. While these global agendas are instruments for achieving mental health for all, there is international recognition that adolescent mental health is a distinct and critical public health and human rights issue [-].
In 2020, the WHO–UNICEF (United Nations International Children’s Emergency Fund)–Lancet Commission called for a renewed focus on the SDGs for advancing child and adolescent health, including mental health []. The Commission made a powerful argument that children and adolescents should be at the center of the SDGs to protect their human rights. In addition, this was an ethical and economic investment []. Despite this, the World Mental Health Report states that approximately 90 WHO member states—less than half of all member states—had a mental health policy or plan specifically for children and adolescents []. The report emphasizes the importance of national mental health plans that are fully compliant with human rights instruments, sufficiently resourced, and regularly monitored and evaluated [].
The onset of mental health disorders typically occurs during adolescence (between the ages of 10-19 years), an important time for developing social and emotional skills [,]. It is here where adolescents form coping strategies that enable mental health. Equally, it is a time when young people become vulnerable to risk-taking behaviors, for example, substance abuse. Indeed, suicide is the leading cause of adolescent death []. Worldwide, 13% of the world’s adolescents (aged 10-19 years) live with a mental disorder []; however, widening treatment gaps mean many conditions remain undiagnosed and untreated []. Moreover, mental health conditions disproportionately affect adolescents in low- and middle-income countries (LMICs) []. LMIC economies are those in which the 2022 gross national income per capita was less than US $13,845 []. Approximately 90% of the world’s 1.2 billion adolescents reside in LMICs []. This population is more vulnerable to human rights violations, and where limited mental health services are available, stigma, discrimination, and social, cultural, and economic challenges are major barriers to treatment access [,,].
The COVID-19 pandemic has worsened the significant burden on this population [,,]. Before the pandemic, researchers explored the feasibility and clinical benefits of digital health technologies for adolescent mental health and care in LMICs [-]. Indeed, COVID-19 has driven the adoption of digital health care platforms in LMICs, including digital interventions for health [,]. Digital health interventions (DHIs) are increasingly accepted tools for the support of adolescent mental health [-].
The WHO defines DHIs as a discrete functionality of digital technology that is applied to achieve health objectives []. In this regard, digital mental health interventions (DMHIs) can be understood as a discrete functionality of digital technology that is applied to achieve mental health objectives. DHIs are classified into 4 overarching groups based on the primary user, for instance, clients (service users), health care providers (health service delivery), health system managers (administration and oversight of public health systems), and data services (data collection, management, use, and exchange) []. For clients, DHIs/DMHIs can be delivered at an individual or population level; in high and low-income settings; and via several devices, for example, mobile apps, websites, wearables, and smart devices. Such interventions may be self-guided [] or delivered with lay support, for instance, from teachers [] or trained psychologists []. Furthermore, some DMHIs include theories of behavior change, such as the self-determination theory [] or the Social Cognitive Theory []. Indeed, research suggests that interventions grounded in behavioral theory are more likely to be effective [,].
Current Barriers to Existing Mental Health Services in LMICs
Presently, there are significant and systemic barriers that limit adolescent access to mental health services, for example, cost, time, geographic location [], stigma [,], lack of community knowledge and education on mental health [], and low mental health literacy []. Mental health systems in LMICs lack trained mental health specialists, leading to significantly understaffed services [,,]. These shortages are unequal, with urban areas benefiting from more health specialists and resources than rural areas [,]. Consequently, this is a potential barrier to treatment because the physical distance may place many services out of reach, thereby reinforcing the rural-urban digital divide []. The term digital divide can be understood as “the gap between people who do and do not have access to forms of information and communication technology” []. The concept is complex, going beyond the connected versus the unconnected. It is a global phenomenon but even more pronounced in LMIC settings, leading to further social, cultural, and economic inequalities [,].
Key Challenges With Adolescent DMHIs in LMICs
Adolescents in LMICs have a distinct disadvantage to digital infrastructure, namely the internet []. In 2019, the rural-urban gap in mobile internet use across LMICs was 37%; however, LMICs in sub-Saharan Africa had the widest rural-urban gap, with those living in rural areas being 60% less likely to access the internet than those residing in urban areas []. Another report found gaps in literacy and related skills had the greatest impact on mobile internet use []. Moreover, the GSM (Global System for Mobile) Communications reports that mobile phone affordability and lack of digital skills were significant barriers in LMICs [].
There are issues with DMHI implementation, posing a barrier to scalability. Many DMHIs, for example, require additional guidance for adolescent adherence and engagement, such as psychologists, therapists, or lay teacher support [,]. This may undermine the scalability of the interventions due to the shortage of health professionals for those roles []. Other challenges relate to their feasibility [], political nature [], cultural appropriateness [], design, deployment, evaluation [], sustainability [], cost-effectiveness [], privacy and data security [], digital maturity, and readiness [-]. Despite this, there is evidence that DMHIs could transform adolescent mental health in low-resource settings [,-].
Rationale and Review Question
We selected the review period 2019-2024 for 3 distinct reasons; first, in 2020, the WHO launched its first guidance on designing DHIs with and for young people, recognizing the significance of youth-centered DHIs and considering their specific needs []. Second, that same year, the WHO–UNICEF–Lancet Commission called for a renewed focus on the SDGs for advancing child and adolescent health, including mental health, prioritizing young people in the urgent call to action []. Finally, the COVID-19 pandemic has profoundly impacted adolescent mental health worldwide, leading to an increased interest and investment in the delivery of quality, person-centered, remote mental health care []. The post–COVID-19 era has provided an opportunity to explore which DMHIs exist for adolescents in LMICs, how they are designed and evaluated, how adolescents are involved in design activities, which theories or models were consulted, and which factors affect adolescent engagement. Examining the current state of adolescent DMHIs in the context of these recent developments allows for a timely and up-to-date insight into gaps in policy, practice, and research. This scoping review aims to collate information on adolescent DMHIs in LIMC settings. The review asks “What is known about DMHIs for adolescents in LMICs, as reported in the literature?”
Subreview Questions
The subreview questions (subRQs) were as follows:
- SubRQ1: How are adolescent DMHIs designed and evaluated within LMICs?
- SubRQ2: What are the reported activities and approaches involving adolescents in designing DMHIs, and what are the perceived outcomes of such involvement?
- SubRQ3: Which frameworks, toolkits, models, or theories were consulted or applied to the DMHI? (including implementation activities related to digital health readiness and preparedness)
- SubRQ4: What factors facilitate or hinder adolescent engagement in DMHIs in LMICs?
Methods
Overview
This scoping review followed the Joanna Briggs Institute Scoping Review Methodology [] and is reported based on the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines []. The PRISMA-ScR checklist is shown in . Scoping reviews are ideal for disciplines with emerging evidence, for example, DMHIs for adolescents in LMICs, where the limited availability of randomized controlled trials (RCTs) hinders researchers from performing systematic reviews and assessing the quality of evidence []. Consistent with the purpose of scoping studies, this review does not seek to assess or appraise the quality and robustness of the evidence, nor does it generalize the findings [,,]. By systematically mapping the available literature, a scoping review can highlight the current state of knowledge, identify research gaps, and potentially reveal patterns or trends that can inform future studies []. Previous researchers have used scoping reviews to map available evidence in a similar field [-].
Eligibility Criteria
The Joanna Briggs Institute recommends the use of the PCC mnemonic, that is, population, concept, and context, to develop the inclusion criteria []. The population consists of adolescents aged 10-19 years, which is consistent with the WHO definition []. The concept under study is DMHIs, understood as a discrete functionality of digital technology that is applied to achieve mental health objectives []. The context of interest is LMIC economies, defined by the World Bank as nations with a gross national income per capita less than US $13,845 [], see 
Textbox 1.
Inclusion criteria
- Population: Adolescents (World Health Organization [WHO] definition []—persons between the ages of 10 and 19 years).
- Concept: Digital mental health interventions (DMHIs) [] that are delivered to support the mental health and well-being of adolescents in low- and middle-income countries (LMICs). DMHIs focus on emotional, behavioral, or eating disorders, psychosis, suicide, self-harm, and risk-taking behaviors. Including mental health promotion and prevention interventions and those for early detection and treatment. DMHIs include mobile apps, web applications, smart devices, telehealth, SMS text messaging, and internet-based interventions.
- Context: Low- and middle-income settings [].
- Sources: Published, peer-reviewed, or gray literature of any research study design (eg, randomized controlled trials, systematic reviews, case studies, qualitative, quantitative, or mixed methods, theses or dissertations, white papers, guidelines, conference proceedings, charity reports, or posters).
- Period: Published between January 2019 and March 2024.
- Language: Studies published in English.
Exclusion criteria
- Population: Participants that do not meet the WHO definition of adolescents, that is, are aged younger than 10 years or older than 19 years.
- Concept: DMHIs that target or only explore the concerns and views of adolescents’ parents, caregivers, guardians, teachers, and clinicians. Studies that do not focus on digital mental health or well-being.
- Context: Not based in low- and middle-income settings.
- Sources: No full text is available or the paper is not retrievable.
- Period: Published before January 2019.
- Language: Not available in English.
Information Sources
The information sources were the following electronic databases: (1) ACM Digital Library, (2) APA PsycINFO, (3) Cochrane Library, (4) Google Scholar (gray literature included, ), (5) IEEE Xplore, (6) ProQuest, (7) PubMed (NLM), (8) ScienceDirect, (9) Scopus, and (10) Web of Science.
These databases were selected because they collectively align with this review’s scope on DHIs and technologies, that is, behavioral science, mental health (psychiatry and psychology), and technology.
Search Strategy
The database identification and search strategy were developed with guidance from a faculty librarian at the University of Strathclyde. The search terms represented the primary concepts of the objectives in the review. These included a range of keywords, free text, and medical subject headings terms and combinations of the Boolean operators. The Joanna Briggs Institute guidelines [] for scoping reviews recommend a 3-step strategy: (1) an initial search of the database (titles and abstracts) using medical subject heading terms, (2) extending the search query to other databases and adjusting the search strategy for each database, and (3) reviewing the reference list of the selected papers.
Databases were searched from January 2019 to March 2024. This period was selected to capture key developments, for example, the WHO’s first digital health guidance for youth [], the renewed focus on adolescents through the SDGs [], and the COVID-19 pandemic’s exposure of deep and preexisting mental health inequalities among adolescent LMICs [,]. Collectively, these highlighted the need for a timely review of adolescent DMHIs in low-resource countries. The search strings and results for each database are presented in . The reference lists of all related reviews and included studies were hand-searched for additional papers.
Screening
In total, 1 reviewer (CW) initially searched the databases, focusing on the first 200-300 results per database to manage the workload []. The most relevant papers appeared at the top of the search results. CW exported all relevant records to EndNote (Clarivate Analytics) to remove duplicate references. CW then imported the remaining records to Rayyan (Rayyan Systems Inc) for title, abstract, and full-text screening. Rayyan places all imported papers into an “undecided” category, allowing reviewers to independently “exclude,” “maybe,” or “include” each paper based on the eligibility criteria. The first screening involved only the title and abstract review (CW, LM, and CR). During the second screening, CW retrieved the full text of papers that potentially met the eligibility criteria. Where a study had a published protocol and study outcome paper, only the study outcome paper was included in the review. The relevant papers were identified by CW, LM, and CR and placed in the “include” category in the final screening step. Any discrepancies were discussed and resolved by consensus. All authors agreed that the included papers fully met the eligibility criteria.
Data Charting and Data Items
The authors CW, LM, and ML developed and piloted a data charting form at the protocol stage. This was further refined iteratively at the review stage, and 2 reviewers (CW and CR) extracted the data (CW extracted the data and CR independently verified it). This approach is appropriate where it is not feasible for reviewers to independently chart the data []. Any inconsistencies were discussed and resolved. The data items extracted for each article aligned with the review aims and objectives and included the following:
- Bibliographic information: lead author, date of publication, and country of lead author.
- Data items related to subRQ1: DMHI name, description, target mental health disorder, start year of intervention implementation, country of intervention, participant age, sample size, study aims, intervention type, control, evaluation, outcome measures, duration, follow-up, and costs.
- Data items related to subRQ2: adolescent involvement in DMHI design.
- Data items related to subRQ3: consulted frameworks, theories, or models.
- Data items related to subRQ4: factors affecting adolescent engagement with the DMHI (facilitators or barriers).
- Key findings.
The data charting document can be found in .
Collating, Summarizing, Synthesizing, and Reporting the Results
Selected papers were reviewed to identify the similarities and differences between the DMHIs. They were then summarized and categorized into themes related to the review questions.
Results
Overview
Initial searches of 10 databases by CW in March 2024 yielded 2226 articles. Of these, 1291 (57.99%) duplicates were removed in EndNote. The remaining 935 papers were exported to Rayyan for title, abstract, and full-text screening. During the first round of screening, 80 more duplicates were removed, leaving 855 papers. CW identified 25 additional articles by searching the reference lists of excluded reviews and articles selected for inclusion. At title and abstract screening, CW reviewed 100% (880/880) of the records, and 2 reviewers (LM and CR) independently screened 25% (220/880) each, ensuring the records fulfilled the inclusion criteria. LM and CR achieved a 99% alignment score with CW’s screened papers. The detailed selection process of the articles is presented in the PRISMA-ScR flow diagram (
Figure 1).
Study Characteristics
Overview
A total of 20 studies were reviewed. The articles were published in English between 2019 and 2024: a total of 20% (4/20) in 2019, a total of 25% (5/20) in 2020, a total of 15% (3/20) in 2021, a total of 20% (4/20) in 2022, a total of 10% (2/20) in 2023, and 10% (2/20) in 2024. Where provided, the start year of the intervention implementation was noted: 2013, a total of 5% (1/20) []; 2017, a total of 5% (1/20) []; 2018, a total of 5% (1/20) []; 2019, a total of 15% (3/20) [,,]; 2020, a total of 15% (3/20) [-]; 2021, a total of 15% (3/20) [-]; 2022, a total of 5% (1/20) []; and 2023, a total of 5% (1/20) []. The implementation date or year was unclear in the following references [-].
Geographically, the lead authors were from India (5/20, 25%); United States (5/20, 25%); Norway (3/20, 15%); and China, Colombia, Finland, Iran, Kenya, Philippines, and Switzerland (1/20, 5% each).
Concerning study design, RCTs were 25% (5/20) [,,,,] of all studies, followed by RCT protocols (3/20, 15%) [,,], qualitative studies (4/20, 20%) [,,,], mixed methods approaches (5/20, 25%) [-,,], a quasi-experimental feasibility study (1/20, 5%) [], a multicycle usability testing approach (1/20, 5%) [], an intervention design (1/20, 5%) [], and a pilot study (1/20, 5%) []. A total of 15% (3/20) were study protocols, therefore the studies were underway.
Regarding the outcome measures, 70% (14/20) reported on primary and secondary measures, while 30% (6/20) did not. The RCTs used a total of 21 different psychometric tests, scales, and outcome measures (see for the charted data). The most common psychometric measures were the Patient Health Questionnaire-8 and Patient Health Questionnaire-9 for measuring depression and the Generalized Anxiety Disorders-7 for measuring anxiety. In 2 studies, the outcome measures were culturally validated for the target population [,].
In total, 100% (20/20) of studies described participants as “adolescents” aged between 10 and 19 years. Overall, the sample size was noted in 90% (18/20) of papers and ranged from 10 to 3960 participants. However, in 2 studies, 10% (2/20) did not report the sample size [,].
The duration of the interventions varied widely; for example, 10% (2/20) were single-session interventions [,], 5% (1/20) lasted 2 to 4 weeks [], 10% (2/20) lasted 4 weeks [,], 5% (1/20) lasted 5 weeks [], 5% (1/20) lasted 7 weeks [], 5% (1/20) lasted 2 months [], 5% (1/20) lasted 10 weeks [], 5% (1/20) lasted 11 weeks [], 5% (1/20) lasted 12 weeks [], 5% (1/20) was anticipated to last between 2 and 4 months [], 20% (4/20) lasted 6 months [,,,], and 5% (1/20) lasted 12 months []. The following did not specify the length of the intervention [,,]. Of the 20 papers, 60% (12/20) had a follow-up period ranging from baseline to 12 months, and 40% (8/20) did not report a follow-up period. In 1 study (1/20, 5%), the authors called for a future economic evaluation of the DMHI []. Another paper (1/20, 5%) [], reported on a cost-effective evaluation, noting that the cost of an incremental increase in well-being was US $37, and the cost of reducing emotional and behavioral issues was US $20. Of the other studies, 90% (18/20) did not report on the cost-effectiveness of their DMHI.
DMHIs
A total of 14 different DMHIs were identified, addressing one or more of the following mental health disorders: depression, 65% (13/20); well-being, 30% (6/20); emotional problem-solving, 20% (4/20); anxiety, 30% (6/20); distress, 5% (1/20); suicidal ideation, 10% (2/20); stigma, 10% (2/20); resilience, 5% (1/20); self-harm, 15% (3/20); and general mental health disorders, 10% (2/20). The main combination was depression and anxiety. Most DMHIs (13/20, 65%) were delivered or planned to be delivered in school settings, while others were delivered in clinics (2/20, 10%), hospitals (2/20, 10%), Urban Primary Health Centre in slum clusters (2/20, 10%), and refugee camps (1/20, 5%). Finally, 5% (1/20) did not specify the setting [] (see 
Table 1).
| Digital mental health intervention and delivery setting | Description | References | |
| Happy Helping Hand (Attensi Global; Lebanon, Ukraine)—schools and refugee camps |
| [,,] | |
| DepisNet-Thai (Thailand)—schools |
| [] | |
| Intervention via WhatsApp (Meta Platforms, Inc; Kenya)—care clinics |
| [,] | |
| SMS text messaging intervention (China)—hospitals |
| [] | |
| DIALOG+S (Unit for Social and Community Psychiatry at Queen Mary University of London and the East London Foundation Trust and Pontificia Universidad Javeriana, Colombia)—schools |
| [] | |
| PRIDEa research program (2016-2022) | |||
| PODb Adventures (India)—schools |
| [] | |
| PRIDE (2016-2022; India)—schools |
| [] | |
| STARSc (South Africa)—N/Ad |
| [] | |
| Shamiri-Digital (Kenya)—schools |
| [,] | |
| DADf course (Iran)—schools |
| [] | |
| ARTEMISg Intervention (India)—Urban Primary Health Centre in slum clusters |
| [,] | |
| Kuamsha app (South Africa and Uganda)—schools |
| [] | |
| Smartteen (India)—hospital |
| [] | |
| Self-help game-based mobile app intervention (Philippines)—schools |
| [] | |
| 3 Computerized SSIs (India)—schools |
| [] | |
aPRIDE: Premium for Adolescents.
bPOD: identifying “Problems,” generating “Options,” and creating a “Do it” plan.
cSTARS: Sustainable Technology for Adolescents and Youth to Reduce Stress.
dN/A: not available.
eSSI: single session intervention.
fDAD: Dorehye Amozeshie Dokhtaran.
gARTEMIS: Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums.
hCBT: cognitive behavioral therapy.
Of the 20 studies, 30% (6/20) were conducted in India, 20% (4/20) in Kenya, 10% (2/20) in Lebanon, 5% (1/20) in China, 5% (1/20) in Colombia, 5% (1/20) in Iran, 5% (1/20) in the Philippines, 5% (1/20) in South Africa, 5% (1/20) in South Africa and Uganda, 5% (1/20) in Thailand, and 5% (1/20) in Ukraine. There were 4 upper-middle–income economies, 6 lower-middle–income economies, and 1 low-income economy. All met the LMIC definition set by the World Bank (see Tables 2 and 3).
| Study country | Studies, n (%) | References |
| India | 6 (30) | [,-,,] |
| Kenya | 4 (20) | [,,,] |
| Lebanon | 2 (10) | [,] |
| China | 1 (5) | [] |
| Colombia | 1 (5) | [] |
| Iran | 1 (5) | [] |
| Philippines | 1 (5) | [] |
| South Africa | 1 (5) | [] |
| South Africa and Uganda | 1 (5) | [] |
| Thailand | 1 (5) | [] |
| Ukraine | 1 (5) | [] |
| Economy typea | Countries, n | List of countries |
| Upper-middle–income economies (US $4466 to US $13,845) | 4 | China, Colombia, South Africa, and Thailand |
| Lower-middle–income economies (US $1136 to US $4465) | 6 | India, Iran, Kenya, Lebanon, Philippines, and Ukraine |
| Low-income economies (US $1135 or less) | 1 | Uganda |
aWorld Bank Classification for the current 2024 fiscal year, low- and middle-income countries are defined as those with a gross national income per capita less than US $13,845.
Adolescent Involvement in DMHI Design
Of the 20 papers, 60% (12/20) involved adolescents in co-designing [] or participatory approaches at the design stage. For instance, the content of DepisNet-Thai [] was developed from discussions with adolescents. Elsewhere, co-designing activities informed the development of POD Adventures [], PRIDE [], antistigma materials for ARTEMIS (Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums) [], and the Kuamsha app []. Adolescents contributed to intervention designs via group discussions for Smartteen [] and Shamiri-Digital []. Some interventions were existing DMHIs adapted to the target users’ cultural contexts, for example, the Happy Helping Hand was initially launched in Norway but was developed in collaboration with adolescent Syrian refugees and Norwegian adolescents. It was later culturally adapted for use with refugee adolescents in Lebanon [,]. Furthermore, the Happy Helping Hand materials were also adapted for Ukrainian adolescents []. In an ongoing study (1/20, 5%), an Adolescent Expert Advisory Group will guide researchers on all aspects of the project design, development, and implementation [].
Consulted Frameworks, Theories, or Models
Of 20 studies, 60% (n=12) engaged with frameworks, toolkits, models, or theories related to behavior change interventions. These included cognitive behavioral models, social learning theory, theories of coping during adolescence, persuasive systems design, and stress-coping theory, among others. Further, 1 (5%) study consulted a digital health framework that included digital maturity and readiness content []. Another study referred to guidance for developing and evaluating complex interventions []. One paper (5%) reported how their intervention aligned with the national mental health program and the government’s national adolescent health program [], ensuring their DMHI meets the needs and priorities of the nation’s adolescents.
Factors Affecting Adolescent Engagement With the DMHI (Facilitators or Barriers)
A total of 90% (18/20) reported on adolescent facilitators and barriers to DMHI engagement. Further, 3 (15%) studies were underway and, therefore, had no findings or results to report at the time of the present review. However, the authors listed factors that facilitated or impeded access to DMHIs for adolescents where possible (see Textbox 2).
Key barriers
- COVID-19 challenges, for example, exposed adolescents’ overlooked social and personal needs.
- Complexity of app implementation, for example, lay counselor training, app development, and mental health service referrals.
- Household and school responsibilities.
- Power outages.
- Affordability, for example, insufficient phone credit, and lack of access to smartphones or mobile phones.
- Limited digital skills, low literacy levels, or both.
- Challenges with user emails reduced acceptance of the program.
- Time constraints on the app delivery by psychosocial staff or teachers.
- Boarding school attendance prevented 1 adolescent participant from participating.
- Duration of intervention.
- Stigma, discrimination, or lack of cultural appropriateness.
- Parent control of device use, for example, parents keeping adolescents’ mobile phones for most of the day.
- Lack of privacy and confidentiality.
- Concerns from teachers and parents on the time spent on phones.
- Poor local infrastructure and connectivity.
- Lack of online functionality.
- Lack of digital skills leading to a dependency on teachers or lay staff guiding the intervention.
- Device affordability, for example, phone data running out often and having to wait to top up again.
- Cost of intervention.
- Digital divide evident, for example, lack of digital device ownership and infrastructure.
Key facilitators
- Adapting digital mental health interventions (DMHIs) to the local context, for example, including language, cultural, and religious values.
- Embedding cultural appropriateness by design.
- Access to counselors or lay support.
- Offline access.
- Reading materials in addition to the DMHI.
- A safe space to discuss sensitive and stigmatized issues, for example, HIV-related concerns.
- Value in face-to-face interaction.
- Private school setting meant the availability of computers and internet connection.
- Family support.
- The timing of the intervention, for example, receiving a SMS text message during evenings when self-harm urges are increased.
- Peer groups comprising 10 to 15 adolescents to provide lay counseling and mental health support for adolescents in the community.
- Gamification.
- Tailored content.
- Privacy and confidentiality.
- Free to use, for example, the provision of a smartphone with the WhatsApp app preinstalled.
Discussion
Summary of Evidence
This scoping review aimed to explore what is known about DMHIs for adolescents in LMICs, as reported in the literature. In 2024, the World Bank categorized 134 countries as LMICs []. This review located only 11 countries engaged with adolescent DMHIs, illustrating the limited implementation of these solutions across LMICs and reflecting the emergent nature of the field []. We analyzed 20 papers to understand how these DMHIs were designed and evaluated, in what capacity adolescents were involved in the design process, what frameworks or theories were applied, and what factors impacted adolescent engagement with the DMHIs.
Overall, 14 different DMHIs were identified (POD Adventures is part of the PRIDE research program 2016-2022). DMHIs were designed for a limited range of mental health disorders, namely depression and anxiety. Almost all included adolescent input at the design stage, underscoring the importance of person-centered or user-centered approaches in DHIs. The DMHIs were delivered in diverse formats, including group delivery and self-guided methods, with lay staff or mental health professionals, including counselors, psychologists, or psychiatrists. Interventions were administered through smartphones, mobile phones, tablets, computers, and the web, often conducted in schools or clinical settings. Significant structural, psychological, and financial barriers exist in engagement with the DMHIs.
The study designs included RCTs, mixed methods, and qualitative studies. Most studies reported positive outcomes for symptom reduction, feasibility, and acceptability, measured by specific outcome measures. However, most were small scale and not trialed on a large scale over extended periods. While DMHIs were feasible and acceptable among adolescents, there remains a gap in the literature about their long-term cost- and clinical effectiveness.
Design of Adolescent DMHIs
The DMHIs were designed for depression, anxiety, self-harm, well-being, resilience, stigma, stress, and suicide ideation. However, this is not reflective of the broad and complex range of mental health disorders that adolescents develop, including substance misuse and posttraumatic stress disorder [,,]. Concerning intervention settings, most were delivered in schools or hospitals. Indeed, previous studies have recognized schools as critical settings that offer a vital entry point for receiving adolescent mental health services [,]. Conversely, out-of-school adolescents or those unlikely to visit those hospitals may miss the opportunity to engage with the DMHIs [].
While RCTs remain the gold standard for evaluating the effectiveness of DHIs [,], they are costly and time-consuming. We identified only 5 RCTs evaluating the effectiveness of adolescent DMHIs [,,,], with a further 3 RCT protocols underway [,,]. In India, Smartteen [] was trialed on 21 adolescents with depression against a treatment-as-usual group. The DMHI was feasible, acceptable, and more effective than the treatment-as-usual at reducing symptoms at 12 weeks. Even with reduced therapist time, adolescents adhered to treatment compliance. However, the authors call for more rigorous evaluations at scale. In another study [], DIALOG+ was adapted for adolescents in educational settings (DIALOG+S), using focus groups with teachers and adolescents. The DMHI was trialed with 70 Colombian adolescents, randomly assigned into DIALOG+S or an active control group (counseling as usual). The intervention was feasible and acceptable and could improve mental health, quality of life, and emotional symptoms. The authors call for larger studies to assess its efficacy. In Iran, the DAD (Dorehye Amozeshie Dokhtaran) [] was trialed with 128 adolescents, randomly assigned into the DAD or a control group. The intervention showed an improvement in depression symptoms. However, the effects decreased after 12 weeks. The intervention did not affect the outcome expectations or self-efficacy. Elsewhere Shamiri-Digital [] was trialed with 103 adolescents, randomly assigned into Shamiri-Digital or a study-skills control condition. Shamiri-Digital reduced depressive symptoms compared to the control. However, there were no significant effects on anxiety symptoms, well-being, or happiness. The authors called for replicate trials with extended follow-up periods. In a secondary analysis of this trial, the authors sought to evaluate the costs and cost-effectiveness of Shamiri-Digital through an economic evaluation []. Their findings indicate that Shamiri-Digital can be delivered for less than US $4 per student, which is more cost-effective than traditional interventions, for example, 12-16-week cognitive behavioral therapy sessions. However, it is difficult to draw inferences from these studies without critically assessing the quality of the evidence.
A critical design decision came from the inclusion of culturally validated elements. Some studies culturally adapted some components of the intervention [,]. Further, 1 study culturally validated an existing psychometric tool for their DMHI, for example, the study [] adapted the Perceived Stress Scale to make it a reliable and valid instrument in keeping with the Thai culture. Additionally, another study [] used the Farsi version of the Perceived Social Support Scale-Revised and the Farsi version of the Sherer General Self-Efficacy Questionnaire, reporting that the adapted scales were more appropriate and aligned to Iranian cultures. This aligns with existing literature emphasizing the importance of culturally appropriate DHIs [,]. Furthermore, they ensure resonance with the target populations’ beliefs, practices, and values leading to improved engagement, treatment adherence, and outcomes [,,,-]. In contrast, the Norwegian-designed Happy Helping Hand app was not adapted to the Arab culture for Syrian adolescents, and this was detrimental to the delivery of a DMHI, as themes were deemed religiously and culturally inappropriate, and therefore, certain aspects of the DMHI were skipped [].
An important design element was the inclusion of lay, peer, or therapist support, with adolescents preferring this option [,,]. The benefits of this approach have been previously reported [,,], where interventions with in-person options were more effective than self-guided or automized interventions [].
Adolescent Engagement With DMHI Design
The advantages of involving adolescents in participatory and co-design approaches for DMHIs are well documented [,-]. In this review, for example, adolescents were engaged in a series of co-design workshops for exploring existing popular apps, for example, Temple Run (Imangi Studios) and Candy Crush (King); story building, paper prototyping, and discussions about prototype ideas for POD Adventures []; the cocreation of an antistigma campaign as part of the main intervention for the ARTEMIS project []; and 1 study incorporated feedback from recent high school graduates into the design of the group-based Shamiri-Digital intervention as an iterative process of the intervention design [].
Collectively these studies show that participatory approaches are invaluable for generating insights for improving the DMHIs. Nevertheless, there are challenges [], and authors are calling for the evaluation of co-design processes in diverse contexts and how that impacts technology [,] with clear guidance around these processes []. These insights will advance understanding of how and why adolescents engage in DHIs [] and the health care outcomes of such engagements. From the review, reported activities included feasibility, acceptability, and usability studies, workshops, interviews, and focus groups. All were used clearly and meaningfully to advance the DMHI design and evaluation [,,,,].
Applied Frameworks and Theories
Incorporating evidence-based theories and techniques that encourage user engagement and behavior change is an essential element of DHI design [,,,]. Most DMHIs in this review were underpinned by cognitive-behavioral techniques involving behavioral activation and problem-solving features [,,,,]. Some used stand-alone theories [], for example, the social cognitive theory, and others used a combination of behavior change techniques [], for example, theories of coping during adolescence, the self-determination theory, and the technology acceptance model. However, not all the papers explained how the behavior change theory or technique impacted the target user’s mental health. Or, indeed, the evidence on which the selection or combinations of theories were based [].
Further, incorporating national or international guidance enhanced the intervention design, development, and evaluation. For example, 1 study [] used the WHO guidelines on monitoring and evaluating DHIs, which addressed digital maturity, readiness, and scalability, whereas another [] consulted the UK Medical Research Council guidance for developing and evaluating complex interventions focused on process evaluation and contextual factors. Finally, 1 study [] accessed the WHO’s Mental Health Gap Action Programme-Intervention Guide, which considers training requirements and treatment protocols in low-resource settings.
Reported Barriers and Facilitators
Textbox 2 summarizes key barriers and facilitators from the included studies. Access and engagement with DMHIs can be impacted by financial, geographical, psychological, cultural, and structural factors [-]. Notable barriers were the affordability of devices, cultural appropriateness, power outages, poor digital and literacy skills, inadequate internet connectivity, and limited access to devices in rural areas. The findings are consistent with other LMICs [,,].
Stigma (self-stigma, eg, individual level, and societal stigma, eg, system level) remains a complex and multifaceted barrier despite the potential for increased privacy, confidentiality, anonymity, and accessibility through DMHIs [-]. Multiple factors contribute to persistent stigma, such as societal attitudes, the digital divide, lack of awareness, fear of exposure, discrimination, and cultural sensitivities [,,]. These factors could limit engagement with DMHIs due to the negative social consequences in areas where mental health is a stigmatized issue [,].
Further, 1 study described how COVID-19 worsened the already dire human rights and socioeconomic conditions for displaced Syrians living in Lebanon [], including the loss of income, food, and essential living services, disrupting education and access to learning resources. Another study [] reported similar findings in Colombia, noting that most adolescents impacted by COVID did not have access to support and treatment. This shortage of mental health professionals and a widening treatment gap, poor education, and community knowledge reinforced the stigma and false beliefs about mental health [].
Adolescents highlighted some facilitators, such as access to a safe space to discuss stigmatized mental health issues, culturally sensitive DMHIs that embed the local cultural and religious values by design, DMHIs that maintain face-to-face contact while offering therapist-led or lay or peer-led options, access to reading materials in addition to the DMHI, appropriate content that is entertaining, personalized, and has gamified elements, privacy and confidentiality, and DMHIs that are free to use. Previous studies have suggested that addressing the digital divide could mitigate these barriers, providing equitable access to DMHIs, yet care must be taken to ensure that the pursuit of DMHIs does not widen the existing inequalities, further reinforcing the digital divide, undermining the equitable delivery of care [-].
Limitations
To the authors’ knowledge, 1 scoping review [] has been published on DMHIs for adolescents in LMICs. Their review [] was limited to mobile phones only; however, this review has a wider scope, which is more inclusive of the technological infrastructure in LMICs. Furthermore, this review has a more extensive search strategy, searching 10 databases and including gray literature such as dissertations and conference proceedings, and is a more robust evidence synthesis, whereas the review [] only searched APA PsycINFO, Web of Science, Psychiatry Online, and EBSCOhost. Finally, the eligibility criteria are different, and the previous review is limited by its lack of specificity of publication parameters per publication dates. This review has been purposefully restricted to a 5-year period (2019-2024), as specified earlier in this paper.
There are some limitations to the present review; 1 reviewer performed the database searches, which may have introduced reviewer bias. However, 2 more reviewers independently screened 25% (220/880) of each of the titles and abstracts of potential papers against the eligibility criteria. This step reduces bias and adds credibility and rigor to the review process. In addition, the quality of included studies was not appraised or assessed for risk of bias therefore, there is no assurance of the quality of the evidence in this scoping review. This is consistent with the aim of a scoping review, which is to rapidly map the available literature and not perform a systematic analysis. Finally, the review was limited to papers published in English. This may have significantly reduced the number of eligible studies from LMICs.
Conclusions
The global shortage of access to mental health treatment demonstrates the critical need for effective, low-cost, scalable solutions. The COVID-19 pandemic has highlighted the need for alternative solutions, and leveraging digital technologies through the international frameworks of the UHC and SDGs can alleviate the significant burden of mental health inequalities among adolescents in LMICs.
In this review, adolescent involvement in various stages of the theory-based design and evaluation cycle enhanced the intervention’s acceptability, feasibility, and usefulness. While RCTs of DMHIs remain the gold standard in this field, they are sparse. Researchers have called for repeat trials at larger scales in diverse settings to assess their effectiveness, scalability, and feasibility. Moreover, only some of the tools and instruments used to evaluate the DMHIs were culturally validated in LMIC contexts, making it difficult to establish how the DMHI would benefit adolescents beyond the test settings.
Addressing the digital divide and related barriers, such as stigma, will be a critical challenge to ensuring equitable access to universal and affordable mental health care. Understanding the long-term clinical implications and the cost-effectiveness of DMHIs will be another challenge for future research because that is an essential consideration for sustainable DMHIs in resource-constrained settings.
Acknowledgments
We are grateful to Elaine Blair (MSc, Faculty Librarian of Science at the University of Strathclyde) for providing professional guidance on database identification and searches. Funding sources were not involved in the writing of this paper.
Authors' Contributions
CW, LM, and ML conceptualized this study. CW wrote the original draft of this paper. CW extracted the data from the articles, and CR validated the data extraction. Titles and abstracts were screened independently by CW (880/880, 100%), LM (220/880, 25%), and CR (220/880, 25%). CW conducted the full-text screening. CW, LM, and CR resolved conflicts during the screening. ML and LM edited multiple drafts; all authors reviewed and approved the final paper.
Conflicts of Interest
LM and ML are the primary and secondary supervisors, respectively, for CW’s PhD at the University of Strathclyde. LM is also the PhD supervisor for CR.
Multimedia Appendix 1
PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.
DOCX File , 109 KBReferences
- World mental health report: Transforming mental health for all. World Health Organization. Geneva.; 2022. URL: https://iris.who.int/bitstream/handle/10665/356119/9789240049338-eng.pdf?sequence=1
- Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, Omigbodun O, et al. Child and adolescent mental health worldwide: evidence for action. Lancet. Oct 22, 2011;378(9801):1515-1525. [CrossRef] [Medline]
- Helping adolescents thrive toolkit: strategies to promote and protect adolescent mental health and reduce self-harm and other risk behaviours. World Health Organization. 2021. URL: https://iris.who.int/bitstream/handle/10665/341327/9789240025554-eng.pdf?sequence=1 [accessed 2024-03-04]
- Comprehensive mental health action plan 2013-2030. World Health Organization. 2021. URL: https://iris.who.int/bitstream/handle/10665/345301/9789240031029-eng.pdf?sequence=1&isAllowed=y [accessed 2024-03-04]
- Clark H, Coll-Seck AM, Banerjee A, Peterson S, Dalglish SL, Ameratunga S, et al. A future for the world's children? A WHO-UNICEF-Lancet Commission. Lancet. Feb 22, 2020;395(10224):605-658. [CrossRef] [Medline]
- Mental health of adolescents. World Health Organization. Oct 10, 2024. URL: https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health [accessed 2024-10-15]
- Sawyer SM, Azzopardi PS, Wickremarathne D, Patton GC. The age of adolescence. Lancet Child Adolesc Health. Mar 2018;2(3):223-228. [CrossRef] [Medline]
- The state of the world's children 2021: on my mind promoting, protecting and caring for children's mental health. United Nations Children's Fund. 2021. URL: https://www.unicef.org/media/108036/file/SOWC-2021-executive-summary.pdf [accessed 2024-03-01]
- World bank country and lending groups. The World Bank. URL: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups [accessed 2024-03-14]
- Babatunde GB, van Rensburg AJ, Bhana A, Petersen I. Barriers and Facilitators to Child and Adolescent Mental Health Services in Low-and-Middle-Income Countries: a Scoping Review. Glob Soc Welf. Jun 07, 2019;8(1):29-46. [CrossRef]
- Kola L, Kohrt BA, Hanlon C, Naslund JA, Sikander S, Balaji M, et al. COVID-19 mental health impact and responses in low-income and middle-income countries: reimagining global mental health. Lancet Psychiatry. Jun 2021;8(6):535-550. [FREE Full text] [CrossRef] [Medline]
- Sharma M, Idele P, Manzini A. Life in lockdown: child and adolescent mental health and well-being in the time of COVID-19. ERIC. 2021. URL: https://files.eric.ed.gov/fulltext/ED615551.pdf [accessed 2022-10-04]
- Caplan S, Lovera AS, Liberato PR. A feasibility study of a mental health mobile app in the Dominican Republic: The untold story. Int J Ment Health. Apr 04, 2019;47(4):311-345. [CrossRef]
- Kaonga NN, Morgan J. Common themes and emerging trends for the use of technology to support mental health and psychosocial well-being in limited resource settings: a review of the literature. Psychiatry Res. Nov 2019;281:112594. [CrossRef] [Medline]
- Naslund JA, Gonsalves PP, Gruebner O, Pendse SR, Smith SL, Sharma A, et al. Digital innovations for global mental health: opportunities for data science, task sharing, and early intervention. Curr Treat Options Psychiatry. Dec 2019;6(4):337-351. [FREE Full text] [CrossRef] [Medline]
- Calvo-Valderrama MG, Marroquín-Rivera A, Burn E, Ospina-Pinillos L, Bird V, Gómez-Restrepo C. Adapting a mental health intervention for adolescents during the COVID-19 pandemic: web-based synchronous focus group study. JMIR Form Res. Nov 03, 2021;5(11):e30293. [FREE Full text] [CrossRef] [Medline]
- Mundt AP, Irarrázaval M, Martínez P, Fernández O, Martínez V, Rojas G. Telepsychiatry consultation for primary care treatment of children and adolescents receiving child protective services in Chile: mixed methods feasibility study. JMIR Public Health Surveill. Jul 22, 2021;7(7):e25836. [FREE Full text] [CrossRef] [Medline]
- Rodriguez-Villa E, Rozatkar AR, Kumar M, Patel V, Bondre A, Naik SS, et al. Cross cultural and global uses of a digital mental health app: results of focus groups with clinicians, patients and family members in India and the United States. Glob Ment Health (Camb). 2021;8:e30. [FREE Full text] [CrossRef] [Medline]
- Balcombe L, de Leo D. Digital mental health challenges and the horizon ahead for solutions. JMIR Ment Health. Mar 29, 2021;8(3):e26811. [FREE Full text] [CrossRef] [Medline]
- Classification of digital health interventions v1.0. World Health Organization. 2018. URL: https://www.who.int/publications/i/item/WHO-RHR-18.06 [accessed 2024-03-24]
- Wasil AR, Park SJ, Gillespie S, Shingleton R, Shinde S, Natu S, et al. Harnessing single-session interventions to improve adolescent mental health and well-being in India: development, adaptation, and pilot testing of online single-session interventions in Indian secondary schools. Asian J Psychiatr. Apr 2020;50:101980. [CrossRef] [Medline]
- Kacmarek CN, Johnson NE, Osborn TL, Wasanga C, Weisz JR, Yates BT. Costs and cost-effectiveness of Shamiri, a brief, layperson-delivered intervention for Kenyan adolescents: a randomized controlled trial. BMC Health Serv Res. Aug 04, 2023;23(1):827. [FREE Full text] [CrossRef] [Medline]
- Moeini B, Bashirian S, Soltanian AR, Ghaleiha A, Taheri M. Examining the effectiveness of a web-based intervention for depressive symptoms in female adolescents: applying social cognitive theory. J Res Health Sci. Aug 19, 2019;19(3):e00454. [FREE Full text] [Medline]
- Anttila M, Sittichai R, Katajisto J, Välimäki M. Impact of a web program to support the mental wellbeing of high school students: a quasi experimental feasibility study. Int J Environ Res Public Health. 2019;16(14):2473. [FREE Full text] [CrossRef] [Medline]
- Mummah SA, Robinson TN, King AC, Gardner CD, Sutton S. IDEAS (integrate, design, assess, and share): a framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J Med Internet Res. Dec 16, 2016;18(12):e317. [FREE Full text] [CrossRef] [Medline]
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. Aug 2013;46(1):81-95. [FREE Full text] [CrossRef] [Medline]
- Hamilton J, Siegel D, Carper M. 16—Digital media interventions for adolescent mental health. Handb Adolesc Digital Media Use Ment Health. 2022:389. [FREE Full text] [CrossRef]
- Hartog K, Hubbard CD, Krouwer AF, Thornicroft G, Kohrt BA, Jordans MJD. Stigma reduction interventions for children and adolescents in low- and middle-income countries: systematic review of intervention strategies. Soc Sci Med. Feb 2020;246:112749. [FREE Full text] [CrossRef] [Medline]
- Kanuri N, Arora P, Talluru S, Colaco B, Dutta R, Rawat A, et al. Examining the initial usability, acceptability and feasibility of a digital mental health intervention for college students in India. Int J Psychol. Aug 2020;55(4):657-673. [FREE Full text] [CrossRef] [Medline]
- Gómez-Restrepo C, Sarmiento-Suárez MJ, Alba-Saavedra M, Bird VJ, Priebe S, van Loggerenberg F. Adapting DIALOG+ in a school setting-a tool to support well-being and resilience in adolescents living in postconflict areas during the COVID-19 pandemic: protocol for a cluster randomized exploratory study. JMIR Res Protoc. Nov 09, 2022;11(11):e40286. [FREE Full text] [CrossRef] [Medline]
- Gonsalves PP, Hodgson ES, Bhat B, Sharma R, Jambhale A, Michelson D, et al. App-based guided problem-solving intervention for adolescent mental health: a pilot cohort study in Indian schools. Evid Based Ment Health. Feb 2021;24(1):11-18. [FREE Full text] [CrossRef] [Medline]
- Gronholm PC, Makhmud A, Barbui C, Brohan E, Chowdhary N. Qualitative evidence regarding the experience of receiving and providing care for mental health conditions in non-specialist settings in low-income and middle-income countries: a systematic review of reviews. BMJ Ment Health. 2023;26(1):e300755. [FREE Full text] [CrossRef] [Medline]
- Raj V, Raykar V, Robinson AM, Islam MR. Child and adolescent mental health training programs for non-specialist mental health professionals in low and middle income countries: a scoping review of literature. Community Ment Health J. Jan 2022;58(1):154-165. [FREE Full text] [CrossRef] [Medline]
- Devassy SM, Scaria L, Cheguvera N. Task sharing and stepped referral model for community mental health promotion in low- and middle-income countries (LMIC): insights from a feasibility study in India. Pilot Feasibility Stud. Aug 30, 2022;8(1):192. [FREE Full text] [CrossRef] [Medline]
- Juengsiragulwit D. Opportunities and obstacles in child and adolescent mental health services in low- and middle-income countries: a review of the literature. WHO South East Asia J Public Health. 2015;4(2):110-122. [CrossRef] [Medline]
- Naeem F, Husain M, Husain M, Javed A. Digital psychiatry in low- and middle-income countries post-COVID-19: opportunities, challenges, and solutions. Indian J Psychiatry. Sep 2020;62(Suppl 3):S380-S382. [FREE Full text] [CrossRef] [Medline]
- van Dijk JAGM. Digital divide: impact of access. Int Encycl Media Eff. 2017:1-11. [CrossRef]
- Ferretti A, Vayena E, Blasimme A. Unlock digital health promotion in LMICs to benefit the youth. PLOS Digit Health. Aug 2023;2(8):e0000315. [FREE Full text] [CrossRef] [Medline]
- Ragnedda M, Gladkova A. Understanding digital inequalities in the global south. Digital Inequalities Global South. 2020:17-30. [FREE Full text] [CrossRef]
- Rural-urban gap in mobile internet use in low-and middle-income countries (LMICs) worldwide from 2017 to 2019, by region. Statista. URL: https://www.statista.com/statistics/264067/global-mobile-internet-use-rural-urban-gap-low-middle-income-countries-region/ [accessed 2024-03-06]
- Most common barriers to mobile internet use in low and middle income countries (LMICs) in 2019, by region. Statista. URL: https://www.statista.com/statistics/265731/share-barriers-mobile-internet-use-low-middle-income-countries-region/ [accessed 2024-03-06]
- The state of mobile internet connectivity 2022. GSMA. 2022. URL: https://www.gsma.com/r/wp-content/uploads/2022/12/The-State-of-Mobile-Internet-Connectivity-Report-2022.pdf?utm_source=website&utm_medium=download-button&utm_campaign=somic22 [accessed 2024-04-03]
- Boucher E, Moskowitz JT, Kackloudis GM, Stafford JL, Kwok I, Parks AC. Immediate and long-term effects of an 8-week digital mental health intervention on adults with poorly managed type 2 diabetes: protocol for a randomized controlled trial. JMIR Res Protoc. Aug 04, 2020;9(8):e18578. [FREE Full text] [CrossRef] [Medline]
- Borghouts J, Eikey E, Mark G, de Leon C, Schueller SM, Schneider M, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res. Mar 24, 2021;23(3):e24387. [FREE Full text] [CrossRef] [Medline]
- Faria M, Zin STP, Chestnov R, Novak AM, Lev-Ari S, Snyder M. Mental health for all: the case for investing in digital mental health to improve global outcomes, access, and innovation in low-resource settings. J Clin Med. 2023;12(21):6735. [FREE Full text] [CrossRef] [Medline]
- van Stam G. Conceptualization and practices in digital health: voices from Africa. Afr Health Sci. Mar 2022;22(1):664-672. [FREE Full text] [CrossRef] [Medline]
- Chakrabarti S. Digital psychiatry in low-and-middle-income countries: new developments and the way forward. World J Psychiatry. Mar 19, 2024;14(3):350-361. [FREE Full text] [CrossRef] [Medline]
- McCool J, Dobson R, Muinga N, Paton C, Pagliari C, Agawal S, et al. Factors influencing the sustainability of digital health interventions in low-resource settings: lessons from five countries. J Glob Health. Dec 2020;10(2):020396. [FREE Full text] [CrossRef] [Medline]
- Greenhalgh T, Abimbola S. The NASSS framework—a synthesis of multiple theories of technology implementation. Stud Health Technol Inform. Jul 30, 2019;263:193-204. [CrossRef] [Medline]
- Carter H, Araya R, Anjur K, Deng D, Naslund JA. The emergence of digital mental health in low-income and middle-income countries: a review of recent advances and implications for the treatment and prevention of mental disorders. J Psychiatr Res. Jan 2021;133:223-246. [FREE Full text] [CrossRef] [Medline]
- Holly L, Smith RD, Ndili N, Franz C, Stevens EAG. A review of digital health strategies in 10 countries with young populations: do they serve the health and wellbeing of children and youth in a digital age? Front Digit Health. 2022;4:817810. [FREE Full text] [CrossRef] [Medline]
- Faulkenberry JG, Luberti A, Craig S. Electronic health records, mobile health, and the challenge of improving global health. Curr Probl Pediatr Adolesc Health Care. Jan 2022;52(1):101111. [CrossRef] [Medline]
- Kickbusch I, Piselli D, Agrawal A, Balicer R, Banner O, Adelhardt M, et al. Secretariat of the LancetFinancial Times Commission. The Lancet and Financial Times Commission on governing health futures 2030: growing up in a digital world. Lancet. Nov 06, 2021;398(10312):1727-1776. [CrossRef] [Medline]
- Schueller SM, Torous J. Scaling evidence-based treatments through digital mental health. Am Psychol. Nov 2020;75(8):1093-1104. [FREE Full text] [CrossRef] [Medline]
- Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. Apr 29, 2021;8(4):e25847. [FREE Full text] [CrossRef] [Medline]
- Osborn TL, Rodriguez M, Wasil AR, Venturo-Conerly KE, Gan J, Alemu RG, et al. Single-session digital intervention for adolescent depression, anxiety, and well-being: outcomes of a randomized controlled trial with Kenyan adolescents. J Consult Clin Psychol. Jul 2020;88(7):657-668. [CrossRef] [Medline]
- Siddiqui S, Gonsalves PP, Naslund JA, Stein DJ, Fineberg NA, Chamberlain SR. 20—Scaling up of mental health services in the digital age: the rise of technology and its application to low- and middle-income countries. Ment Health Digital World. 2022:459-479. [CrossRef]
- Youth-centred digital health interventions: a framework for planning, developing and implementing solutions with and for young people. World Health Organization. 2020. URL: https://iris.who.int/bitstream/handle/10665/336223/9789240011717-eng.pdf?sequence=1&isAllowed=y [accessed 2024-03-16]
- Kim J, Aryee LMD, Bang H, Prajogo S, Choi YK, Hoch JS, et al. Effectiveness of digital mental health tools to reduce depressive and anxiety symptoms in low- and middle-income countries: systematic review and meta-analysis. JMIR Ment Health. Mar 20, 2023;10:e43066. [FREE Full text] [CrossRef] [Medline]
- Peters MDJ, McInerney P, Munn Z, Tricco A, Khalil H. Scoping reviews. JBI. URL: https://jbi-global-wiki.refined.site/space/MANUAL/355862497/10.+Scoping+reviews [accessed 2024-03-28]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [FREE Full text] [CrossRef] [Medline]
- Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci. Sep 20, 2010;5:69. [FREE Full text] [CrossRef] [Medline]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. Feb 2005;8(1):19-32. [CrossRef]
- Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. Nov 19, 2018;18(1):143. [FREE Full text] [CrossRef] [Medline]
- Kruzan KP, Williams KDA, Meyerhoff J, Yoo DW, O'Dwyer LC, de Choudhury M, et al. Social media-based interventions for adolescent and young adult mental health: a scoping review. Internet Interv. Dec 2022;30:100578. [FREE Full text] [CrossRef] [Medline]
- Malloy JA, Partridge SR, Kemper JA, Braakhuis A, Roy R. Co-design of digital health interventions for young adults: protocol for a scoping review. JMIR Res Protoc. Oct 24, 2022;11(10):e38635. [FREE Full text] [CrossRef] [Medline]
- Povey J, Raphiphatthana B, Torok M, Nagel T, Shand F, Sweet M, et al. Involvement of Indigenous young people in the design and evaluation of digital mental health interventions: a scoping review protocol. Syst Rev. May 05, 2021;10(1):133. [FREE Full text] [CrossRef] [Medline]
- Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS One. 2015;10(9):e0138237. [FREE Full text] [CrossRef] [Medline]
- Gonsalves PP, Hodgson ES, Kumar A, Aurora T, Chandak Y, Sharma R, et al. Design and development of the "POD Adventures" smartphone game: a blended problem-solving intervention for adolescent mental health in India. Front Public Health. 2019;7:238. [FREE Full text] [CrossRef] [Medline]
- Pozuelo JR, Moffett BD, Davis M, Stein A, Cohen H, Craske MG, et al. User-centered design of a gamified mental health app for adolescents in Sub-Saharan Africa: multicycle usability testing study. JMIR Form Res. Nov 30, 2023;7:e51423. [FREE Full text] [CrossRef] [Medline]
- Al-Khayat A. Digitalized psychosocial support in education. Exploring the impact of the Happy Helping Hand app for displaced Syrian adolescents in Lebanon. Oslo Metropolitan University. 2021. URL: https://oda.oslomet.no/oda-xmlui/bitstream/handle/11250/2976713/Al-Khayat_flkm2021.pdf?sequence=1&isAllowed=y [accessed 2024-09-19]
- Raknes S. The happy helping hand used by Syrian displaced adolescents in Lebanon: a pilot study of feasibility, usefulness and impact. White Paper. 2020:1-9.
- Mukherjee A, Yatirajula SK, Kallakuri S, Paslawar S, Lempp H, Raman U, et al. Using formative research to inform a mental health intervention for adolescents living in Indian slums: the ARTEMIS study. Child Adolesc Psychiatry Ment Health. Jan 20, 2024;18(1):14. [FREE Full text] [CrossRef] [Medline]
- Gonsalves PP, Sharma R, Hodgson E, Bhat B, Jambhale A, Weiss HA, et al. A guided internet-based problem-solving intervention delivered through smartphones for secondary school pupils during the COVID-19 pandemic in India: protocol for a pilot randomized controlled trial. JMIR Res Protoc. Oct 06, 2021;10(10):e30339. [FREE Full text] [CrossRef] [Medline]
- Wasil AR, Osborn TL, Weisz JR, DeRubeis RJ. Online single-session interventions for Kenyan adolescents: study protocol for a comparative effectiveness randomised controlled trial. Gen Psychiatr. 2021;34(3):e100446. [FREE Full text] [CrossRef] [Medline]
- Superable GM. Guided self-help game-based app for adolescents in the Philippines and low-to middle-income countries. 2022. URL: https://www.academia.edu/102203331/Guided_Self_Help_Game_Based_App_for_Adolescents_in_the_Philippines_and_Low_to_Middle_Income_Countries [accessed 2024-09-19]
- Yatirajula SK, Kallakuri S, Paslawar S, Mukherjee A, Bhattacharya A, Chatterjee S, et al. An intervention to reduce stigma and improve management of depression, risk of suicide/self-harm and other significant emotional or medically unexplained complaints among adolescents living in urban slums: protocol for the ARTEMIS project. Trials. Jul 29, 2022;23(1):612. [FREE Full text] [CrossRef] [Medline]
- Raknes S, Chorna T. The Helping Hand in Ukraine: feasibility and potential impact. Pilot Feasibility Stud. Jun 29, 2024;10(1):96. [FREE Full text] [CrossRef] [Medline]
- Chory A, Nyandiko W, Martin R, Aluoch J, Scanlon M, Ashimosi C, et al. HIV-related knowledge, attitudes, behaviors and experiences of Kenyan adolescents living with HIV revealed in WhatsApp group chats. J Int Assoc Provid AIDS Care. 2021;20:2325958221999579. [FREE Full text] [CrossRef] [Medline]
- Chory A, Callen G, Nyandiko W, Njoroge T, Ashimosi C, Aluoch J, et al. A pilot study of a mobile intervention to support mental health and adherence among adolescents living with HIV in Western Kenya. AIDS Behav. Jan 2022;26(1):232-242. [FREE Full text] [CrossRef] [Medline]
- Duan S, Wang H, Wilson A, Qiu J, Chen G, He Y, et al. Developing a text messaging intervention to reduce deliberate self-harm in Chinese adolescents: qualitative study. JMIR mHealth uHealth. Jun 11, 2020;8(6):e16963. [FREE Full text] [CrossRef] [Medline]
- Gómez-Restrepo C, Sarmiento-Suárez MJ, Alba-Saavedra M, Calvo-Valderrama MG, Rincón-Rodríguez CJ, Bird VJ, et al. Development and implementation of DIALOG+S in the school setting as a tool for promoting adolescent mental well-being and resilience in a post-armed conflict area in Colombia: exploratory cluster randomized controlled trial. JMIR Form Res. Oct 04, 2023;7:e46757. [FREE Full text] [CrossRef] [Medline]
- Hall J, Jordan S, van Ommeren M, Au T, Sway RA, Crawford J, et al. Sustainable Technology for Adolescents and youth to Reduce Stress (STARS): a WHO transdiagnostic chatbot for distressed youth. World Psychiatry. Feb 2022;21(1):156-157. [FREE Full text] [CrossRef] [Medline]
- Srivastava P, Mehta M, Sagar R, Ambekar A. Smartteen—a computer assisted cognitive behavior therapy for Indian adolescents with depression—a pilot study. Asian J Psychiatr. Apr 2020;50:101970. [CrossRef] [Medline]
- Sanders EB, Stappers PJ. Co-creation and the new landscapes of design. CoDesign. Mar 2008;4(1):5-18. [CrossRef]
- Ali MM, West K, Teich JL, Lynch S, Mutter R, Dubenitz J. Utilization of mental health services in educational setting by adolescents in the United States. J Sch Health. May 2019;89(5):393-401. [CrossRef] [Medline]
- Parikh R, Michelson D, Sapru M, Sahu R, Singh A, Cuijpers P, et al. Priorities and preferences for school-based mental health services in India: a multi-stakeholder study with adolescents, parents, school staff, and mental health providers. Glob Ment Health (Camb). 2019;6:e18. [FREE Full text] [CrossRef] [Medline]
- Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T, et al. What works and what doesn't work? A systematic review of digital mental health interventions for depression and anxiety in young people. Front Psychiatry. 2019;10:759. [FREE Full text] [CrossRef] [Medline]
- Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. Nov 2016;51(5):843-851. [FREE Full text] [CrossRef] [Medline]
- Zanaboni P, Ngangue P, Mbemba GIC, Schopf TR, Bergmo TS, Gagnon MP. Methods to evaluate the effects of internet-based digital health interventions for citizens: systematic review of reviews. J Med Internet Res. Jun 07, 2018;20(6):e10202. [FREE Full text] [CrossRef] [Medline]
- Wasil AR, Kacmarek CN, Osborn TL, Palermo EH, DeRubeis RJ, Weisz JR, et al. Economic evaluation of an online single-session intervention for depression in Kenyan adolescents. J Consult Clin Psychol. Aug 2021;89(8):657-667. [CrossRef] [Medline]
- Spanhel K, Balci S, Feldhahn F, Bengel J, Baumeister H, Sander LB. Cultural adaptation of internet- and mobile-based interventions for mental disorders: a systematic review. NPJ Digit Med. Aug 25, 2021;4(1):128. [FREE Full text] [CrossRef] [Medline]
- Nittas V, Daniore P, Chavez SJ, Wray TB. Challenges in implementing cultural adaptations of digital health interventions. Commun Med (Lond). Jan 05, 2024;4(1):7. [FREE Full text] [CrossRef] [Medline]
- Piers R, Williams JM, Sharpe H. Review: Can digital mental health interventions bridge the 'digital divide' for socioeconomically and digitally marginalised youth? A systematic review. Child Adolesc Ment Health. Feb 2023;28(1):90-104. [CrossRef] [Medline]
- Pineda BS, Mejia R, Qin YZ, Martinez J, Delgadillo LG, Muñoz RF. Updated taxonomy of digital mental health interventions: a conceptual framework. mHealth. 2023;9:28. [FREE Full text] [CrossRef] [Medline]
- Abuwalla Z, Kadhem Z, Gladstone T, Mikhael E, Bishay A, van Voorhees BW. Proposed model for the cultural adaptation of an internet-based depression prevention intervention (CATCH-IT) for Arab adolescents. Int J Adolesc Med Health. 2017;31(1):28719365. [FREE Full text] [CrossRef] [Medline]
- Galagali PM, Brooks MJ. Psychological care in low-resource settings for adolescents. Clin Child Psychol Psychiatry. Jul 2020;25(3):698-711. [FREE Full text] [CrossRef] [Medline]
- Brotherdale R, Berry K, Branitsky A, Bucci S. Co-producing digital mental health interventions: a systematic review. Digit Health. 2024;10:20552076241239172. [FREE Full text] [CrossRef] [Medline]
- Li S, Achilles M, Spanos S, Habak S, Werner-Seidler A, O’Dea B. A cognitive behavioural therapy smartphone app for adolescent depression and anxiety: co-design of ClearlyMe. tCBT. Mar 16, 2022;15:e13. [FREE Full text] [CrossRef]
- Brown M, Lord E, John A. Adaptation of ACTivate your wellbeing, a digital health and well-being program for young persons: co-design approach. JMIR Form Res. Apr 13, 2023;7:e39913. [FREE Full text] [CrossRef] [Medline]
- Liverpool S, Edbrooke-Childs J. A caregiver digital intervention to support shared decision making in child and adolescent mental health services: development process and stakeholder involvement analysis. JMIR Form Res. Jun 15, 2021;5(6):e24896. [FREE Full text] [CrossRef] [Medline]
- Ludlow K, Russell JK, Ryan B, Brown RL, Joynt T, Uhlmann LR, et al. Co-designing a digital mental health platform, "Momentum", with young people aged 7-17: a qualitative study. Digit Health. 2023;9:20552076231216410. [FREE Full text] [CrossRef] [Medline]
- Jones RB, Stallard P, Agha SS, Rice S, Werner-Seidler A, Stasiak K, et al. Practitioner review: co-design of digital mental health technologies with children and young people. J Child Psychol Psychiatry. Aug 2020;61(8):928-940. [FREE Full text] [CrossRef] [Medline]
- Bergin AD, Vallejos EP, Davies EB, Daley D, Ford T, Harold G, et al. Preventive digital mental health interventions for children and young people: a review of the design and reporting of research. NPJ Digit Med. 2020;3:133. [FREE Full text] [CrossRef] [Medline]
- Gan DZQ, McGillivray L, Larsen ME, Christensen H, Torok M. Technology-supported strategies for promoting user engagement with digital mental health interventions: a systematic review. Digit Health. 2022;8:20552076221098268. [FREE Full text] [CrossRef] [Medline]
- Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res. Jan 30, 2015;17(1):e30. [FREE Full text] [CrossRef] [Medline]
- Wies B, Landers C, Ienca M. Digital mental health for young people: a scoping review of ethical promises and challenges. Front Digit Health. 2021;3:697072. [FREE Full text] [CrossRef] [Medline]
- Poulsen A, Hickie I, Alam M. Overcoming barriers to mHealth co-design in low- and middle-income countries: a research toolkit. Inf Technol Dev. 2024:1-20. [FREE Full text] [CrossRef]
- Koly KN, Saba J, Muzaffar R, Modasser RB, Colon-Cabrera D, Warren N. Exploring the potential of delivering mental health care services using digital technologies in Bangladesh: a qualitative analysis. Internet Interv. Sep 2022;29:100544. [FREE Full text] [CrossRef] [Medline]
- Karyotaki E, Miguel C, Panagiotopoulou OM, Harrer M, Seward N, Sijbrandij M, et al. Digital interventions for common mental disorders in low- and middle-income countries: a systematic review and meta-analysis. Glob Ment Health (Camb). 2023;10:e68. [FREE Full text] [CrossRef] [Medline]
- Yao R, Zhang W, Evans R, Cao G, Rui T, Shen L. Inequities in health care services caused by the adoption of digital health technologies: scoping review. J Med Internet Res. Mar 21, 2022;24(3):e34144. [FREE Full text] [CrossRef] [Medline]
- Ahmed T, Rizvi SJR, Rasheed S, Iqbal M, Bhuiya A, Standing H, et al. Digital health and inequalities in access to health services in Bangladesh: mixed methods study. JMIR mHealth uHealth. Jul 21, 2020;8(7):e16473. [FREE Full text] [CrossRef] [Medline]
- Naslund JA, Deng D. Addressing mental health stigma in low-income and middle-income countries: a new frontier for digital mental health. Ethics Med Public Health. 2021;19:100719. [FREE Full text] [CrossRef] [Medline]
- Baños R, Herrero R, Vara MD. What is the current and future status of digital mental health interventions? Span J Psychol. Feb 02, 2022;25:e5. [CrossRef] [Medline]
- Zhao X, Stadnick NA, Ceballos-Corro E, Castro J, Mallard-Swanson K, Palomares KJ, et al. Facilitators of and barriers to integrating digital mental health into county mental health services: qualitative interview analyses. JMIR Form Res. May 16, 2023;7:e45718. [FREE Full text] [CrossRef] [Medline]
- Jardine J, Nadal C, Robinson S, Enrique A, Hanratty M, Doherty G. Between rhetoric and reality: real-world barriers to uptake and early engagement in digital mental health interventions. ACM Trans Comput-Hum Interact. Feb 05, 2024;31(2):1-59. [FREE Full text] [CrossRef]
- Thornicroft G, Sunkel C, Aliev AA, Baker S, Brohan E, El Chammay R, et al. The Lancet Commission on ending stigma and discrimination in mental health. Lancet. Oct 22, 2022;400(10361):1438-1480. [CrossRef] [Medline]
- Martínez V, Espinosa-Duque D, Jiménez-Molina Á, Rojas G, Vöhringer PA, Fernández-Arcila M, et al. Feasibility and acceptability of "Cuida tu Ánimo" (take care of your mood): an internet-based program for prevention and early intervention of adolescent depression in Chile and Colombia. Int J Environ Res Public Health. 2021;18(18):9628. [FREE Full text] [CrossRef] [Medline]
- Maloney CA, Abel WD, McLeod HJ. Jamaican adolescents' receptiveness to digital mental health services: a cross-sectional survey from rural and urban communities. Internet Interv. Sep 2020;21:100325. [FREE Full text] [CrossRef] [Medline]
- Whitehead L, Talevski J, Fatehi F, Beauchamp A. Barriers to and facilitators of digital health among culturally and linguistically diverse populations: qualitative systematic review. J Med Internet Res. Feb 28, 2023;25:e42719. [FREE Full text] [CrossRef] [Medline]
- Veinot TC, Mitchell H, Ancker JS. Good intentions are not enough: how informatics interventions can worsen inequality. J Am Med Inform Assoc. Aug 01, 2018;25(8):1080-1088. [FREE Full text] [CrossRef] [Medline]
- The state of mobile internet connectivity 2023. GSMA. 2023. URL: https://www.gsma.com/r/wp-content/uploads/2023/10/The-State-of-Mobile-Internet-Connectivity-Report-2023.pdf?utm_source=website&utm_medium=button&utm_campaign=somic23 [accessed 2024-04-03]
- Madonsela S, Ware LJ, Scott M, Watermeyer J. The development and use of adolescent mobile mental health (m-mhealth) interventions in low- and middle-income countries: a scoping review. S Afr J Psychol. Jul 08, 2023;53(4):471-483. [CrossRef]
Abbreviations
| ARTEMIS: Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums |
| DHI: digital health intervention |
| DMHI: digital mental health intervention |
| GSM: Global System for Mobile |
| LMIC: low- and middle-income country |
| PCC: population, concept, and context |
| POD: identifying “Problems,” generating “Options,” and creating a “Do it” plan |
| PRIDE: Premium for Adolescents |
| PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews |
| RCT: randomized controlled trial |
| SDG: Sustainable Development Goal |
| subRQ: subreview question |
| UHC: universal health coverage |
| UNICEF: United Nations International Children’s Emergency Fund |
| WHO: World Health Organization |
Edited by T de Azevedo Cardoso; submitted 29.07.23; peer-reviewed by SK Choi, S Lehtimaki, Y Ranjit; comments to author 05.11.23; revised version received 07.05.24; accepted 04.09.24; published 29.10.24.
Copyright©Carolina Wani, Lisa McCann, Marilyn Lennon, Caterina Radu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.10.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.