Published on in Vol 24, No 5 (2022): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26015, first published .
eHealth Tools That Assess and Track Health and Well-being in Children and Young People: Systematic Review

eHealth Tools That Assess and Track Health and Well-being in Children and Young People: Systematic Review

eHealth Tools That Assess and Track Health and Well-being in Children and Young People: Systematic Review

Review

1Brain and Mind Centre, The University of Sydney, Sydney, Australia

2Department of Psychiatry, The University of British Columbia, Vancouver, BC, Canada

Corresponding Author:

Elizabeth Stewart, BA, PhD, DPsych

Brain and Mind Centre, The University of Sydney

94-100 Mallett Street Camperdown

Sydney, 2050

Australia

Phone: 61 0404060549

Email: lizziestewart@gmail.com


Background: eHealth tools that assess and track health outcomes in children or young people are an emerging type of technology that has the potential to reform health service delivery and facilitate integrated, interdisciplinary care.

Objective: The aim of this review is to summarize eHealth tools that have assessed and tracked health in children or young people to provide greater clarity around the populations and settings in which they have been used, characteristics of digital devices (eg, health domains, respondents, presence of tracking, and connection to care), primary outcomes, and risks and challenges of implementation.

Methods: A search was conducted in PsycINFO, PubMed or MEDLINE, and Embase in April 2020. Studies were included if they evaluated a digital device whose primary purpose was to assess and track health, focused on children or young people (birth to the age of 24 years), reported original research, and were published in peer-reviewed journals in English.

Results: A total of 39 papers were included in this review. The sample sizes ranged from 7 to 149,329 participants (median 163, mean 5155). More studies were conducted in urban (18/39, 46%) regions than in rural (3/39, 8%) regions or a combination of urban and rural areas (8/39, 21%). Devices were implemented in three main settings: outpatient health clinics (12/39, 31%), hospitals (14/39, 36%), community outreach (10/39, 26%), or a combination of these settings (3/39, 8%). Mental and general health were the most common health domains assessed, with a single study assessing multiple health domains. Just under half of the devices tracked children’s health over time (16/39, 41%), and two-thirds (25/39, 64%) connected children or young people to clinical care. It was more common for information to be collected from a single informant (ie, the child or young person, trained health worker, clinician, and parent or caregiver) than from multiple informants. The health of children or young people was assessed as a primary or secondary outcome in 36% (14/39) of studies; however, only 3% (1/39) of studies assessed whether using the digital tool improved the health of users. Most papers reported early phase research (formative or process evaluations), with fewer outcome evaluations and only 3 randomized controlled trials. Identified challenges or risks were related to accessibility, clinical utility and safety, uptake, data quality, user interface or design aspects of the device, language proficiency or literacy, sociocultural barriers, and privacy or confidentiality concerns; ways to address these barriers were not thoroughly explored.

Conclusions: eHealth tools that assess and track health in children or young people have the potential to enhance health service delivery; however, a strong evidence base validating the clinical utility, efficacy, and safety of tools is lacking, and more thorough investigation is needed to address the risks and challenges of using these emerging technologies in clinical care. At present, there is greater potential for the tools to facilitate multi-informant, multidomain assessments and longitudinally track health over time and room for further implementation in rural or remote regions and community settings around the world.

J Med Internet Res 2022;24(5):e26015

doi:10.2196/26015

Keywords



In 2018, the United Nations Children’s Fund released a report on digital technologies in health [1] alongside its Strategic Plan, 2018-2021, which emphasized the importance of using digital (internet- and mobile-based) technology to facilitate health care for children and young people. At the same time, the Early Childhood Developmental Interventions Review Group for the Lancet issued a report making several recommendations for the improvement of assessments and interventions for children or young people [2]. These recommendations included improving the capacity for services to conduct multidomain and multi-informant assessments, connecting children or young people and families with personalized care options, and using digital solutions within health services to allow for broader-scale change [2]. Together, these reports highlight the emerging need to use digital technologies to enhance the delivery of health care for children or young people and their families.

Over the past decade, there has been a rapid growth in the development of digital tools in the health and well-being space [3]. These tools have served various purposes in health care, with the most common uses among children or young people being to deliver interventions (eg, clinician-assisted evidence-based treatments and self-monitoring and self-care), provide education, and facilitate communication for both consumers and clinicians (eg, telehealth or teleconferencing and online peer support groups) [2,4]. Another more recent use of eHealth has been to facilitate the assessment and triage of children or young people through health services [5-12]. These emerging technologies provide users (ie, clinicians and consumers) with secure, web-based platforms for submitting health data (sometimes automatically via biosensors or wearables) without having to be physically present in a hospital or health clinic. The information can be securely shared with health professionals with expertise in children or young people’s areas of need, allowing them to be triaged to appropriate services and connected with ongoing care [5,7-10,13,14]. Thus, these eHealth solutions differ from existing technologies in that their goal is not to deliver interventions or ongoing treatment but rather to facilitate a connection between consumers and pre-existing health services, allow for routine outcome monitoring, and place the person (or family) at the center of care. Furthermore, although some of these tools provide education resources (eg, fact sheets) or communication pathways (eg, web-based chats), they do this with the goal of triaging children or young people to appropriate care.

The literature on eHealth tools that assess and track health outcomes in children or young people is still in its infancy; however, a growing number of studies have reported on such devices over the past decade [4,15]. These tools differ in their health focus (eg, infectious diseases and mental health) [16,17] and locations in which they have been used (ie, rural or urban areas, high- or low-income countries, and specific health settings) [7,18-20]. There has also been variability in terms of the respondent who enters data into the tool (ie, clinician and consumer), the type of data (ie, questionnaires and physiological data), whether the tools have facilitated only assessment or assessment and tracking, and whether they have connected children or young people to clinical care. Given this variability, the specific features of eHealth tools, as well as their efficacy for improving health outcomes and clinical care delivery for children or young people, remain unclear.

Despite the potential benefits of eHealth solutions for children or young people, numerous challenges have been documented in their development, implementation, and uptake among other groups [10,13,21-24]. To be successful, the technologies must be user-friendly, engaging, and accessible to diverse populations. Issues of language, literacy, and culture have all been found to affect accessibility, uptake, and the quality of data [12,21,22,25]. The validity and integrity of data also depend on the availability of appropriately trained health care workers to enter or interpret information, emphasizing the importance of developing and evaluating these tools within the contexts in which they will be used. Finally, issues of privacy, confidentiality, and security are paramount to ensuring that the tools respect the rights of users and are likely to affect the uptake of these technologies [22,25,26].

To our knowledge, no comprehensive reviews have been conducted to examine the efficacy of eHealth tools that assess and track health outcomes in children or young people. As such, it is unclear in which health domains and settings these tools may have the potential to shape clinical care and, importantly, whether their use has been associated with improved health outcomes for children or young people. There is also a need to identify potential challenges and risks of using eHealth tools to ensure that best practice methods are established and consistently used [1,27]. Understanding the available eHealth solutions and their efficacies is critical for shaping future research and development efforts and ensuring efficient expansion of knowledge in this field.

The aim of this systematic review is to summarize eHealth tools that have been developed to assess and track health in children or young people to provide greater clarity about (1) the populations and settings in which these tools have been used (ie, locations, languages, and age groups); (2) characteristics of the tools (ie, health domains assessed, respondent, type of data, presence of tracking, and connection to care); (3) primary outcomes of the study, including whether the use of the tool has been associated with improved health outcomes; and (4) risks and challenges identified during implementation and evaluation.


Overview

The term eHealth has been variously defined in the literature. According to a systematic review, 51 unique definitions have been used for the term, without a clear consensus on a single definition, and the definitions differ in how inclusively they are conceptualized [28,29]. We have chosen to use a definition based on the conceptualization of eHealth offered by Vegesna et al [30] because of its relevance and consistency with the overarching aims of this review; digital technologies are thus defined as noninvasive digital devices that have been used to assess and track the health of a patient or consumer. We used the World Health Organization’s definition of childhood and youth as the period spanning birth to 24 years, whereby children are aged 0 to 9 years, and young people are aged 10 to 24 years [31].

Search Strategy

The search was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [32]. PsycINFO, PubMed or MEDLINE, and Embase were searched via OVID by 3 members of the research team (ES, HY, and AR) on April 27, 2020. The following terms were used ((child*) OR (adolescen*) OR (young person) OR (infan*)) AND ((wellbeing) OR (health)) AND (((digital tool) OR (digital AND tool)) OR (eHealth) OR ((mobile application) OR (mobile AND application))). A wildcard (*) was placed at the end of each applicable search term to ensure that all relevant terms were captured. All Medical Subject Heading terms were explored to broaden the search for relevant studies. Date limits were not set on any of the database searches. The reference lists of relevant reviews and identified empirical studies were searched to identify further studies, as per the ancestry method.

Study Selection Criteria

Studies were included if they met the following criteria:

  1. Included children and young people (birth to the age of 24 years) or their parents or carers, as per the World Health Organization’s definition [31]
  2. Evaluated a digital device, including internet- or mobile-based technology (ie, noninvasive digital devices, including internet- or mobile-based e-tools and wearable devices), the primary purpose of which was to assess or track the health of the child or young person
  3. Focused on a domain of health
  4. Evaluation studies, meaning the authors evaluated some aspects of the digital device, including effectiveness, validity, or feasibility; we included all or any type of evaluation studies, which were categorized according to the Center for Disease Control definition (ie, formative, process, and outcome) [33] and National Health and Medical Research Council criteria for study design [34]
  5. Reported original research
  6. Published in English in a peer-reviewed journal and included human participants

Studies were excluded if they had the following characteristics:

  1. Included adults only with no child or young person focus or if >25% of participants were outside our age criteria (birth to the age of 24 years)
  2. Evaluated a digital device that was primarily an interventional tool (ie, clinician-led and self-management tools), an educational device (eg, an e-course), a communication device (ie, assistive communication with images or written or spoken language; and teleconferencing only without additional assessment or tracking functionality), or digital technology that did not use internet or mobile technology (eg, electronic medical record systems)
  3. Reported results from development or description of the tool that had not yet been evaluated (eg, protocol papers)

Procedure

Figure 1 displays the process of study selection. The search retrieved 3688 papers, and an additional 16 papers were identified by searching the reference lists of relevant papers and reviews. Of these papers, 95.59% (3541/3704) remained after duplicates were removed. The titles and abstracts of all papers were screened by 2 reviewers (ES and HY). Of the 3541 papers, 84 (2.37%) full-text papers met the inclusion criteria and were obtained. A conservative approach was taken to ensure that relevant papers were not missed, and full-text papers were reviewed if the reviewers could not determine with certainty whether the inclusion criteria were met. The manuscripts of these 84 papers were reviewed by 2 independent raters (ES and HY), and discrepancies were resolved via discussion. Of the 84 papers, 45 (54%) papers were excluded, leaving 39 (46%) papers that were included in the review.

Figure 1. Flow diagram of identification and selection of studies.
View this figure

Data Extraction

The following information was extracted from each paper:

  1. Name of the first author, year of publication, and age range of children or young people
  2. Language or languages used in the digital application
  3. Location where the study was conducted: country, locality (urban or rural), and setting; locality was defined according to the Organization for Economic Co-operation and Development harmonized definition of global urbanization, which uses the population density of the area, that is, rural (<5000 inhabitants) or urban (≥5000 inhabitants) [35]; some studies were conducted in multiple locations, which was considered in categorizing study locality as urban, rural, or a mixture of urban and rural settings
  4. Characteristics of the digital tool: health domain assessed, respondent (parent or caregiver, child or young person, clinician, trained health worker, and other), device type (mobile, desktop, and tablet), type of data (questionnaire or survey, images, and physiological), whether the tool allowed for tracking over time (ie, data collected at multiple time points), and whether the device facilitated connection to care (ie, linking patients to health care providers or services)
  5. Study characteristics: type of evaluation study, defined according to the Center for Disease Control definition of study evaluation types, that is, formative, process, or evaluation [33]; study type: qualitative, quantitative, or mixed methods; and (3) study design, based on the National Health and Medical Research Council guidelines [34]
  6. The primary outcome and main findings from the study, including whether the health of the child or young person was measured as an outcome in the study
  7. Funding source, categorized as public sector (ie, government, universities, research institutes, and professional associations), commercial or not-for-profit (NFP) organizations; these categories were guided by an Australian Government resource on university research funding (REF)
  8. Any documented risks or challenges associated with the use of the eHealth tool

Data Analysis

Descriptive analyses were used to summarize variables of interest, including health domain, location, language, type of data, intended user, presence of certain features (ie, tracking over time and connection to care), and type of evaluation. Frequency data and percentages were used to examine and compare studies on key outcome measures. This approach to analysis was taken because of considerable variability in study objectives and designs and as most studies reported simple quantitative, descriptive statistics or qualitative findings.

Quality Appraisal of Studies

To evaluate the methodological quality of the studies, 2 checklists were used. The Downs and Black checklist [36] was completed for quantitative studies, which measures the quality of both randomized and nonrandomized studies evaluating novel health interventions. The National Institute for Health and Care Excellence Quality Appraisal Checklist was completed for studies reporting qualitative findings [37]. Studies reporting both qualitative and quantitative data were appraised using both checklists. A full description of the checklists and scoring criteria is included in Multimedia Appendix 1 [5-9,16-20,38-66].


Demographics of Studies

Table 1 summarizes the characteristics of the 39 studies included in this review. All (38/39, 97%) but a single (1/39, 3%) study was published in the past decade (2010-2020), and over one-third of the studies (15/39, 39%) were published in the past year (2019-2020; Figure 2). Most studies were conducted in a single country (35/39, 90%), most commonly America (7/39, 18%) or Australia (6/39, 15%). English was the sole language of communication in 49% (19/39) of studies; 13% (5/39) of studies evaluated tools that used English and at least one other language, and 21% (8/39) used languages other than English; the remaining 18% (7/39) of studies did not report enough information to determine which language was used in the tool. Regarding locality, studies were conducted in urban (18/39, 46%), rural (3/39, 8%), or a mixture of urban and rural settings (8/39, 21%); 26% (10/39) studies did not report enough information to determine locality. Digital devices were implemented across 3 main settings: outpatient health clinics (12/39, 31%), hospitals (ie, inpatient units and emergency departments; 14/39, 36%), and community outreach (ie, community spaces that were not formal health clinics; 10/39, 26%) or a combination of these settings (3/39, 8%).

Table 1. Demographic characteristics of studies.
StudyAge range of childrenCountryLocalityaLanguage used in the deviceSetting
Alawna et al, 2019 [50]19-27 years (mean 22.0)TurkeyNRbNROutpatient health clinic
Binotti et al, 2019 [56]Infants (age range NR)ItalyUrbanNRHospital
Boyce et al, 2019 [48]2-59 monthsMalawiNREnglishOutpatient health clinic
Den Boer et al, 2018 [18]3-17 yearsNetherlandsUrban (81%) and rural (19%)English, German, Spanish, and DutchOutpatient health clinic
Detsomboonrat and Pisarnturakit, 2019 [58]Children in primary school (age range NR)ThailandNRThaiCommunity outreach
Dexheimer et al, 2014 [19]2-18 yearsUnited StatesUrbanEnglishHospital
Eikelboom et al, 2005 [5]9 months-16 yearsAustraliaRuralEnglishOutpatient health clinic
Estai et al, 2016 [57]2-18 yearsAustraliaUrbanEnglishOutpatient health clinic
Finocchario-Kessler et al, 2015 [6]Children (age range NR)KenyaUrban (50%) and rural (50%)EnglishHospital
Franke et al, 2018 [20]18 months-14 yearsGhanaUrbanTwiHospital
Galvez et al 2017 [52]Children (age range NR)55 countries (worldwide)NREnglishHospital
Ginsburg et al, 2015 [16]Children (age range NR)GhanaUrbanEnglishOutpatient health clinic
Gregory et al, 2017 [39]<18 yearsUnited KingdomUrbanEnglishHospital
Han et al, 2019 [53]13-26 yearsChina and AustraliaUrbanNRHospital
Hashemi et al, 2017 [7]6-18 yearsGazaUrbanEnglish and ArabicCommunity outreach
Heida et al, 2018 [62]10-19 yearsNetherlandsUrban (55%) and rural (45%)DutchOutpatient health clinic
Hussey and Flynn, 2019 [41]0-21 yearsUnited StatesUrbanEnglishOutpatient health clinic
Iorfino et al, 2017 [8]16-24 yearsAustraliaUrban (85%) and rural (15%)EnglishOutpatient health clinic
Jeong et al, 2020 [40]15-19 yearsSouth KoreaUrbanKoreanHospital, outpatient health clinic, and community outreach
Jiam et al, 2017 [66]3-22 yearsUnited StatesNREnglishCommunity outreach
Kassam-Adams et al, 2019 [42]6-14 yearsUnited StatesUrban (50%) and rural (50%)EnglishHospital
Kim et al, 2019 [60]0-5 yearsSouth KoreaNRNRCommunity outreach
Li et al, 2019 [63]1-18 yearsChinaUrbanMandarinHospital
March et al, 2018 [17]5-12 yearsAustraliaUrbanEnglishOutpatient health clinic and community outreach
Matin et al, 2020 [59]0-7 daysUgandaRuralLusoga and EnglishCommunity outreach
McCulloh et al, 2018 [49]0-2 monthsUnited StatesNREnglishHospital
Mohammed et al, 2018 [9]0-5 yearsGhanaUrbanTwiCommunity outreach
Padidar et al, 2019 [64]0-9 daysIranUrbanNRHospital
Rath et al, 2018 [45]0-24 yearsGermanyNRArab, Farsi, and RussianCommunity outreach
Rath et al, 2019 [65]0-5 yearsGermany and GreeceUrbanNRHospital
Reid et al, 2011 [43]14-24 yearsAustraliaUrban (50%) and rural (50%)EnglishOutpatient health clinic
Singh et al, 2017 [51]0-2 yearsIndiaUrban (85%) and rural (15%)Hindi, Gujarati, and EnglishHospital and outpatient health clinic
Svedberg et al, 2019 [46]6-13 yearsSwedenUrban (50%) and rural (50%)SwedishHospital
Thabrew et al, 2019 [44]13-14 yearsNew ZealandNREnglishOutpatient health clinic
Thabtah, 2018 [54]0-17 years10 countriesNR11 languagesCommunity outreach
Thompson et al, 2016 [47]12-18 yearsUnited StatesUrbanEnglishOutpatient health clinic
Valdes-Angues et al, 2018 [55]3-18 yearsUganda and United StatesRuralEnglishCommunity outreach
van Karnebeek et al, 2012 [61]0-18 yearsCanadaUrbanEnglishHospital
Wang et al, 2017 [38]5-17 yearsChinaUrbanEnglish and ChineseOutpatient health clinic

aLocality: region in which the eHealth tool was implemented, defined as rural (<5000 inhabitants) or urban (≥5000 inhabitants), according to the Organization for Economic Co-operation and Development’s harmonized definition of global urbanization [35].

bNR: not reported.

Figure 2. Cumulative number of studies published each year.
View this figure

Characteristics of eHealth Tools

Table 2 summarizes the characteristics of the studied digital devices.

Table 2. Device characteristics.
StudyHealth domainDeviceType of dataRespondentTracking over timeConnection to care
Alawna et al, 2019 [50]General healthMobilePhysiologicalTrained health workeraUnclearYes
Binotti et al, 2019 [56]DevelopmentalMobilePhysiologicalTrained health workerNoNo
Boyce et al, 2019 [48]General healthMobileQuestionnaire or surveyTrained health workerNoYes
Den Boer et al, 2018 [18]Oral healthMobileQuestionnaire or surveyClinicianb and child or young personcNoYes
Detsomboonrat and Pisarnturakit 2019 [58]OralMobile and desktopQuestionnaire or surveyClinicianNoYes
Dexheimer et al, 2014 [19]General healthDesktopQuestionnaire or surveyClinicianYesYes
Eikelboom et al, 2005 [5]Ear, nose, and throatDesktopImagesClinicianNoYes
Estai et al, 2016 [57]Oral healthDesktopImagesClinician and trained health workerNoNo
Finocchario-Kessler et al, 2015 [6]InfectiousDesktopPhysiologicalParent or caregiverd and trained health workerYesYes
Franke et al, 2018 [20]InfectiousMobileQuestionnaire or surveyParent or caregiverNoYes
Galvez et al, 2017 [52]EmergencyMobileQuestionnaire or surveyClinicianYesNo
Ginsburg et al, 2015 [16]InfectiousMobilePhysiologicalTrained health workerNoYes
Gregory et al, 2017 [39]Mental healthMobileQuestionnaire or surveyClinician and child or young personNoYes
Han et al, 2019 [53]VisionMobilePhysiologicalChild or young personNoNo
Hashemi et al, 2017 [7]Mental healthDesktop and mobileQuestionnaire or surveyTrained health workerNoNo
Heida et al, 2018 [62]Physical healthDesktopQuestionnaire or survey and physiologicalChild or young person and parent or caregiverYesYes
Hussey and Flynn, 2019 [41]Mental healthMobileQuestionnaire or surveyClinician and child or young personYesYes
Iorfino et al, 2017 [8]Mental healthDesktopQuestionnaire or surveyChild or young personYesYes
Jeong et al, 2020 [40]Mental healthMobileQuestionnaire or surveyClinician and child or young personNoYes
Jiam et al, 2017 [66]NeurologicalDesktopQuestionnaire or surveyParent or caregiver and child or young personYesNo
Kassam-Adams et al, 2019 [42]Mental healthMobileQuestionnaire or surveyChild or young personYesNo
Kim et al, 2019 [60]InfectiousMobileQuestionnaire or survey and physiologicalParent or caregiverYesNo
Li et al, 2019 [63]SurgeryMobileQuestionnaire or surveyChild or young personNoYes
March et al, 2018 [17]Mental healthDesktop, mobile, and tabletQuestionnaire or surveyClinician, parent or caregiver, education provider, and child or young personNoNo
Matin et al, 2020 [59]DevelopmentalMobileQuestionnaire or survey and physiologicalParent or caregiverYesYes
McCulloh et al, 2018 [49]General healthMobileQuestionnaire or surveyClinicianNoYes
Mohammed et al, 2018 [9]General healthMobileQuestionnaire or surveyParent or caregiverNoYes
Padidar et al, 2019 [64]DevelopmentalMobilePhysiological imagesClinician and parent or caregiverNoNo
Rath et al, 2018 [45]General healthMobile tabletQuestionnaire or surveyChild or young person and parent or caregiverNoNo
Rath et al, 2019 [65]InfectiousMobileQuestionnaire or surveyChild or young personNoNo
Reid et al, 2011 [43]Mental healthDesktop and mobileQuestionnaire or surveyChild or young personYesYes
Singh et al, 2017 [51]General healthDesktop and mobileQuestionnaire or survey and physiologicalClinician and parent or caregiverYesYes
Svedberg et al, 2019 [46]General healthMobileQuestionnaire or surveyChild or young personYesYes
Thabrew et al, 2019 [44]Mental healthMobile and tabletQuestionnaire or surveyChild or young personNoYes
Thabtah, 2018 [54]DevelopmentalMobileQuestionnaire or surveyClinician and parent or caregiverNoNo
Thompson et al, 2016 [47]General healthDesktopQuestionnaire or survey and physiologicalParent or caregiver and young personYesYes
Valdes-Angues et al, 2018 [55]NeurologicalDesktop and mobileQuestionnaire or surveyTrained health workerYesYes
van Karnebeek et al, 2012 [61]DevelopmentalDesktop, mobile, and tabletQuestionnaire or surveyClinicianNoNo
Wang et al, 2017 [38]General health and mental healthMobileQuestionnaire or surveyClinician, parent or caregiver, and child or young personYesYes

aTrained health workers are staff without professional training who received specific training in the use of the digital tool and associated health domain.

bClinician is defined as a health professional with qualifications in a particular field of practice (including medical doctors and allied health workers).

cChild or young person is the individual for whom the eHealth tool was developed.

dParent or caregiver is the primary carer of the child or young person.

Health Domains

Mental and general health were the most common eHealth domains assessed, with each evaluated in 26% (10/39) of studies. Other health domains assessed included child development (5/39, 13%), infectious diseases (5/39, 13%), oral health (3/39, 8%), neurological illnesses (2/39, 5%), ear nose and throat (1/39, 3%), emergency medicine (1/39, 3%), physical health (1/39, 3%), vision (1/39, 3%), and pediatric surgery (1/39, 3%). A single study assessed multiple health domains (mental and general health) [38]. Given that mental health was more commonly assessed than other health domains, we examined these studies further to determine their aim or purpose and the type of information collected. Of the 23% (9/39) of studies that solely assessed mental health, 33% (3/9) focused on suicide prevention [8,39,40], 22% (2/9) focused on early intervention and prevention of mental illness [7,17], and 44% (4/9) focused on multidimensional assessment or management of mental health symptoms [41-44]. Of the 23% (9/39) of studies that solely assessed general health, 22% (2/9) focused on symptom detection and monitoring [9,45], 22% (2/9) provided a platform for patients to view and monitor their health information [46,47], 33% (3/9) focused on digitalized tracking of clinical decision-making [19,48,49], and 22% (2/9) were primarily for assessment [50,51].

Data Collection: Respondent, Type of Data, and Device

All devices measured the health of a child or young person; however, devices differed in the person who entered the health information (ie, the respondent: child or young person, parent or caregiver, clinician, and trained health worker). Under half of the devices collected information from multiple respondents (16/39, 41%); other tools collected information solely from a child or young person (8/39, 21%), clinician (6/39, 15%), trained health worker (6/39, 15%), or parent or caregiver (3/39, 8%). Approximately 15% (6/39) of studies collected data in multiple forms (ie, questionnaire or survey, physiological data, or images); otherwise, data were collected solely in the form of questionnaires or surveys (26/39, 67%), physiological data (5/39, 13%), or images (2/39, 5%). Most eHealth tools (31/39, 80%) were configured to collect data on a mobile phone, of which some (9/39, 23%) were also configured to collect data on another device (ie, desktop or tablet).

Device Features: Health Tracking and Connection to Care

Just under half of the devices tracked children’s health over time (16/39, 41%), and two-thirds (25/39, 64%) connected children or young people to clinical care, whereas the remainder did not.

Outcome Evaluation: Primary Outcome Measures and Findings

Table 3 summarizes the sample size, type of evaluation, study type and design, and primary outcomes, and a more detailed description of the main findings for each study is presented in Multimedia Appendix 1 (see Table S1). The sample sizes ranged from 7 to 149,329 participants (median 163, mean 5155). Most studies were formative (20/39, 51%) or process (11/39, 28%) evaluations, with fewer outcome evaluation studies (8/39, 21%). Just over one-third of the studies (14/39, 36%) assessed the health of children or young people as either a primary or secondary outcome; however, only a single (1/14, 7%) study assessed whether using the digital tool improved the health of children or young people [43]. This study examined whether the use of Mobiletype, an eHealth tool that allowed general practitioners and young people to monitor symptoms of mood, stress, and daily activities in general practice, was associated with improved mental health outcomes compared with treatment as usual. The authors found that use of the device was associated with a significant improvement in emotional self-awareness but found no changes in symptoms of depression, anxiety, or stress; post hoc analyses showed enhanced mental health care at the initial assessment among general practitioners using the tool compared with those who did not.

Table 3. Outcomes of studies.
StudySample (N)Type of evaluationaStudy typeStudy designbHealth as outcomecPrimary outcomeChallenges or risks of using the tools
Alawna et al, 2019 [50]58FormativeQuantitativeDescriptive studyNoReliability (intra- and interrater reliability)
  • Clinical utility: questionable accuracy of readings in people with certain health conditions (eg, obesity and limb deformity)
Binotti et al, 2019 [56]40FormativeQuantitativeDescriptive studyNoConcordance ratingd
  • Clinical safety: partial overestimation of heart rate when <60 beats per minute
Boyce et al, 2019 [48]799ProcessMixed methodsQuasi-experimentalNoEfficacy
  • Accessibility: hardware and software issues (eg, uploading data)
  • Uptake: time consuming
Den Boer et al, 2018 [18]653FormativeMixed methodsDescriptive studyYesUsability and efficacy
  • Accessibility: slow internet connection
  • UXe: buttons lacked visual response to input
  • Sociocultural: parents or carers said questions about smoking for children aged 6-11 years were inappropriate and insulting
Detsomboonrat and Pisarnturakit 2019 [58]441FormativeQuantitativeDescriptive studyNoAcceptability and efficacy
  • Accessibility: poor internet connection for some users
Dexheimer et al, 2014 [19]13,896OutcomeQuantitativeRCTfNoEfficacy (time from triage to clinical decision)
  • Clinical utility: clinicians were already implementing best practice guidelines and conducting education without the eHealth tool
Eikelboom et al, 2005 [5]66FormativeQuantitativeDescriptive studyNoConcordance rating
  • Data quality: poor image quality
  • Clinical safety: using eHealth tool alone (without input from a qualified clinician) could result in inaccurate diagnosis and treatment
Estai et al, 2016 [57]126FormativeQuantitativeDescriptive studyNoConcordance rating
  • Data quality: poor image quality
Finocchario-Kessler et al, 2015 [6]NRgOutcomeMixed methodsCross-sectional studyNoFeasibility and efficacy
  • Accessibility: slow internet connection in some regions
  • Language proficiency or literacy: some users unable to use the tool because of low literacy levels
  • Privacy: concerns about the privacy of data
  • Clinical safety: high turnover of health care workers requiring continuous retraining of staff or risk of inaccurate use of the tool
Franke et al, 2018 [20]237ProcessQuantitativeCross-sectional studyNoConcordance rating
  • Clinical utility: data only entered by parent or caregiver and mostly in binary (yes or no) format; information from clinician said to be important but not possible as multi-informant assessment not available
Galvez et al, 2017 [52]1252ProcessQuantitativeDescriptive studyNoUse and uptake
  • Accessibility: only available in countries with internet access and where Google was not blocked
Ginsburg et al, 2015 [16]7FormativeMixed methodsDescriptive studyNoUsability and acceptability
  • UX: buttons difficult to navigate, pop-ups distracting, difficulty launching application and recording results, too text heavy or more images needed
Gregory et al, 2017 [39]76FormativeQuantitativeDescriptive studyNoFeasibility of uptake
  • Uptake: lower than expected uptake by young people
Han et al, 2019 [53]150OutcomeQuantitativeCohort studyYesValidity and reliability
  • Data quality: mobile phones with low resolution may not clearly show results
Hashemi et al, 2017 [7]986OutcomeQuantitativeDescriptive studyYesFeasibility
  • Clinical safety: efficacy of the tool in screening for psychological symptoms not yet validated
Heida et al, 2018 [62]170OutcomeMixed methodsRCTYesEfficacy
  • Uptake: clinicians not adequately prepared for changes in traditional ways of working and reluctant to enter data twice
Hussey and Flynn, 2019 [41]56FormativeMixed methodsComparative study with historical control groupNoUse and efficacy
  • UX: many features needing improvement (eg, emergency alert button, survey tool, SMS text messaging, and notifications)
Iorfino et al, 2017 [8]232ProcessQuantitativeNonrandomized experimental trialYesEfficacy
  • Clinical utility or safety: efficacy for individuals with low to moderate suicidality not studied
Jeong et al, 2020 [40]13FormativeMixed methodsDescriptive studyNoFeasibility, acceptability, and usability
  • Accessibility: health professionals unable to use the tool because of inadequate training
Jiam et al, 2017 [66]7ProcessQualitativeDescriptive studyNoUsability
  • Language proficiency or literacy: information beyond children’s comprehension capacity and literacy levels
Kassam-Adams et al, 2019 [42]167ProcessQuantitativeDescriptive studyNoAcceptability and efficacy
  • Accessibility: lower-income families could not use the tool because of the cost of mobile data
Kim et al, 2019 [60]149,329ProcessMixed methodsDescriptive studyNoUptake, usability, and efficacy
  • Uptake: of the 3 countries where the tool was implemented, uptake was only seen in Korea and not China or Japan
  • Accessibility: only users with a smartphone could use the eHealth tool
  • Clinical utility: question as to whether increased rates of influenza signaled a local outbreak or new interest in using the tool
Li et al, 2019 [63]137OutcomeQuantitativePseudo-RCTYesUtility and efficacy
  • NR
March et al, 2018 [17]18FormativeMixed methodsDescriptive studyNoFeasibility and acceptability
  • NR
Matin et al, 2020 [59]18FormativeQuantitativePretest–posttest case seriesNoFeasibility and acceptability
  • Clinical utility: did not assess parents’ accuracy in identifying symptoms aided by the tool; outside of the research study, parents may not receive the same on-call support
  • Clinical safety: only 1 parent attached the wearable band correctly, leading to many incorrect recordings
  • UX: device lacked notifications to encourage care seeking when necessary
McCulloh et al, 2018 [49]3805FormativeMixed methodDescriptive studyNoUse or uptake and usability
  • NR
Mohammed et al, 2018 [9]1446FormativeQuantitativeDescriptive studyYesFeasibility and concordance ratings
  • Accessibility: poor internet connection in some areas; low ownership of mobile phones
  • Data quality: incomplete data entered by some parents
Padidar et al, 2019 [64]113FormativeQuantitativeDescriptive studyYesEfficacy (concordance rating)
  • NR
Rath et al, 2018 [45]405FormativeMixed methodsDescriptive studyYesUsability and efficacy
  • Clinical utility: the anonymity of users prevented verification of health conditions and initiation of follow-up care
Rath et al, 2019 [65]1615FormativeQuantitativeCohort studyYesEfficacy
  • NR
Reid et al, 2011 [43]163OutcomeQuantitativeRCTYesChange in mental health status
  • NR
Singh et al, 2017 [51]16,490ProcessQuantitativeDescriptive studyYesFeasibility
  • Language proficiency or literacy: many parents could not read English messages (Hindi translations integrated to address this issue)
  • Uptake: clinicians and parents were initially resistant to use the new digital system
  • Data quality: errors in data entry related to free text input
  • Clinical utility: customization of question sets needed depending on user characteristics
Svedberg et al, 2019 [46]46ProcessQualitativeDescriptive studyNoFeasibility and acceptability
  • Uptake: low uptake because of required organizational restructuring and competing workplace demands (eg, high workload)
  • UX: software issues related to printing reports and unwanted termination of sessions
Thabrew et al, 2019 [44]129FormativeMixed methodsPseudo-RCTNoEfficacy and acceptability
  • Accessibility: some internet connection issues
  • Language proficiency or literacy: information beyond the comprehension and literacy levels of some low socioeconomic groups
Thabtah, 2018 [54]1452OutcomeQuantitativeDescriptive studyYesFeasibility and efficacy
  • NR
Thompson et al, 2016 [47]937ProcessQuantitativeDescriptive studyNoUse and uptake
  • NR
Valdes-Angues et al, 2018 [55]326ProcessMixed methodsDescriptive studyNoFeasibility
  • Accessibility: poor internet connection; power cuts; inability to recharge device; slow upload speed of data
  • Data quality: errors in data entry
van Karnebeek et al, 2012 [61]15FormativeQualitativeDescriptive studyYesFeasibility and acceptability
  • Clinical utility: additional features needed to add value to standard care (eg, entering differential diagnosis and accessing databases with medical information)
Wang et al, 2017 [38]31FormativeQualitativeDescriptive studyNoUsability
  • NR

aType of evaluation defined as follows: (1) formative evaluation: assessed feasibility, appropriateness, or acceptability of the digital device before full implementation; (2) process evaluation: assessed whether the digital device had been implemented as intended; (3) outcome evaluation: measured the effectiveness of the digital device by assessing progress in primary outcomes [33].

bStudy design based on the National Health and Medical Research Council guidelines: randomized controlled trials (RCTs); pseudo-RCTs; comparative studies with concurrent controls, including nonrandomized experimental trials, cohort studies, case–control studies, or interrupted time series with a control group; comparative studies without a control group, including historical control studies, ≥2 single-arm studies or interrupted time series without a parallel control; case series with either posttest or pre- and posttest outcomes; descriptive studies; or other [34].

cWhether an aspect of the child’s or young person’s health was measured as a primary or secondary outcome of the study.

dThe amount of agreement between the digital tool and clinician ratings.

eUX: user experience (user interface or design aspects of the device).

fRCT: randomized controlled trial.

gNR: not reported.

Challenges and Risks Identified in Studies

Table 3 summarizes the challenges and risks of using the tools identified in each study. Of the studies examined, most (30/39, 77%) identified at least one challenge or risk, which was related to accessibility (11/39, 28%), clinical utility (9/39, 23%) or clinical safety (5/39, 13%) of the tool, uptake by users (6/39, 15%), data quality (6/39, 15%), user interface or design aspects of the device (user experience; 5/39, 13%); language proficiency or literacy barriers (4/39, 10%), sociocultural barriers (1/39, 3%), and privacy concerns (1/39, 3%). More specifically, accessibility problems were related to poor internet connection, inability to recharge devices because of power cuts, slow or inefficient upload of information, lack of access to a device, and low technological literacy of end users. Clinical utility and clinical safety concerns were related to the validity of data among people with different health conditions, lack of appropriate training of staff, input from a health care professional rather than entirely self-report data to ensure safe and accurate interpretation of results, whether the tool added value over and above standard clinical care, and the safety of tools that had not yet been validated to detect clinical symptoms. Uptake of tools was a frequently cited barrier; however, there was often no further investigation or explanation as to why uptake was lower than expected. Data quality concerns were centered on inaccurate or incomplete data entry (because of human or computer error) and poor-quality images. User experience or design barriers referred to the eHealth tool lacking the necessary features to make it functional and usable for end users. Language proficiency or literacy barriers were centered on users lacking the comprehension and literacy levels to understand and take action from the presented information; this was a concern reported when end users were children, non–English speaking, or from lower socioeconomic backgrounds. Sociocultural barriers were related to the appropriateness of questions and the risk of causing offense or harm.

Research Funding

Studies were financially supported by the public sector (ie, government, universities, research institutes, and professional associations) and commercial or NFP organizations (28/39, 72%). Receiving funding from ≥1 sector was the most common (16/39, 41%), followed by funding solely from public sources (6/39, 15%) and NFPs (5/39, 13%). No study was funded solely by the commercial sector; however, commercial funding contributed to nearly one-third of studies with combined funding sources (5/16, 31%). The remainder of the studies did not receive external financial support or did not report it in the paper (11/39, 28%).

Quality Ratings of Selected Papers

The methodological quality of the Downs and Black checklist was rated for 95% (35/39) of studies that included quantitative data: 64% (25/39) of studies had a low chance of bias, 36% (14/39) of studies had a moderate chance of bias, and no studies had a high chance of bias (see Table S2, Multimedia Appendix 1). The National Institute for Health and Care Excellence Quality Appraisal Checklist was completed for 44% (17/39) of studies that included qualitative data: 59% (10/17) of studies received a maximum score of 2 for quality, and 41% (7/17) of studies received a partial score of 1; no studies received a score of 0 (see Table S3, Multimedia Appendix 1).


Principal Findings

The aim of this systematic review was to summarize eHealth tools designed to assess and track health outcomes in children and young people to clarify the current scope, nature, and efficacy of this emerging type of technology in health care. Our findings revealed exponential growth in the development and evaluation of these tools over the past 10 years; however, the results showed that the research is still in its infancy, with most studies assessing feasibility, acceptability, usability, or uptake of a device rather than the efficacy of tools in relation to health outcomes. Overall, the current tools showed potential to enhance the assessment and tracking of children or young people in health services around the world. Further research is needed to evaluate the efficacy of tools for improving health outcomes and clinical care delivery, as well as to identify and address the risks and challenges of implementing these tools as part of standard clinical care.

There are numerous potential advantages of using eHealth solutions for children and young people, including the ability to conduct multidomain and multi-informant assessments, undertake continuous monitoring, and assist with timely connection to personalized clinical care [1,2,13]. Encouragingly, over half of the tools facilitated a connection between the child or young person and a health care provider; however, less than half tracked children’s health data over time or collected information from multiple informants (ie, child or young person, parent or caregiver, and health care professional). These findings demonstrate an untapped potential of eHealth solutions in facilitating multi-informant assessments and longitudinally tracking health over time among children or young people, which is key to achieving comprehensive, multidisciplinary care [2]. In addition, data were most commonly collected in the form of surveys or questionnaires, illustrating a lag in uptake and integration of newer technologies (eg, biosensors or wearables to collect physiological data). Such technology has the potential to enhance symptom detection and clinical decision-making [13,14,19,48,49] and may be an important area for future research to explore.

Another potential advantage of eHealth is its ability to overcome geographical, financial, and social barriers that hinder the provision of health services in specific populations and locations [10,67]. A small number of studies evaluated devices that were implemented across multiple countries [45,52-55], highlighting the ability of digital technologies to provide health care with greater reach. However, fewer tools were implemented in rural areas compared with urban areas, and there was less implementation in community outreach settings compared with hospitals and health clinics. Although using eHealth tools in health clinics and hospitals is a step forward from traditional paper-based methods in terms of data management and integrated care, there is greater potential for the tools to engage hard-to-reach populations in regional and community settings [9,55]. The higher percentage of devices used in health clinics and hospitals may be as community settings do not always incorporate systematic health tracking into their procedures or reflect a lack of availability of skilled health professionals to collect and enter health information in community settings. A number of studies overcame this issue by using trained health workers (ie, staff who received specific training in the health condition and digital device but were not specialists in the field); these trained health workers were able to collect information in the community, with studies finding that this did not compromise the validity or reliability of data or clinical care [6,7,48,50,56,57]. Another solution was to collect information solely from the consumer (ie, child or young person or their parent or guardian), which is particularly common in studies examining mental health [8,43,44]. Together, these results demonstrate room for broader implementation in rural or remote regions and community settings around the world. Nevertheless, an important issue to recognize is that rural and vulnerable populations are currently experiencing the largest digital divide [68-70]. Inequalities in access exist because of variations in location, age, education, and income level. For instance, the cost of internet access is higher in rural or remote regions than in urban areas, which is compounded by the fact that some rural residents have less disposable income than their metropolitan counterparts. Thus, to truly overcome geographical, financial, and social barriers and reach these populations, researchers must consider the broader socioeconomic context from which these access issues stem.

The eHealth tools studied focused on various domains of health, including infectious diseases, child development, and neurological conditions; however, the most widely assessed domains were general and mental health. The focus on mental health tools may reflect a growing need and demand for mental health care among children and young people, increasing evidence supporting eHealth in the field of mental health, and increased funding for mental health tools [71,72]. Only one of the eHealth tools assessed multiple health domains [38], despite this often being important for gaining a holistic picture of a child or young person’s health concerns. The development of eHealth tools that assess health multidimensionally is likely to be important in future eHealth tools, perhaps assessing not only current symptoms but also broader social or environmental factors related to the etiology and trajectory of illness and barriers to or facilitators of accessing care [10,13].

Various challenges and risks were identified in relation to the implementation and use of eHealth tools. These barriers were related to the accessibility and functionality of devices, including poor internet connection [6,9,15,18,44,48,52,55,58] and user interface or design aspects of the tool [16,18,41,46,59]. Clinical utility was another barrier identified in papers, mainly relating to the tool lacking features that provided added value to standard care [8,19,45,51,59-61]. User uptake was a challenge, with multiple studies reporting lower than expected uptake and reluctance to use the digital tool; this was an issue reported among diverse user groups, including clinicians, parents or carers, and children or young people [39,46,48,51,59,60,62]. These findings are in line with the Eysenbach [73] law of attrition, which is based on the observation that high participant dropout rates are common in eHealth research focusing on novel digital health tools; although researchers may dismiss or underreport this information, the observation meaningfully reflects the real-world uptake of digital tools currently. Some of the reasons for low uptake included implementation barriers (eg, competing time, modified professional roles, and organizational restructuring) [46,48,62], privacy concerns [6], socioeconomic factors (eg, cost of data) [42], and language proficiency or literacy issues [6,44,51]. Incorporating participatory design (co-design) and user testing methodologies into future protocols may help to understand and address these barriers [10,22]. Data quality was another barrier that was reported, which was related to human error in data entry [51,55] or incomplete data input [9]. A study overcame the issue of human error by minimizing free-text input and using predefined options [51]. Although this is not feasible for all tools, such as when obtaining qualitative health information, it provides a solution for quantitative health data. Sociocultural issues were mentioned in just 1 study; Den Boer [18] reported that parents or carers in some communities found questions about smoking in children aged 6 to 11 years insulting and inappropriate. The study researchers justified the inclusion of the questions by saying that they were important and relevant for certain communities or user groups. This raises the issue of whether universal questionnaires can be used in eHealth tools or whether customized question sets need to be developed for the target group. The findings of Singh [51] supported a configurable or individualized approach, with the researchers concluding that individualization was critical to the clinical utility and safety of eHealth tools. Despite studies identifying challenges and risks, there was minimal discussion on how to address the identified issues; moreover, just under a quarter of studies did not report any potential risks or challenges of implementing the device [17,38,43,47,49,54,63-65].

Our review of funding sources, which showed that financial support came from a mixture of public, commercial, and NFP bodies, is unsurprising, as the development and implementation of digital tools often involve the collaboration of professionals from multiple disciplines who belong to different bodies [25,74]. Interestingly, funding from commercial bodies was uncommon. This may reflect the fact that the tools reviewed were in the preliminary stages of research (development and implementation) rather than at a more advanced stage of commercialization, the latter of which we would expect to attract more investment from commercial organizations [74].

Recommendations for Future Research

The findings of this review demonstrate a clear need for further research into the efficacy and validity of eHealth tools that assess and track health outcomes in children and young people. Future evaluation studies should focus on changes in the health outcomes of users, as well as the clinical care pathways. Further investigation of the risks and challenges of implementing devices is also important, particularly relating to sociocultural factors, language proficiency or literacy, and privacy concerns, as these were seldom mentioned but are likely to affect the clinical utility, safety, and uptake of tools [10,23]. Overall, these findings are consistent with results from a prior systematic review of eHealth solutions in adults, which found a gap between the postulated and empirically demonstrated benefits of eHealth technologies, a lack of robust research trials into validity and efficacy, and inadequate investigation of risks or challenges of using these technologies in health care [75]. This review has uncovered several features of eHealth tools that may facilitate comprehensive assessments and integrated care in future technologies:

  1. Capacity for multi-informant assessment, including input from a health professional and the child or young person or their parent or caregiver
  2. Multidomain assessments, allowing for a holistic picture of the child or young person’s health to be captured rather than assessing health in one domain
  3. Tracking over time (ie, capacity and use of tools for assessment at multiple time points)
  4. Configurability of question sets or content depending on characteristics (eg, demographic, sociocultural, and health concerns) of the target group
  5. Connection to clinical care that is tailored to the child or young person’s current needs
  6. Trialing integration of newer technologies (eg, biosensors or wearables to collect physiological data) for relevant health domains

Limitations

Although this review provides important insights into a novel field of eHealth, the conclusions that can be drawn about the efficacy and validity of eHealth solutions are limited as most studies were formative and process evaluations that assessed feasibility, acceptability, usability, or uptake of a device. Outcome evaluation studies were rare, with just 3 randomized controlled trials conducted to date. These early phase research studies are necessary precursors to more rigorous validity and efficacy studies; however, they need to be followed by more thorough evaluation studies to determine whether the tools are effective in improving health outcomes and clinical care. We limited our search to studies published in English, which may have biased our results. Furthermore, although the strength of this review is that it presents the state of eHealth tools for supporting health in children or young people, it inevitably fails to consider the immense variation that lies within each health domain. Our search strategy was not without limitations. We did not include all relevant terms (eg, internet and technology) as the inclusion of these broader terms returned >15,000 articles, which was not considered realistic for screening. Nevertheless, we believe that the search strategy balanced scientific rigor and feasibility and was sufficiently rigorous to pick up relevant articles. Finally, this paper was not preregistered with PROSPERO; however, the search strategy remained the same over time.

Conclusions

eHealth tools that assess and track health outcomes in children or young people and connect individuals with personalized care options have enormous potential in health services around the world. Many of the existing tools are in the early stages of pilot and feasibility testing; however, the literature is promising in the potential to use these tools in future clinical care. Further research is needed to evaluate the validity and efficacy of these eHealth tools and investigate the potential risks and challenges of implementation as part of standard clinical care. With future research and development efforts in place, these tools have the potential to facilitate collaborative decision-making, improved communication, transmission of remote health data, and real-time assessment and tracking and take a positive step forward in digitalizing health practices.

Conflicts of Interest

TD is now Director, Research & Insights at the Australian Digital Health Agency.

Multimedia Appendix 1

Digital review.

DOCX File , 72 KB

  1. UNICEF Annual Report 2017. UNICEF. 2018.   URL: https://www.unicef.org/reports/unicef-annual-report-2017 [accessed 2021-12-19]
  2. Britto PR, Lye SJ, Proulx K, Yousafzai AK, Matthews SG, Vaivada T, Early Childhood Development Interventions Review Group‚ for the Lancet Early Childhood Development Series Steering Committee. Nurturing care: promoting early childhood development. Lancet 2017 Jan 07;389(10064):91-102. [CrossRef] [Medline]
  3. Lupton D. Health promotion in the digital era: a critical commentary. Health Promot Int 2015 Mar;30(1):174-183. [CrossRef] [Medline]
  4. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual Research Review: digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef] [Medline]
  5. Eikelboom RH, Mbao MN, Coates HL, Atlas MD, Gallop MA. Validation of tele-otology to diagnose ear disease in children. Int J Pediatr Otorhinolaryngol 2005 Jun;69(6):739-744. [CrossRef] [Medline]
  6. Finocchario-Kessler S, Odera I, Okoth V, Bawcom C, Gautney B, Khamadi S, et al. Lessons learned from implementing the HIV infant tracking system (HITSystem): a web-based intervention to improve early infant diagnosis in Kenya. Healthc (Amst) 2015 Dec;3(4):190-195 [FREE Full text] [CrossRef] [Medline]
  7. Hashemi B, Ali S, Awaad R, Soudi L, Housel L, Sosebee SJ. Facilitating mental health screening of war-torn populations using mobile applications. Soc Psychiatry Psychiatr Epidemiol 2017 Jan;52(1):27-33. [CrossRef] [Medline]
  8. Iorfino F, Davenport TA, Ospina-Pinillos L, Hermens DF, Cross S, Burns J, et al. Using new and emerging technologies to identify and respond to suicidality among help-seeking young people: a cross-sectional study. J Med Internet Res 2017 Jul 12;19(7):e247 [FREE Full text] [CrossRef] [Medline]
  9. Mohammed A, Franke K, Okyere P, Brinkel J, Marinovic A, Kreuels B, et al. Feasibility of Electronic Health Information and Surveillance System (eHISS) for disease symptom monitoring: a case of rural Ghana. PLoS One 2018 May 24;13(5):e0197756 [FREE Full text] [CrossRef] [Medline]
  10. Hickie IB, Davenport TA, Burns JM, Milton AC, Ospina-Pinillos L, Whittle L, et al. Project Synergy: co-designing technology-enabled solutions for Australian mental health services reform. Med J Aust 2019 Oct;211 Suppl 7:3-39. [CrossRef] [Medline]
  11. Davenport TA, LaMonica HM, Whittle L, English A, Iorfino F, Cross S, et al. Validation of the InnoWell platform: protocol for a clinical trial. JMIR Res Protoc 2019 May 31;8(5):e13955 [FREE Full text] [CrossRef] [Medline]
  12. Rowe SC, Davenport TA, Easton MA, Jackson TA, Melsness J, Ottavio A, et al. Co-designing the InnoWell Platform to deliver the right mental health care first time to regional youth. Aust J Rural Health 2020 Apr;28(2):190-194. [CrossRef] [Medline]
  13. Hickie IB, Scott EM, Cross SP, Iorfino F, Davenport TA, Guastella AJ, et al. Right care, first time: a highly personalised and measurement-based care model to manage youth mental health. Med J Aust 2019 Nov;211 Suppl 9:3-46. [CrossRef] [Medline]
  14. Iorfino F, Cross SP, Davenport T, Carpenter JS, Scott E, Shiran S, et al. A digital platform designed for youth mental health services to deliver personalized and measurement-based care. Front Psychiatry 2019 Aug 23;10:595 [FREE Full text] [CrossRef] [Medline]
  15. Dunton GF, Liao Y, Intille SS, Spruijt-Metz D, Pentz M. Investigating children's physical activity and sedentary behavior using ecological momentary assessment with mobile phones. Obesity (Silver Spring) 2011 Jun;19(6):1205-1212 [FREE Full text] [CrossRef] [Medline]
  16. Ginsburg AS, Delarosa J, Brunette W, Levari S, Sundt M, Larson C, et al. mPneumonia: development of an innovative mHealth application for diagnosing and treating childhood pneumonia and other childhood illnesses in low-resource settings. PLoS One 2015 Oct 16;10(10):e0139625 [FREE Full text] [CrossRef] [Medline]
  17. March S, Day J, Zieschank K, Ireland M. The interactive child distress screener: development and preliminary feasibility testing. JMIR Mhealth Uhealth 2018 Apr 19;6(4):e90 [FREE Full text] [CrossRef] [Medline]
  18. den Boer JC, van Dijk W, Horn V, Hescot P, Bruers JJ. Collecting standardised oral health data via mobile application: a proof of concept study in the Netherlands. PLoS One 2018 Feb 7;13(2):e0191385 [FREE Full text] [CrossRef] [Medline]
  19. Dexheimer JW, Abramo TJ, Arnold DH, Johnson K, Shyr Y, Ye F, et al. Implementation and evaluation of an integrated computerized asthma management system in a pediatric emergency department: a randomized clinical trial. Int J Med Inform 2014 Nov;83(11):805-813 [FREE Full text] [CrossRef] [Medline]
  20. Franke KH, Krumkamp R, Mohammed A, Sarpong N, Owusu-Dabo E, Brinkel J, et al. A mobile phone based tool to identify symptoms of common childhood diseases in Ghana: development and evaluation of the integrated clinical algorithm in a cross-sectional study. BMC Med Inform Decis Mak 2018 Mar 27;18(1):23 [FREE Full text] [CrossRef] [Medline]
  21. Kreps GL, Neuhauser L. New directions in eHealth communication: opportunities and challenges. Patient Educ Couns 2010 Mar;78(3):329-336. [CrossRef] [Medline]
  22. LaMonica HM, Davenport TA, Braunstein K, Ottavio A, Piper S, Martin C, et al. Technology-enabled person-centered mental health services reform: strategy for implementation science. JMIR Ment Health 2019 Sep 19;6(9):e14719 [FREE Full text] [CrossRef] [Medline]
  23. LaMonica HM, Davenport TA, Burns J, Cross S, Hodson S, Veitch J, et al. Technology-enabled mental health service reform for open arms - veterans and families counselling: participatory design study. JMIR Form Res 2019 Sep 19;3(3):e13662 [FREE Full text] [CrossRef] [Medline]
  24. Ospina-Pinillos L, Davenport T, Diaz AM, Navarro-Mancilla A, Scott EM, Hickie IB. Using participatory design methodologies to co-design and culturally adapt the Spanish version of the mental health eClinic: qualitative study. J Med Internet Res 2019 Aug 02;21(8):e14127 [FREE Full text] [CrossRef] [Medline]
  25. Eng TR. eHealth research and evaluation: challenges and opportunities. J Health Commun 2002;7(4):267-272. [CrossRef] [Medline]
  26. National Safety and Quality Digital Mental Health (NSQDMH) standards. Australian Commission on Safety and Quality in Healthcare. 2020.   URL: https:/​/www.​safetyandquality.gov.au/​sites/​default/​files/​2020-04/​UPDATED%20national_safety_and_quality_digital_mental_health_standards_-_consultation_draft_-_feb_2020_0.​pdf [accessed 2021-12-19]
  27. Draft global strategy on digital health 2020–2025. World Health Organization. 2020.   URL: https:/​/www.​who.int/​docs/​default-source/​documents/​gs4dhdaa2a9f352b0445bafbc79ca799dce4d.​pdf?sfvrsn=f112ede5_42 [accessed 2021-12-19]
  28. Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. J Med Internet Res 2005 Feb 24;7(1):e1 [FREE Full text] [CrossRef] [Medline]
  29. Bashshur R, Shannon G, Krupinski E, Grigsby J. The taxonomy of telemedicine. Telemed J E Health 2011;17(6):484-494. [CrossRef] [Medline]
  30. Vegesna A, Tran M, Angelaccio M, Arcona S. Remote patient monitoring via non-invasive digital technologies: a systematic review. Telemed J E Health 2017 Jan;23(1):3-17 [FREE Full text] [CrossRef] [Medline]
  31. Adolescent health and development. World Health Organization. 2019.   URL: https://tinyurl.com/232vmuy4 [accessed 2021-12-19]
  32. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
  33. Types of evaluation. Centers for Disease Control and Prevention. 2019.   URL: https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf [accessed 2021-12-19]
  34. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Australian Government National Health and Medical Research Council. 2008.   URL: https://www.mja.com.au/sites/default/files/NHMRC.levels.of.evidence.2008-09.pdf [accessed 2021-12-19]
  35. Dijkstra L, Florczyk A, Freire S, Kemper T, Melchiorri M, Pesaresi M, et al. Applying the Degree of Urbanisation to the globe: a new harmonised definition reveals a different picture of global urbanisation. J Urban Econ 2021 Sep;125:103312 [FREE Full text] [CrossRef]
  36. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998 Jun;52(6):377-384 [FREE Full text] [CrossRef] [Medline]
  37. Methods for the development of NICE public health guidance. National Institute for Health and Care Excellence.   URL: https:/​/www.​nice.org.uk/​process/​pmg4/​chapter/​appendix-h-quality-appraisal-checklist-qualitative-studies [accessed 2021-12-19]
  38. Wang J, Yao NA, Liu Y, Geng Z, Wang Y, Shen N, et al. Development of a smartphone application to monitor pediatric patient-reported outcomes. Stud Health Technol Inform 2017;245:253-257. [Medline]
  39. Gregory JM, Sukhera J, Taylor-Gates M. Integrating smartphone technology at the time of discharge from a child and adolescent inpatient psychiatry unit. J Can Acad Child Adolesc Psychiatry 2017;26(1):45-50 [FREE Full text] [Medline]
  40. Jeong Y, Chang H, Kim J. Development and feasibility of a safety plan mobile application for adolescent suicide attempt survivors. Comput Inform Nurs 2020 Aug;38(8):382-392. [CrossRef] [Medline]
  41. Hussey D, Flynn KC. The utility and impact of the addiction comprehensive health enhancement support system (ACHESS) on substance abuse treatment adherence among youth in an intensive outpatient program. Psychiatry Res 2019 Nov;281:112580. [CrossRef] [Medline]
  42. Kassam-Adams N, Kohser KL, McLaughlin J, Winston F, Marsac ML. Evaluating the acceptability and validity of assessing pain and posttraumatic stress symptoms in an adaptable eHealth system for school-age children. Clin Pract Pediatr Psychol 2019 Mar;7(1):9-19 [FREE Full text] [CrossRef] [Medline]
  43. Reid SC, Kauer SD, Hearps SJ, Crooke AH, Khor AS, Sanci LA, et al. A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial. BMC Fam Pract 2011 Nov 29;12:131 [FREE Full text] [CrossRef] [Medline]
  44. Thabrew H, D'Silva S, Darragh M, Goldfinch M, Meads J, Goodyear-Smith F. Comparison of YouthCHAT, an electronic composite psychosocial screener, with a clinician interview assessment for young people: randomized controlled trial. J Med Internet Res 2019 Dec 03;21(12):e13911 [FREE Full text] [CrossRef] [Medline]
  45. Rath B, Swenshon S, Haase K, Szeles T, Jung C, Jacobi F, et al. Using a mobile application to detect health needs among children and adolescents who are newly arrived migrants in Europe. J Public Health (Oxf) 2019 Dec 20;41(4):840-849. [CrossRef] [Medline]
  46. Svedberg P, Arvidsson S, Larsson I, Carlsson I, Nygren JM. Barriers and enablers affecting successful implementation of the electronic health service Sisom: multicenter study of child participation in pediatric care. J Med Internet Res 2019 Nov 15;21(11):e14271 [FREE Full text] [CrossRef] [Medline]
  47. Thompson LA, Martinko T, Budd P, Mercado R, Schentrup AM. Meaningful use of a confidential adolescent patient portal. J Adolesc Health 2016 Feb;58(2):134-140. [CrossRef] [Medline]
  48. Boyce S, Nyangara F, Kamunyori J. A mixed-methods quasi-experimental evaluation of a mobile health application and quality of care in the integrated community case management program in Malawi. J Glob Health 2019 Jun;9(1):010811 [FREE Full text] [CrossRef] [Medline]
  49. McCulloh RJ, Fouquet SD, Herigon J, Biondi EA, Kennedy B, Kerns E, et al. Development and implementation of a mobile device-based pediatric electronic decision support tool as part of a national practice standardization project. J Am Med Inform Assoc 2018 Sep 01;25(9):1175-1182 [FREE Full text] [CrossRef] [Medline]
  50. Alawna M, Unver B, Yuksel E. The reliability of a smartphone goniometer application compared with a traditional goniometer for measuring ankle joint range of motion. J Am Podiatr Med Assoc 2019 Jan;109(1):22-29. [CrossRef] [Medline]
  51. Singh H, Mallaiah R, Yadav G, Verma N, Sawhney A, Brahmachari SK. iCHRCloud: web and mobile based child health imprints for smart healthcare. J Med Syst 2017 Nov 29;42(1):14. [CrossRef] [Medline]
  52. Gálvez JA, Lockman JL, Schleelein LE, Simpao AF, Ahumada LM, Wolf BA, et al. Interactive pediatric emergency checklists to the palm of your hand - How the Pedi Crisis App traveled around the world. Paediatr Anaesth 2017 Aug;27(8):835-840. [CrossRef] [Medline]
  53. Han X, Scheetz J, Keel S, Liao C, Liu C, Jiang Y, et al. Development and validation of a smartphone-based visual acuity test (vision at home). Transl Vis Sci Technol 2019 Jul;8(4):27 [FREE Full text] [CrossRef] [Medline]
  54. Thabtah F. An accessible and efficient autism screening method for behavioural data and predictive analyses. Health Informatics J 2019 Dec;25(4):1739-1755 [FREE Full text] [CrossRef] [Medline]
  55. Angues RV, Suits A, Palmer VS, Okot C, Okot RA, Atonywalo C, et al. A real-time medical cartography of epidemic disease (Nodding syndrome) using village-based lay mHealth reporters. PLoS Negl Trop Dis 2018 Jun;12(6):e0006588 [FREE Full text] [CrossRef] [Medline]
  56. Binotti M, Cavallin F, Ingrassia PL, Pejovic NJ, Monzani A, Genoni G, et al. Heart rate assessment using NeoTapAdvancedSupport: a simulation study. Arch Dis Child Fetal Neonatal Ed 2019 Jul;104(4):440-442. [CrossRef] [Medline]
  57. Estai M, Winters J, Kanagasingam Y, Shiikha J, Checker H, Kruger E, et al. Validity and reliability of remote dental screening by different oral health professionals using a store-and-forward telehealth model. Br Dent J 2016 Oct 07;221(7):411-414. [CrossRef] [Medline]
  58. Detsomboonrat P, Pisarnturakit PP. Development and evaluation: the satisfaction of using an oral health survey mobile application. Telemed J E Health 2019 Jan;25(1):55-59. [CrossRef] [Medline]
  59. Matin SB, Wallingford A, Xu S, Ng N, Ho A, Vanosdoll M, et al. Feasibility of a mobile health tool for mothers to identify neonatal illness in rural Uganda: acceptability study. JMIR Mhealth Uhealth 2020 Feb 18;8(2):e16426 [FREE Full text] [CrossRef] [Medline]
  60. Kim M, Yune S, Chang S, Jung Y, Sa SO, Han HW. The fever coach mobile app for participatory influenza surveillance in children: usability study. JMIR Mhealth Uhealth 2019 Oct 17;7(10):e14276 [FREE Full text] [CrossRef] [Medline]
  61. van Karnebeek CD, Houben RF, Lafek M, Giannasi W, Stockler S. The treatable intellectual disability APP www. treatable-id. org: a digital tool to enhance diagnosis and care for rare diseases. Orphanet J Rare Dis 2012 Jul 23;7:47 [FREE Full text] [CrossRef] [Medline]
  62. Heida A, Dijkstra A, Kobold AM, Rossen J, Kindermann A, Kokke F, et al. Efficacy of home telemonitoring versus conventional follow-up: a randomized controlled trial among teenagers with inflammatory bowel disease. J Crohns Colitis 2018 Mar 28;12(4):432-441. [CrossRef] [Medline]
  63. Li C, Huang S, Su X, Zhang T, Jiang K. Monitoring of home recovery using the 317-nursing mobile application following day-case surgery in children: perspectives from both nurses and patients. Medicine (Baltimore) 2019 Aug;98(31):e16639 [FREE Full text] [CrossRef] [Medline]
  64. Padidar P, Shaker M, Amoozgar H, Khorraminejad-Shirazi M, Hemmati F, Najib KS, et al. Detection of neonatal jaundice by using an android OS-based smartphone application. Iran J Pediatr 2019 Feb 02 (forthcoming). [CrossRef]
  65. Rath B, Maltezou HC, Papaevangelou V, Papagrigoriou-Theodoridou MA, Alchikh M, Myles P, PEDSIDEA Network. Partnering for enhanced digital surveillance of influenza-like disease and the effect of antivirals and vaccines (PEDSIDEA). Influenza Respir Viruses 2019 Jul;13(4):309-318 [FREE Full text] [CrossRef] [Medline]
  66. Jiam NT, Hoon AH, Hostetter CF, Khare MM. IIAM (important information about me): a patient portability profile app for adults, children and families with neurodevelopmental disabilities. Disabil Rehabil Assist Technol 2017 Aug;12(6):599-604. [CrossRef] [Medline]
  67. The rural and remote implications of a national e-health strategy. National Health Rural Alliance. 2009.   URL: https://www.ruralhealth.org.au/sites/default/files/position-papers/position-paper-09-07-23.pdf [accessed 2021-12-19]
  68. Curtin J. A digital divide in rural and regional Australia? Current Issues Briefs - Parliament of Australia. 2001.   URL: https:/​/www.​aph.gov.au/​About_Parliament/​Parliamentary_Departments/​Parliamentary_Library/​Publications_Archive/​CIB/​cib0102/​02CIB01#top [accessed 2021-12-19]
  69. Henry L. Bridging the urban-rural digital divide and mobilizing technology for poverty eradication: challenges and gaps. GSM Association.   URL: https:/​/www.​un.org/​development/​desa/​dspd/​wp-content/​uploads/​sites/​22/​2019/​03/​Henry-Bridging-the-Digital-Divide-2019.​pdf [accessed 2021-12-19]
  70. Walker J. Adding to the list of rural inequalities-Digital technology exclusion. Aust J Rural Health 2019 Oct 30;27(5):440-441. [CrossRef] [Medline]
  71. Christensen H, Hickie IB. Using e-health applications to deliver new mental health services. Med J Aust 2010 Jun 07;192(S11):53-56. [CrossRef] [Medline]
  72. Griffiths KM, Farrer L, Christensen H. The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust 2010 Jun 07;192(S11):4-11. [CrossRef] [Medline]
  73. Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
  74. Wu YP, Steele RG, Connelly MA, Palermo TM, Ritterband LM. Commentary: pediatric eHealth interventions: common challenges during development, implementation, and dissemination. J Pediatr Psychol 2014 Jul;39(6):612-623 [FREE Full text] [CrossRef] [Medline]
  75. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan 18;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]


NFP: not-for-profit
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses


Edited by R Kukafka, G Eysenbach; submitted 26.11.20; peer-reviewed by L Ospina-Pinillos, A Čuš; comments to author 14.01.21; revised version received 15.04.21; accepted 14.10.21; published 12.05.22

Copyright

©Elizabeth Stewart, Alyssa Milton, Hannah Frances Yee, Michael Jae Song, Anna Roberts, Tracey Davenport, Ian Hickie. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 12.05.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.