Review
Abstract
Background: eHealth tools that assess and track health outcomes in children or young people are an emerging type of technology that has the potential to reform health service delivery and facilitate integrated, interdisciplinary care.
Objective: The aim of this review is to summarize eHealth tools that have assessed and tracked health in children or young people to provide greater clarity around the populations and settings in which they have been used, characteristics of digital devices (eg, health domains, respondents, presence of tracking, and connection to care), primary outcomes, and risks and challenges of implementation.
Methods: A search was conducted in PsycINFO, PubMed or MEDLINE, and Embase in April 2020. Studies were included if they evaluated a digital device whose primary purpose was to assess and track health, focused on children or young people (birth to the age of 24 years), reported original research, and were published in peer-reviewed journals in English.
Results: A total of 39 papers were included in this review. The sample sizes ranged from 7 to 149,329 participants (median 163, mean 5155). More studies were conducted in urban (18/39, 46%) regions than in rural (3/39, 8%) regions or a combination of urban and rural areas (8/39, 21%). Devices were implemented in three main settings: outpatient health clinics (12/39, 31%), hospitals (14/39, 36%), community outreach (10/39, 26%), or a combination of these settings (3/39, 8%). Mental and general health were the most common health domains assessed, with a single study assessing multiple health domains. Just under half of the devices tracked children’s health over time (16/39, 41%), and two-thirds (25/39, 64%) connected children or young people to clinical care. It was more common for information to be collected from a single informant (ie, the child or young person, trained health worker, clinician, and parent or caregiver) than from multiple informants. The health of children or young people was assessed as a primary or secondary outcome in 36% (14/39) of studies; however, only 3% (1/39) of studies assessed whether using the digital tool improved the health of users. Most papers reported early phase research (formative or process evaluations), with fewer outcome evaluations and only 3 randomized controlled trials. Identified challenges or risks were related to accessibility, clinical utility and safety, uptake, data quality, user interface or design aspects of the device, language proficiency or literacy, sociocultural barriers, and privacy or confidentiality concerns; ways to address these barriers were not thoroughly explored.
Conclusions: eHealth tools that assess and track health in children or young people have the potential to enhance health service delivery; however, a strong evidence base validating the clinical utility, efficacy, and safety of tools is lacking, and more thorough investigation is needed to address the risks and challenges of using these emerging technologies in clinical care. At present, there is greater potential for the tools to facilitate multi-informant, multidomain assessments and longitudinally track health over time and room for further implementation in rural or remote regions and community settings around the world.
doi:10.2196/26015
Keywords
Introduction
In 2018, the United Nations Children’s Fund released a report on digital technologies in health [
] alongside its Strategic Plan, 2018-2021, which emphasized the importance of using digital (internet- and mobile-based) technology to facilitate health care for children and young people. At the same time, the Early Childhood Developmental Interventions Review Group for the Lancet issued a report making several recommendations for the improvement of assessments and interventions for children or young people [ ]. These recommendations included improving the capacity for services to conduct multidomain and multi-informant assessments, connecting children or young people and families with personalized care options, and using digital solutions within health services to allow for broader-scale change [ ]. Together, these reports highlight the emerging need to use digital technologies to enhance the delivery of health care for children or young people and their families.Over the past decade, there has been a rapid growth in the development of digital tools in the health and well-being space [
]. These tools have served various purposes in health care, with the most common uses among children or young people being to deliver interventions (eg, clinician-assisted evidence-based treatments and self-monitoring and self-care), provide education, and facilitate communication for both consumers and clinicians (eg, telehealth or teleconferencing and online peer support groups) [ , ]. Another more recent use of eHealth has been to facilitate the assessment and triage of children or young people through health services [ - ]. These emerging technologies provide users (ie, clinicians and consumers) with secure, web-based platforms for submitting health data (sometimes automatically via biosensors or wearables) without having to be physically present in a hospital or health clinic. The information can be securely shared with health professionals with expertise in children or young people’s areas of need, allowing them to be triaged to appropriate services and connected with ongoing care [ , - , , ]. Thus, these eHealth solutions differ from existing technologies in that their goal is not to deliver interventions or ongoing treatment but rather to facilitate a connection between consumers and pre-existing health services, allow for routine outcome monitoring, and place the person (or family) at the center of care. Furthermore, although some of these tools provide education resources (eg, fact sheets) or communication pathways (eg, web-based chats), they do this with the goal of triaging children or young people to appropriate care.The literature on eHealth tools that assess and track health outcomes in children or young people is still in its infancy; however, a growing number of studies have reported on such devices over the past decade [
, ]. These tools differ in their health focus (eg, infectious diseases and mental health) [ , ] and locations in which they have been used (ie, rural or urban areas, high- or low-income countries, and specific health settings) [ , - ]. There has also been variability in terms of the respondent who enters data into the tool (ie, clinician and consumer), the type of data (ie, questionnaires and physiological data), whether the tools have facilitated only assessment or assessment and tracking, and whether they have connected children or young people to clinical care. Given this variability, the specific features of eHealth tools, as well as their efficacy for improving health outcomes and clinical care delivery for children or young people, remain unclear.Despite the potential benefits of eHealth solutions for children or young people, numerous challenges have been documented in their development, implementation, and uptake among other groups [
, , - ]. To be successful, the technologies must be user-friendly, engaging, and accessible to diverse populations. Issues of language, literacy, and culture have all been found to affect accessibility, uptake, and the quality of data [ , , , ]. The validity and integrity of data also depend on the availability of appropriately trained health care workers to enter or interpret information, emphasizing the importance of developing and evaluating these tools within the contexts in which they will be used. Finally, issues of privacy, confidentiality, and security are paramount to ensuring that the tools respect the rights of users and are likely to affect the uptake of these technologies [ , , ].To our knowledge, no comprehensive reviews have been conducted to examine the efficacy of eHealth tools that assess and track health outcomes in children or young people. As such, it is unclear in which health domains and settings these tools may have the potential to shape clinical care and, importantly, whether their use has been associated with improved health outcomes for children or young people. There is also a need to identify potential challenges and risks of using eHealth tools to ensure that best practice methods are established and consistently used [
, ]. Understanding the available eHealth solutions and their efficacies is critical for shaping future research and development efforts and ensuring efficient expansion of knowledge in this field.The aim of this systematic review is to summarize eHealth tools that have been developed to assess and track health in children or young people to provide greater clarity about (1) the populations and settings in which these tools have been used (ie, locations, languages, and age groups); (2) characteristics of the tools (ie, health domains assessed, respondent, type of data, presence of tracking, and connection to care); (3) primary outcomes of the study, including whether the use of the tool has been associated with improved health outcomes; and (4) risks and challenges identified during implementation and evaluation.
Methods
Overview
The term eHealth has been variously defined in the literature. According to a systematic review, 51 unique definitions have been used for the term, without a clear consensus on a single definition, and the definitions differ in how inclusively they are conceptualized [
, ]. We have chosen to use a definition based on the conceptualization of eHealth offered by Vegesna et al [ ] because of its relevance and consistency with the overarching aims of this review; digital technologies are thus defined as noninvasive digital devices that have been used to assess and track the health of a patient or consumer. We used the World Health Organization’s definition of childhood and youth as the period spanning birth to 24 years, whereby children are aged 0 to 9 years, and young people are aged 10 to 24 years [ ].Search Strategy
The search was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [
]. PsycINFO, PubMed or MEDLINE, and Embase were searched via OVID by 3 members of the research team (ES, HY, and AR) on April 27, 2020. The following terms were used ((child*) OR (adolescen*) OR (young person) OR (infan*)) AND ((wellbeing) OR (health)) AND (((digital tool) OR (digital AND tool)) OR (eHealth) OR ((mobile application) OR (mobile AND application))). A wildcard (*) was placed at the end of each applicable search term to ensure that all relevant terms were captured. All Medical Subject Heading terms were explored to broaden the search for relevant studies. Date limits were not set on any of the database searches. The reference lists of relevant reviews and identified empirical studies were searched to identify further studies, as per the ancestry method.Study Selection Criteria
Studies were included if they met the following criteria:
- Included children and young people (birth to the age of 24 years) or their parents or carers, as per the World Health Organization’s definition [ ]
- Evaluated a digital device, including internet- or mobile-based technology (ie, noninvasive digital devices, including internet- or mobile-based e-tools and wearable devices), the primary purpose of which was to assess or track the health of the child or young person
- Focused on a domain of health
- Evaluation studies, meaning the authors evaluated some aspects of the digital device, including effectiveness, validity, or feasibility; we included all or any type of evaluation studies, which were categorized according to the Center for Disease Control definition (ie, formative, process, and outcome) [ ] and National Health and Medical Research Council criteria for study design [ ]
- Reported original research
- Published in English in a peer-reviewed journal and included human participants
Studies were excluded if they had the following characteristics:
- Included adults only with no child or young person focus or if >25% of participants were outside our age criteria (birth to the age of 24 years)
- Evaluated a digital device that was primarily an interventional tool (ie, clinician-led and self-management tools), an educational device (eg, an e-course), a communication device (ie, assistive communication with images or written or spoken language; and teleconferencing only without additional assessment or tracking functionality), or digital technology that did not use internet or mobile technology (eg, electronic medical record systems)
- Reported results from development or description of the tool that had not yet been evaluated (eg, protocol papers)
Procedure
displays the process of study selection. The search retrieved 3688 papers, and an additional 16 papers were identified by searching the reference lists of relevant papers and reviews. Of these papers, 95.59% (3541/3704) remained after duplicates were removed. The titles and abstracts of all papers were screened by 2 reviewers (ES and HY). Of the 3541 papers, 84 (2.37%) full-text papers met the inclusion criteria and were obtained. A conservative approach was taken to ensure that relevant papers were not missed, and full-text papers were reviewed if the reviewers could not determine with certainty whether the inclusion criteria were met. The manuscripts of these 84 papers were reviewed by 2 independent raters (ES and HY), and discrepancies were resolved via discussion. Of the 84 papers, 45 (54%) papers were excluded, leaving 39 (46%) papers that were included in the review.
Data Extraction
The following information was extracted from each paper:
- Name of the first author, year of publication, and age range of children or young people
- Language or languages used in the digital application
- Location where the study was conducted: country, locality (urban or rural), and setting; locality was defined according to the Organization for Economic Co-operation and Development harmonized definition of global urbanization, which uses the population density of the area, that is, rural (<5000 inhabitants) or urban (≥5000 inhabitants) [ ]; some studies were conducted in multiple locations, which was considered in categorizing study locality as urban, rural, or a mixture of urban and rural settings
- Characteristics of the digital tool: health domain assessed, respondent (parent or caregiver, child or young person, clinician, trained health worker, and other), device type (mobile, desktop, and tablet), type of data (questionnaire or survey, images, and physiological), whether the tool allowed for tracking over time (ie, data collected at multiple time points), and whether the device facilitated connection to care (ie, linking patients to health care providers or services)
- Study characteristics: type of evaluation study, defined according to the Center for Disease Control definition of study evaluation types, that is, formative, process, or evaluation [ ]; study type: qualitative, quantitative, or mixed methods; and (3) study design, based on the National Health and Medical Research Council guidelines [ ]
- The primary outcome and main findings from the study, including whether the health of the child or young person was measured as an outcome in the study
- Funding source, categorized as public sector (ie, government, universities, research institutes, and professional associations), commercial or not-for-profit (NFP) organizations; these categories were guided by an Australian Government resource on university research funding (REF)
- Any documented risks or challenges associated with the use of the eHealth tool
Data Analysis
Descriptive analyses were used to summarize variables of interest, including health domain, location, language, type of data, intended user, presence of certain features (ie, tracking over time and connection to care), and type of evaluation. Frequency data and percentages were used to examine and compare studies on key outcome measures. This approach to analysis was taken because of considerable variability in study objectives and designs and as most studies reported simple quantitative, descriptive statistics or qualitative findings.
Quality Appraisal of Studies
To evaluate the methodological quality of the studies, 2 checklists were used. The Downs and Black checklist [
] was completed for quantitative studies, which measures the quality of both randomized and nonrandomized studies evaluating novel health interventions. The National Institute for Health and Care Excellence Quality Appraisal Checklist was completed for studies reporting qualitative findings [ ]. Studies reporting both qualitative and quantitative data were appraised using both checklists. A full description of the checklists and scoring criteria is included in [ - , - , - ].Results
Demographics of Studies
summarizes the characteristics of the 39 studies included in this review. All (38/39, 97%) but a single (1/39, 3%) study was published in the past decade (2010-2020), and over one-third of the studies (15/39, 39%) were published in the past year (2019-2020; ). Most studies were conducted in a single country (35/39, 90%), most commonly America (7/39, 18%) or Australia (6/39, 15%). English was the sole language of communication in 49% (19/39) of studies; 13% (5/39) of studies evaluated tools that used English and at least one other language, and 21% (8/39) used languages other than English; the remaining 18% (7/39) of studies did not report enough information to determine which language was used in the tool. Regarding locality, studies were conducted in urban (18/39, 46%), rural (3/39, 8%), or a mixture of urban and rural settings (8/39, 21%); 26% (10/39) studies did not report enough information to determine locality. Digital devices were implemented across 3 main settings: outpatient health clinics (12/39, 31%), hospitals (ie, inpatient units and emergency departments; 14/39, 36%), and community outreach (ie, community spaces that were not formal health clinics; 10/39, 26%) or a combination of these settings (3/39, 8%).
Study | Age range of children | Country | Localitya | Language used in the device | Setting |
Alawna et al, 2019 [ | ]19-27 years (mean 22.0) | Turkey | NRb | NR | Outpatient health clinic |
Binotti et al, 2019 [ | ]Infants (age range NR) | Italy | Urban | NR | Hospital |
Boyce et al, 2019 [ | ]2-59 months | Malawi | NR | English | Outpatient health clinic |
Den Boer et al, 2018 [ | ]3-17 years | Netherlands | Urban (81%) and rural (19%) | English, German, Spanish, and Dutch | Outpatient health clinic |
Detsomboonrat and Pisarnturakit, 2019 [ | ]Children in primary school (age range NR) | Thailand | NR | Thai | Community outreach |
Dexheimer et al, 2014 [ | ]2-18 years | United States | Urban | English | Hospital |
Eikelboom et al, 2005 [ | ]9 months-16 years | Australia | Rural | English | Outpatient health clinic |
Estai et al, 2016 [ | ]2-18 years | Australia | Urban | English | Outpatient health clinic |
Finocchario-Kessler et al, 2015 [ | ]Children (age range NR) | Kenya | Urban (50%) and rural (50%) | English | Hospital |
Franke et al, 2018 [ | ]18 months-14 years | Ghana | Urban | Twi | Hospital |
Galvez et al 2017 [ | ]Children (age range NR) | 55 countries (worldwide) | NR | English | Hospital |
Ginsburg et al, 2015 [ | ]Children (age range NR) | Ghana | Urban | English | Outpatient health clinic |
Gregory et al, 2017 [ | ]<18 years | United Kingdom | Urban | English | Hospital |
Han et al, 2019 [ | ]13-26 years | China and Australia | Urban | NR | Hospital |
Hashemi et al, 2017 [ | ]6-18 years | Gaza | Urban | English and Arabic | Community outreach |
Heida et al, 2018 [ | ]10-19 years | Netherlands | Urban (55%) and rural (45%) | Dutch | Outpatient health clinic |
Hussey and Flynn, 2019 [ | ]0-21 years | United States | Urban | English | Outpatient health clinic |
Iorfino et al, 2017 [ | ]16-24 years | Australia | Urban (85%) and rural (15%) | English | Outpatient health clinic |
Jeong et al, 2020 [ | ]15-19 years | South Korea | Urban | Korean | Hospital, outpatient health clinic, and community outreach |
Jiam et al, 2017 [ | ]3-22 years | United States | NR | English | Community outreach |
Kassam-Adams et al, 2019 [ | ]6-14 years | United States | Urban (50%) and rural (50%) | English | Hospital |
Kim et al, 2019 [ | ]0-5 years | South Korea | NR | NR | Community outreach |
Li et al, 2019 [ | ]1-18 years | China | Urban | Mandarin | Hospital |
March et al, 2018 [ | ]5-12 years | Australia | Urban | English | Outpatient health clinic and community outreach |
Matin et al, 2020 [ | ]0-7 days | Uganda | Rural | Lusoga and English | Community outreach |
McCulloh et al, 2018 [ | ]0-2 months | United States | NR | English | Hospital |
Mohammed et al, 2018 [ | ]0-5 years | Ghana | Urban | Twi | Community outreach |
Padidar et al, 2019 [ | ]0-9 days | Iran | Urban | NR | Hospital |
Rath et al, 2018 [ | ]0-24 years | Germany | NR | Arab, Farsi, and Russian | Community outreach |
Rath et al, 2019 [ | ]0-5 years | Germany and Greece | Urban | NR | Hospital |
Reid et al, 2011 [ | ]14-24 years | Australia | Urban (50%) and rural (50%) | English | Outpatient health clinic |
Singh et al, 2017 [ | ]0-2 years | India | Urban (85%) and rural (15%) | Hindi, Gujarati, and English | Hospital and outpatient health clinic |
Svedberg et al, 2019 [ | ]6-13 years | Sweden | Urban (50%) and rural (50%) | Swedish | Hospital |
Thabrew et al, 2019 [ | ]13-14 years | New Zealand | NR | English | Outpatient health clinic |
Thabtah, 2018 [ | ]0-17 years | 10 countries | NR | 11 languages | Community outreach |
Thompson et al, 2016 [ | ]12-18 years | United States | Urban | English | Outpatient health clinic |
Valdes-Angues et al, 2018 [ | ]3-18 years | Uganda and United States | Rural | English | Community outreach |
van Karnebeek et al, 2012 [ | ]0-18 years | Canada | Urban | English | Hospital |
Wang et al, 2017 [ | ]5-17 years | China | Urban | English and Chinese | Outpatient health clinic |
aLocality: region in which the eHealth tool was implemented, defined as rural (<5000 inhabitants) or urban (≥5000 inhabitants), according to the Organization for Economic Co-operation and Development’s harmonized definition of global urbanization [
].bNR: not reported.
Characteristics of eHealth Tools
summarizes the characteristics of the studied digital devices.
Study | Health domain | Device | Type of data | Respondent | Tracking over time | Connection to care |
Alawna et al, 2019 [ | ]General health | Mobile | Physiological | Trained health workera | Unclear | Yes |
Binotti et al, 2019 [ | ]Developmental | Mobile | Physiological | Trained health worker | No | No |
Boyce et al, 2019 [ | ]General health | Mobile | Questionnaire or survey | Trained health worker | No | Yes |
Den Boer et al, 2018 [ | ]Oral health | Mobile | Questionnaire or survey | Clinicianb and child or young personc | No | Yes |
Detsomboonrat and Pisarnturakit 2019 [ | ]Oral | Mobile and desktop | Questionnaire or survey | Clinician | No | Yes |
Dexheimer et al, 2014 [ | ]General health | Desktop | Questionnaire or survey | Clinician | Yes | Yes |
Eikelboom et al, 2005 [ | ]Ear, nose, and throat | Desktop | Images | Clinician | No | Yes |
Estai et al, 2016 [ | ]Oral health | Desktop | Images | Clinician and trained health worker | No | No |
Finocchario-Kessler et al, 2015 [ | ]Infectious | Desktop | Physiological | Parent or caregiverd and trained health worker | Yes | Yes |
Franke et al, 2018 [ | ]Infectious | Mobile | Questionnaire or survey | Parent or caregiver | No | Yes |
Galvez et al, 2017 [ | ]Emergency | Mobile | Questionnaire or survey | Clinician | Yes | No |
Ginsburg et al, 2015 [ | ]Infectious | Mobile | Physiological | Trained health worker | No | Yes |
Gregory et al, 2017 [ | ]Mental health | Mobile | Questionnaire or survey | Clinician and child or young person | No | Yes |
Han et al, 2019 [ | ]Vision | Mobile | Physiological | Child or young person | No | No |
Hashemi et al, 2017 [ | ]Mental health | Desktop and mobile | Questionnaire or survey | Trained health worker | No | No |
Heida et al, 2018 [ | ]Physical health | Desktop | Questionnaire or survey and physiological | Child or young person and parent or caregiver | Yes | Yes |
Hussey and Flynn, 2019 [ | ]Mental health | Mobile | Questionnaire or survey | Clinician and child or young person | Yes | Yes |
Iorfino et al, 2017 [ | ]Mental health | Desktop | Questionnaire or survey | Child or young person | Yes | Yes |
Jeong et al, 2020 [ | ]Mental health | Mobile | Questionnaire or survey | Clinician and child or young person | No | Yes |
Jiam et al, 2017 [ | ]Neurological | Desktop | Questionnaire or survey | Parent or caregiver and child or young person | Yes | No |
Kassam-Adams et al, 2019 [ | ]Mental health | Mobile | Questionnaire or survey | Child or young person | Yes | No |
Kim et al, 2019 [ | ]Infectious | Mobile | Questionnaire or survey and physiological | Parent or caregiver | Yes | No |
Li et al, 2019 [ | ]Surgery | Mobile | Questionnaire or survey | Child or young person | No | Yes |
March et al, 2018 [ | ]Mental health | Desktop, mobile, and tablet | Questionnaire or survey | Clinician, parent or caregiver, education provider, and child or young person | No | No |
Matin et al, 2020 [ | ]Developmental | Mobile | Questionnaire or survey and physiological | Parent or caregiver | Yes | Yes |
McCulloh et al, 2018 [ | ]General health | Mobile | Questionnaire or survey | Clinician | No | Yes |
Mohammed et al, 2018 [ | ]General health | Mobile | Questionnaire or survey | Parent or caregiver | No | Yes |
Padidar et al, 2019 [ | ]Developmental | Mobile | Physiological images | Clinician and parent or caregiver | No | No |
Rath et al, 2018 [ | ]General health | Mobile tablet | Questionnaire or survey | Child or young person and parent or caregiver | No | No |
Rath et al, 2019 [ | ]Infectious | Mobile | Questionnaire or survey | Child or young person | No | No |
Reid et al, 2011 [ | ]Mental health | Desktop and mobile | Questionnaire or survey | Child or young person | Yes | Yes |
Singh et al, 2017 [ | ]General health | Desktop and mobile | Questionnaire or survey and physiological | Clinician and parent or caregiver | Yes | Yes |
Svedberg et al, 2019 [ | ]General health | Mobile | Questionnaire or survey | Child or young person | Yes | Yes |
Thabrew et al, 2019 [ | ]Mental health | Mobile and tablet | Questionnaire or survey | Child or young person | No | Yes |
Thabtah, 2018 [ | ]Developmental | Mobile | Questionnaire or survey | Clinician and parent or caregiver | No | No |
Thompson et al, 2016 [ | ]General health | Desktop | Questionnaire or survey and physiological | Parent or caregiver and young person | Yes | Yes |
Valdes-Angues et al, 2018 [ | ]Neurological | Desktop and mobile | Questionnaire or survey | Trained health worker | Yes | Yes |
van Karnebeek et al, 2012 [ | ]Developmental | Desktop, mobile, and tablet | Questionnaire or survey | Clinician | No | No |
Wang et al, 2017 [ | ]General health and mental health | Mobile | Questionnaire or survey | Clinician, parent or caregiver, and child or young person | Yes | Yes |
aTrained health workers are staff without professional training who received specific training in the use of the digital tool and associated health domain.
bClinician is defined as a health professional with qualifications in a particular field of practice (including medical doctors and allied health workers).
cChild or young person is the individual for whom the eHealth tool was developed.
dParent or caregiver is the primary carer of the child or young person.
Health Domains
Mental and general health were the most common eHealth domains assessed, with each evaluated in 26% (10/39) of studies. Other health domains assessed included child development (5/39, 13%), infectious diseases (5/39, 13%), oral health (3/39, 8%), neurological illnesses (2/39, 5%), ear nose and throat (1/39, 3%), emergency medicine (1/39, 3%), physical health (1/39, 3%), vision (1/39, 3%), and pediatric surgery (1/39, 3%). A single study assessed multiple health domains (mental and general health) [
]. Given that mental health was more commonly assessed than other health domains, we examined these studies further to determine their aim or purpose and the type of information collected. Of the 23% (9/39) of studies that solely assessed mental health, 33% (3/9) focused on suicide prevention [ , , ], 22% (2/9) focused on early intervention and prevention of mental illness [ , ], and 44% (4/9) focused on multidimensional assessment or management of mental health symptoms [ - ]. Of the 23% (9/39) of studies that solely assessed general health, 22% (2/9) focused on symptom detection and monitoring [ , ], 22% (2/9) provided a platform for patients to view and monitor their health information [ , ], 33% (3/9) focused on digitalized tracking of clinical decision-making [ , , ], and 22% (2/9) were primarily for assessment [ , ].Data Collection: Respondent, Type of Data, and Device
All devices measured the health of a child or young person; however, devices differed in the person who entered the health information (ie, the respondent: child or young person, parent or caregiver, clinician, and trained health worker). Under half of the devices collected information from multiple respondents (16/39, 41%); other tools collected information solely from a child or young person (8/39, 21%), clinician (6/39, 15%), trained health worker (6/39, 15%), or parent or caregiver (3/39, 8%). Approximately 15% (6/39) of studies collected data in multiple forms (ie, questionnaire or survey, physiological data, or images); otherwise, data were collected solely in the form of questionnaires or surveys (26/39, 67%), physiological data (5/39, 13%), or images (2/39, 5%). Most eHealth tools (31/39, 80%) were configured to collect data on a mobile phone, of which some (9/39, 23%) were also configured to collect data on another device (ie, desktop or tablet).
Device Features: Health Tracking and Connection to Care
Just under half of the devices tracked children’s health over time (16/39, 41%), and two-thirds (25/39, 64%) connected children or young people to clinical care, whereas the remainder did not.
Outcome Evaluation: Primary Outcome Measures and Findings
summarizes the sample size, type of evaluation, study type and design, and primary outcomes, and a more detailed description of the main findings for each study is presented in (see Table S1). The sample sizes ranged from 7 to 149,329 participants (median 163, mean 5155). Most studies were formative (20/39, 51%) or process (11/39, 28%) evaluations, with fewer outcome evaluation studies (8/39, 21%). Just over one-third of the studies (14/39, 36%) assessed the health of children or young people as either a primary or secondary outcome; however, only a single (1/14, 7%) study assessed whether using the digital tool improved the health of children or young people [ ]. This study examined whether the use of Mobiletype, an eHealth tool that allowed general practitioners and young people to monitor symptoms of mood, stress, and daily activities in general practice, was associated with improved mental health outcomes compared with treatment as usual. The authors found that use of the device was associated with a significant improvement in emotional self-awareness but found no changes in symptoms of depression, anxiety, or stress; post hoc analyses showed enhanced mental health care at the initial assessment among general practitioners using the tool compared with those who did not.
Study | Sample (N) | Type of evaluationa | Study type | Study designb | Health as outcomec | Primary outcome | Challenges or risks of using the tools |
Alawna et al, 2019 [ | ]58 | Formative | Quantitative | Descriptive study | No | Reliability (intra- and interrater reliability) |
|
Binotti et al, 2019 [ | ]40 | Formative | Quantitative | Descriptive study | No | Concordance ratingd |
|
Boyce et al, 2019 [ | ]799 | Process | Mixed methods | Quasi-experimental | No | Efficacy |
|
Den Boer et al, 2018 [ | ]653 | Formative | Mixed methods | Descriptive study | Yes | Usability and efficacy |
|
Detsomboonrat and Pisarnturakit 2019 [ | ]441 | Formative | Quantitative | Descriptive study | No | Acceptability and efficacy |
|
Dexheimer et al, 2014 [ | ]13,896 | Outcome | Quantitative | RCTf | No | Efficacy (time from triage to clinical decision) |
|
Eikelboom et al, 2005 [ | ]66 | Formative | Quantitative | Descriptive study | No | Concordance rating |
|
Estai et al, 2016 [ | ]126 | Formative | Quantitative | Descriptive study | No | Concordance rating |
|
Finocchario-Kessler et al, 2015 [ | ]NRg | Outcome | Mixed methods | Cross-sectional study | No | Feasibility and efficacy |
|
Franke et al, 2018 [ | ]237 | Process | Quantitative | Cross-sectional study | No | Concordance rating |
|
Galvez et al, 2017 [ | ]1252 | Process | Quantitative | Descriptive study | No | Use and uptake |
|
Ginsburg et al, 2015 [ | ]7 | Formative | Mixed methods | Descriptive study | No | Usability and acceptability |
|
Gregory et al, 2017 [ | ]76 | Formative | Quantitative | Descriptive study | No | Feasibility of uptake |
|
Han et al, 2019 [ | ]150 | Outcome | Quantitative | Cohort study | Yes | Validity and reliability |
|
Hashemi et al, 2017 [ | ]986 | Outcome | Quantitative | Descriptive study | Yes | Feasibility |
|
Heida et al, 2018 [ | ]170 | Outcome | Mixed methods | RCT | Yes | Efficacy |
|
Hussey and Flynn, 2019 [ | ]56 | Formative | Mixed methods | Comparative study with historical control group | No | Use and efficacy |
|
Iorfino et al, 2017 [ | ]232 | Process | Quantitative | Nonrandomized experimental trial | Yes | Efficacy |
|
Jeong et al, 2020 [ | ]13 | Formative | Mixed methods | Descriptive study | No | Feasibility, acceptability, and usability |
|
Jiam et al, 2017 [ | ]7 | Process | Qualitative | Descriptive study | No | Usability |
|
Kassam-Adams et al, 2019 [ | ]167 | Process | Quantitative | Descriptive study | No | Acceptability and efficacy |
|
Kim et al, 2019 [ | ]149,329 | Process | Mixed methods | Descriptive study | No | Uptake, usability, and efficacy |
|
Li et al, 2019 [ | ]137 | Outcome | Quantitative | Pseudo-RCT | Yes | Utility and efficacy |
|
March et al, 2018 [ | ]18 | Formative | Mixed methods | Descriptive study | No | Feasibility and acceptability |
|
Matin et al, 2020 [ | ]18 | Formative | Quantitative | Pretest–posttest case series | No | Feasibility and acceptability |
|
McCulloh et al, 2018 [ | ]3805 | Formative | Mixed method | Descriptive study | No | Use or uptake and usability |
|
Mohammed et al, 2018 [ | ]1446 | Formative | Quantitative | Descriptive study | Yes | Feasibility and concordance ratings |
|
Padidar et al, 2019 [ | ]113 | Formative | Quantitative | Descriptive study | Yes | Efficacy (concordance rating) |
|
Rath et al, 2018 [ | ]405 | Formative | Mixed methods | Descriptive study | Yes | Usability and efficacy |
|
Rath et al, 2019 [ | ]1615 | Formative | Quantitative | Cohort study | Yes | Efficacy |
|
Reid et al, 2011 [ | ]163 | Outcome | Quantitative | RCT | Yes | Change in mental health status |
|
Singh et al, 2017 [ | ]16,490 | Process | Quantitative | Descriptive study | Yes | Feasibility |
|
Svedberg et al, 2019 [ | ]46 | Process | Qualitative | Descriptive study | No | Feasibility and acceptability |
|
Thabrew et al, 2019 [ | ]129 | Formative | Mixed methods | Pseudo-RCT | No | Efficacy and acceptability |
|
Thabtah, 2018 [ | ]1452 | Outcome | Quantitative | Descriptive study | Yes | Feasibility and efficacy |
|
Thompson et al, 2016 [ | ]937 | Process | Quantitative | Descriptive study | No | Use and uptake |
|
Valdes-Angues et al, 2018 [ | ]326 | Process | Mixed methods | Descriptive study | No | Feasibility |
|
van Karnebeek et al, 2012 [ | ]15 | Formative | Qualitative | Descriptive study | Yes | Feasibility and acceptability |
|
Wang et al, 2017 [ | ]31 | Formative | Qualitative | Descriptive study | No | Usability |
|
aType of evaluation defined as follows: (1) formative evaluation: assessed feasibility, appropriateness, or acceptability of the digital device before full implementation; (2) process evaluation: assessed whether the digital device had been implemented as intended; (3) outcome evaluation: measured the effectiveness of the digital device by assessing progress in primary outcomes [
].bStudy design based on the National Health and Medical Research Council guidelines: randomized controlled trials (RCTs); pseudo-RCTs; comparative studies with concurrent controls, including nonrandomized experimental trials, cohort studies, case–control studies, or interrupted time series with a control group; comparative studies without a control group, including historical control studies, ≥2 single-arm studies or interrupted time series without a parallel control; case series with either posttest or pre- and posttest outcomes; descriptive studies; or other [
].cWhether an aspect of the child’s or young person’s health was measured as a primary or secondary outcome of the study.
dThe amount of agreement between the digital tool and clinician ratings.
eUX: user experience (user interface or design aspects of the device).
fRCT: randomized controlled trial.
gNR: not reported.
Challenges and Risks Identified in Studies
summarizes the challenges and risks of using the tools identified in each study. Of the studies examined, most (30/39, 77%) identified at least one challenge or risk, which was related to accessibility (11/39, 28%), clinical utility (9/39, 23%) or clinical safety (5/39, 13%) of the tool, uptake by users (6/39, 15%), data quality (6/39, 15%), user interface or design aspects of the device (user experience; 5/39, 13%); language proficiency or literacy barriers (4/39, 10%), sociocultural barriers (1/39, 3%), and privacy concerns (1/39, 3%). More specifically, accessibility problems were related to poor internet connection, inability to recharge devices because of power cuts, slow or inefficient upload of information, lack of access to a device, and low technological literacy of end users. Clinical utility and clinical safety concerns were related to the validity of data among people with different health conditions, lack of appropriate training of staff, input from a health care professional rather than entirely self-report data to ensure safe and accurate interpretation of results, whether the tool added value over and above standard clinical care, and the safety of tools that had not yet been validated to detect clinical symptoms. Uptake of tools was a frequently cited barrier; however, there was often no further investigation or explanation as to why uptake was lower than expected. Data quality concerns were centered on inaccurate or incomplete data entry (because of human or computer error) and poor-quality images. User experience or design barriers referred to the eHealth tool lacking the necessary features to make it functional and usable for end users. Language proficiency or literacy barriers were centered on users lacking the comprehension and literacy levels to understand and take action from the presented information; this was a concern reported when end users were children, non–English speaking, or from lower socioeconomic backgrounds. Sociocultural barriers were related to the appropriateness of questions and the risk of causing offense or harm.
Research Funding
Studies were financially supported by the public sector (ie, government, universities, research institutes, and professional associations) and commercial or NFP organizations (28/39, 72%). Receiving funding from ≥1 sector was the most common (16/39, 41%), followed by funding solely from public sources (6/39, 15%) and NFPs (5/39, 13%). No study was funded solely by the commercial sector; however, commercial funding contributed to nearly one-third of studies with combined funding sources (5/16, 31%). The remainder of the studies did not receive external financial support or did not report it in the paper (11/39, 28%).
Quality Ratings of Selected Papers
The methodological quality of the Downs and Black checklist was rated for 95% (35/39) of studies that included quantitative data: 64% (25/39) of studies had a low chance of bias, 36% (14/39) of studies had a moderate chance of bias, and no studies had a high chance of bias (see Table S2,
). The National Institute for Health and Care Excellence Quality Appraisal Checklist was completed for 44% (17/39) of studies that included qualitative data: 59% (10/17) of studies received a maximum score of 2 for quality, and 41% (7/17) of studies received a partial score of 1; no studies received a score of 0 (see Table S3, ).Discussion
Principal Findings
The aim of this systematic review was to summarize eHealth tools designed to assess and track health outcomes in children and young people to clarify the current scope, nature, and efficacy of this emerging type of technology in health care. Our findings revealed exponential growth in the development and evaluation of these tools over the past 10 years; however, the results showed that the research is still in its infancy, with most studies assessing feasibility, acceptability, usability, or uptake of a device rather than the efficacy of tools in relation to health outcomes. Overall, the current tools showed potential to enhance the assessment and tracking of children or young people in health services around the world. Further research is needed to evaluate the efficacy of tools for improving health outcomes and clinical care delivery, as well as to identify and address the risks and challenges of implementing these tools as part of standard clinical care.
There are numerous potential advantages of using eHealth solutions for children and young people, including the ability to conduct multidomain and multi-informant assessments, undertake continuous monitoring, and assist with timely connection to personalized clinical care [
, , ]. Encouragingly, over half of the tools facilitated a connection between the child or young person and a health care provider; however, less than half tracked children’s health data over time or collected information from multiple informants (ie, child or young person, parent or caregiver, and health care professional). These findings demonstrate an untapped potential of eHealth solutions in facilitating multi-informant assessments and longitudinally tracking health over time among children or young people, which is key to achieving comprehensive, multidisciplinary care [ ]. In addition, data were most commonly collected in the form of surveys or questionnaires, illustrating a lag in uptake and integration of newer technologies (eg, biosensors or wearables to collect physiological data). Such technology has the potential to enhance symptom detection and clinical decision-making [ , , , , ] and may be an important area for future research to explore.Another potential advantage of eHealth is its ability to overcome geographical, financial, and social barriers that hinder the provision of health services in specific populations and locations [
, ]. A small number of studies evaluated devices that were implemented across multiple countries [ , - ], highlighting the ability of digital technologies to provide health care with greater reach. However, fewer tools were implemented in rural areas compared with urban areas, and there was less implementation in community outreach settings compared with hospitals and health clinics. Although using eHealth tools in health clinics and hospitals is a step forward from traditional paper-based methods in terms of data management and integrated care, there is greater potential for the tools to engage hard-to-reach populations in regional and community settings [ , ]. The higher percentage of devices used in health clinics and hospitals may be as community settings do not always incorporate systematic health tracking into their procedures or reflect a lack of availability of skilled health professionals to collect and enter health information in community settings. A number of studies overcame this issue by using trained health workers (ie, staff who received specific training in the health condition and digital device but were not specialists in the field); these trained health workers were able to collect information in the community, with studies finding that this did not compromise the validity or reliability of data or clinical care [ , , , , , ]. Another solution was to collect information solely from the consumer (ie, child or young person or their parent or guardian), which is particularly common in studies examining mental health [ , , ]. Together, these results demonstrate room for broader implementation in rural or remote regions and community settings around the world. Nevertheless, an important issue to recognize is that rural and vulnerable populations are currently experiencing the largest digital divide [ - ]. Inequalities in access exist because of variations in location, age, education, and income level. For instance, the cost of internet access is higher in rural or remote regions than in urban areas, which is compounded by the fact that some rural residents have less disposable income than their metropolitan counterparts. Thus, to truly overcome geographical, financial, and social barriers and reach these populations, researchers must consider the broader socioeconomic context from which these access issues stem.The eHealth tools studied focused on various domains of health, including infectious diseases, child development, and neurological conditions; however, the most widely assessed domains were general and mental health. The focus on mental health tools may reflect a growing need and demand for mental health care among children and young people, increasing evidence supporting eHealth in the field of mental health, and increased funding for mental health tools [
, ]. Only one of the eHealth tools assessed multiple health domains [ ], despite this often being important for gaining a holistic picture of a child or young person’s health concerns. The development of eHealth tools that assess health multidimensionally is likely to be important in future eHealth tools, perhaps assessing not only current symptoms but also broader social or environmental factors related to the etiology and trajectory of illness and barriers to or facilitators of accessing care [ , ].Various challenges and risks were identified in relation to the implementation and use of eHealth tools. These barriers were related to the accessibility and functionality of devices, including poor internet connection [
, , , , , , , , ] and user interface or design aspects of the tool [ , , , , ]. Clinical utility was another barrier identified in papers, mainly relating to the tool lacking features that provided added value to standard care [ , , , , - ]. User uptake was a challenge, with multiple studies reporting lower than expected uptake and reluctance to use the digital tool; this was an issue reported among diverse user groups, including clinicians, parents or carers, and children or young people [ , , , , , , ]. These findings are in line with the Eysenbach [ ] law of attrition, which is based on the observation that high participant dropout rates are common in eHealth research focusing on novel digital health tools; although researchers may dismiss or underreport this information, the observation meaningfully reflects the real-world uptake of digital tools currently. Some of the reasons for low uptake included implementation barriers (eg, competing time, modified professional roles, and organizational restructuring) [ , , ], privacy concerns [ ], socioeconomic factors (eg, cost of data) [ ], and language proficiency or literacy issues [ , , ]. Incorporating participatory design (co-design) and user testing methodologies into future protocols may help to understand and address these barriers [ , ]. Data quality was another barrier that was reported, which was related to human error in data entry [ , ] or incomplete data input [ ]. A study overcame the issue of human error by minimizing free-text input and using predefined options [ ]. Although this is not feasible for all tools, such as when obtaining qualitative health information, it provides a solution for quantitative health data. Sociocultural issues were mentioned in just 1 study; Den Boer [ ] reported that parents or carers in some communities found questions about smoking in children aged 6 to 11 years insulting and inappropriate. The study researchers justified the inclusion of the questions by saying that they were important and relevant for certain communities or user groups. This raises the issue of whether universal questionnaires can be used in eHealth tools or whether customized question sets need to be developed for the target group. The findings of Singh [ ] supported a configurable or individualized approach, with the researchers concluding that individualization was critical to the clinical utility and safety of eHealth tools. Despite studies identifying challenges and risks, there was minimal discussion on how to address the identified issues; moreover, just under a quarter of studies did not report any potential risks or challenges of implementing the device [ , , , , , , - ].Our review of funding sources, which showed that financial support came from a mixture of public, commercial, and NFP bodies, is unsurprising, as the development and implementation of digital tools often involve the collaboration of professionals from multiple disciplines who belong to different bodies [
, ]. Interestingly, funding from commercial bodies was uncommon. This may reflect the fact that the tools reviewed were in the preliminary stages of research (development and implementation) rather than at a more advanced stage of commercialization, the latter of which we would expect to attract more investment from commercial organizations [ ].Recommendations for Future Research
The findings of this review demonstrate a clear need for further research into the efficacy and validity of eHealth tools that assess and track health outcomes in children and young people. Future evaluation studies should focus on changes in the health outcomes of users, as well as the clinical care pathways. Further investigation of the risks and challenges of implementing devices is also important, particularly relating to sociocultural factors, language proficiency or literacy, and privacy concerns, as these were seldom mentioned but are likely to affect the clinical utility, safety, and uptake of tools [
, ]. Overall, these findings are consistent with results from a prior systematic review of eHealth solutions in adults, which found a gap between the postulated and empirically demonstrated benefits of eHealth technologies, a lack of robust research trials into validity and efficacy, and inadequate investigation of risks or challenges of using these technologies in health care [ ]. This review has uncovered several features of eHealth tools that may facilitate comprehensive assessments and integrated care in future technologies:- Capacity for multi-informant assessment, including input from a health professional and the child or young person or their parent or caregiver
- Multidomain assessments, allowing for a holistic picture of the child or young person’s health to be captured rather than assessing health in one domain
- Tracking over time (ie, capacity and use of tools for assessment at multiple time points)
- Configurability of question sets or content depending on characteristics (eg, demographic, sociocultural, and health concerns) of the target group
- Connection to clinical care that is tailored to the child or young person’s current needs
- Trialing integration of newer technologies (eg, biosensors or wearables to collect physiological data) for relevant health domains
Limitations
Although this review provides important insights into a novel field of eHealth, the conclusions that can be drawn about the efficacy and validity of eHealth solutions are limited as most studies were formative and process evaluations that assessed feasibility, acceptability, usability, or uptake of a device. Outcome evaluation studies were rare, with just 3 randomized controlled trials conducted to date. These early phase research studies are necessary precursors to more rigorous validity and efficacy studies; however, they need to be followed by more thorough evaluation studies to determine whether the tools are effective in improving health outcomes and clinical care. We limited our search to studies published in English, which may have biased our results. Furthermore, although the strength of this review is that it presents the state of eHealth tools for supporting health in children or young people, it inevitably fails to consider the immense variation that lies within each health domain. Our search strategy was not without limitations. We did not include all relevant terms (eg, internet and technology) as the inclusion of these broader terms returned >15,000 articles, which was not considered realistic for screening. Nevertheless, we believe that the search strategy balanced scientific rigor and feasibility and was sufficiently rigorous to pick up relevant articles. Finally, this paper was not preregistered with PROSPERO; however, the search strategy remained the same over time.
Conclusions
eHealth tools that assess and track health outcomes in children or young people and connect individuals with personalized care options have enormous potential in health services around the world. Many of the existing tools are in the early stages of pilot and feasibility testing; however, the literature is promising in the potential to use these tools in future clinical care. Further research is needed to evaluate the validity and efficacy of these eHealth tools and investigate the potential risks and challenges of implementation as part of standard clinical care. With future research and development efforts in place, these tools have the potential to facilitate collaborative decision-making, improved communication, transmission of remote health data, and real-time assessment and tracking and take a positive step forward in digitalizing health practices.
Conflicts of Interest
TD is now Director, Research & Insights at the Australian Digital Health Agency.
Digital review.
DOCX File , 72 KBReferences
- UNICEF Annual Report 2017. UNICEF. 2018. URL: https://www.unicef.org/reports/unicef-annual-report-2017 [accessed 2021-12-19]
- Britto PR, Lye SJ, Proulx K, Yousafzai AK, Matthews SG, Vaivada T, Early Childhood Development Interventions Review Group‚ for the Lancet Early Childhood Development Series Steering Committee. Nurturing care: promoting early childhood development. Lancet 2017 Jan 07;389(10064):91-102. [CrossRef] [Medline]
- Lupton D. Health promotion in the digital era: a critical commentary. Health Promot Int 2015 Mar;30(1):174-183. [CrossRef] [Medline]
- Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual Research Review: digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry 2017 Apr;58(4):474-503. [CrossRef] [Medline]
- Eikelboom RH, Mbao MN, Coates HL, Atlas MD, Gallop MA. Validation of tele-otology to diagnose ear disease in children. Int J Pediatr Otorhinolaryngol 2005 Jun;69(6):739-744. [CrossRef] [Medline]
- Finocchario-Kessler S, Odera I, Okoth V, Bawcom C, Gautney B, Khamadi S, et al. Lessons learned from implementing the HIV infant tracking system (HITSystem): a web-based intervention to improve early infant diagnosis in Kenya. Healthc (Amst) 2015 Dec;3(4):190-195 [FREE Full text] [CrossRef] [Medline]
- Hashemi B, Ali S, Awaad R, Soudi L, Housel L, Sosebee SJ. Facilitating mental health screening of war-torn populations using mobile applications. Soc Psychiatry Psychiatr Epidemiol 2017 Jan;52(1):27-33. [CrossRef] [Medline]
- Iorfino F, Davenport TA, Ospina-Pinillos L, Hermens DF, Cross S, Burns J, et al. Using new and emerging technologies to identify and respond to suicidality among help-seeking young people: a cross-sectional study. J Med Internet Res 2017 Jul 12;19(7):e247 [FREE Full text] [CrossRef] [Medline]
- Mohammed A, Franke K, Okyere P, Brinkel J, Marinovic A, Kreuels B, et al. Feasibility of Electronic Health Information and Surveillance System (eHISS) for disease symptom monitoring: a case of rural Ghana. PLoS One 2018 May 24;13(5):e0197756 [FREE Full text] [CrossRef] [Medline]
- Hickie IB, Davenport TA, Burns JM, Milton AC, Ospina-Pinillos L, Whittle L, et al. Project Synergy: co-designing technology-enabled solutions for Australian mental health services reform. Med J Aust 2019 Oct;211 Suppl 7:3-39. [CrossRef] [Medline]
- Davenport TA, LaMonica HM, Whittle L, English A, Iorfino F, Cross S, et al. Validation of the InnoWell platform: protocol for a clinical trial. JMIR Res Protoc 2019 May 31;8(5):e13955 [FREE Full text] [CrossRef] [Medline]
- Rowe SC, Davenport TA, Easton MA, Jackson TA, Melsness J, Ottavio A, et al. Co-designing the InnoWell Platform to deliver the right mental health care first time to regional youth. Aust J Rural Health 2020 Apr;28(2):190-194. [CrossRef] [Medline]
- Hickie IB, Scott EM, Cross SP, Iorfino F, Davenport TA, Guastella AJ, et al. Right care, first time: a highly personalised and measurement-based care model to manage youth mental health. Med J Aust 2019 Nov;211 Suppl 9:3-46. [CrossRef] [Medline]
- Iorfino F, Cross SP, Davenport T, Carpenter JS, Scott E, Shiran S, et al. A digital platform designed for youth mental health services to deliver personalized and measurement-based care. Front Psychiatry 2019 Aug 23;10:595 [FREE Full text] [CrossRef] [Medline]
- Dunton GF, Liao Y, Intille SS, Spruijt-Metz D, Pentz M. Investigating children's physical activity and sedentary behavior using ecological momentary assessment with mobile phones. Obesity (Silver Spring) 2011 Jun;19(6):1205-1212 [FREE Full text] [CrossRef] [Medline]
- Ginsburg AS, Delarosa J, Brunette W, Levari S, Sundt M, Larson C, et al. mPneumonia: development of an innovative mHealth application for diagnosing and treating childhood pneumonia and other childhood illnesses in low-resource settings. PLoS One 2015 Oct 16;10(10):e0139625 [FREE Full text] [CrossRef] [Medline]
- March S, Day J, Zieschank K, Ireland M. The interactive child distress screener: development and preliminary feasibility testing. JMIR Mhealth Uhealth 2018 Apr 19;6(4):e90 [FREE Full text] [CrossRef] [Medline]
- den Boer JC, van Dijk W, Horn V, Hescot P, Bruers JJ. Collecting standardised oral health data via mobile application: a proof of concept study in the Netherlands. PLoS One 2018 Feb 7;13(2):e0191385 [FREE Full text] [CrossRef] [Medline]
- Dexheimer JW, Abramo TJ, Arnold DH, Johnson K, Shyr Y, Ye F, et al. Implementation and evaluation of an integrated computerized asthma management system in a pediatric emergency department: a randomized clinical trial. Int J Med Inform 2014 Nov;83(11):805-813 [FREE Full text] [CrossRef] [Medline]
- Franke KH, Krumkamp R, Mohammed A, Sarpong N, Owusu-Dabo E, Brinkel J, et al. A mobile phone based tool to identify symptoms of common childhood diseases in Ghana: development and evaluation of the integrated clinical algorithm in a cross-sectional study. BMC Med Inform Decis Mak 2018 Mar 27;18(1):23 [FREE Full text] [CrossRef] [Medline]
- Kreps GL, Neuhauser L. New directions in eHealth communication: opportunities and challenges. Patient Educ Couns 2010 Mar;78(3):329-336. [CrossRef] [Medline]
- LaMonica HM, Davenport TA, Braunstein K, Ottavio A, Piper S, Martin C, et al. Technology-enabled person-centered mental health services reform: strategy for implementation science. JMIR Ment Health 2019 Sep 19;6(9):e14719 [FREE Full text] [CrossRef] [Medline]
- LaMonica HM, Davenport TA, Burns J, Cross S, Hodson S, Veitch J, et al. Technology-enabled mental health service reform for open arms - veterans and families counselling: participatory design study. JMIR Form Res 2019 Sep 19;3(3):e13662 [FREE Full text] [CrossRef] [Medline]
- Ospina-Pinillos L, Davenport T, Diaz AM, Navarro-Mancilla A, Scott EM, Hickie IB. Using participatory design methodologies to co-design and culturally adapt the Spanish version of the mental health eClinic: qualitative study. J Med Internet Res 2019 Aug 02;21(8):e14127 [FREE Full text] [CrossRef] [Medline]
- Eng TR. eHealth research and evaluation: challenges and opportunities. J Health Commun 2002;7(4):267-272. [CrossRef] [Medline]
- National Safety and Quality Digital Mental Health (NSQDMH) standards. Australian Commission on Safety and Quality in Healthcare. 2020. URL: https://www.safetyandquality.gov.au/sites/default/files/2020-04/UPDATED%20national_safety_and_quality_digital_mental_health_standards_-_consultation_draft_-_feb_2020_0.pdf [accessed 2021-12-19]
- Draft global strategy on digital health 2020–2025. World Health Organization. 2020. URL: https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf?sfvrsn=f112ede5_42 [accessed 2021-12-19]
- Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. J Med Internet Res 2005 Feb 24;7(1):e1 [FREE Full text] [CrossRef] [Medline]
- Bashshur R, Shannon G, Krupinski E, Grigsby J. The taxonomy of telemedicine. Telemed J E Health 2011;17(6):484-494. [CrossRef] [Medline]
- Vegesna A, Tran M, Angelaccio M, Arcona S. Remote patient monitoring via non-invasive digital technologies: a systematic review. Telemed J E Health 2017 Jan;23(1):3-17 [FREE Full text] [CrossRef] [Medline]
- Adolescent health and development. World Health Organization. 2019. URL: https://tinyurl.com/232vmuy4 [accessed 2021-12-19]
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
- Types of evaluation. Centers for Disease Control and Prevention. 2019. URL: https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf [accessed 2021-12-19]
- NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Australian Government National Health and Medical Research Council. 2008. URL: https://www.mja.com.au/sites/default/files/NHMRC.levels.of.evidence.2008-09.pdf [accessed 2021-12-19]
- Dijkstra L, Florczyk A, Freire S, Kemper T, Melchiorri M, Pesaresi M, et al. Applying the Degree of Urbanisation to the globe: a new harmonised definition reveals a different picture of global urbanisation. J Urban Econ 2021 Sep;125:103312 [FREE Full text] [CrossRef]
- Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998 Jun;52(6):377-384 [FREE Full text] [CrossRef] [Medline]
- Methods for the development of NICE public health guidance. National Institute for Health and Care Excellence. URL: https://www.nice.org.uk/process/pmg4/chapter/appendix-h-quality-appraisal-checklist-qualitative-studies [accessed 2021-12-19]
- Wang J, Yao NA, Liu Y, Geng Z, Wang Y, Shen N, et al. Development of a smartphone application to monitor pediatric patient-reported outcomes. Stud Health Technol Inform 2017;245:253-257. [Medline]
- Gregory JM, Sukhera J, Taylor-Gates M. Integrating smartphone technology at the time of discharge from a child and adolescent inpatient psychiatry unit. J Can Acad Child Adolesc Psychiatry 2017;26(1):45-50 [FREE Full text] [Medline]
- Jeong Y, Chang H, Kim J. Development and feasibility of a safety plan mobile application for adolescent suicide attempt survivors. Comput Inform Nurs 2020 Aug;38(8):382-392. [CrossRef] [Medline]
- Hussey D, Flynn KC. The utility and impact of the addiction comprehensive health enhancement support system (ACHESS) on substance abuse treatment adherence among youth in an intensive outpatient program. Psychiatry Res 2019 Nov;281:112580. [CrossRef] [Medline]
- Kassam-Adams N, Kohser KL, McLaughlin J, Winston F, Marsac ML. Evaluating the acceptability and validity of assessing pain and posttraumatic stress symptoms in an adaptable eHealth system for school-age children. Clin Pract Pediatr Psychol 2019 Mar;7(1):9-19 [FREE Full text] [CrossRef] [Medline]
- Reid SC, Kauer SD, Hearps SJ, Crooke AH, Khor AS, Sanci LA, et al. A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial. BMC Fam Pract 2011 Nov 29;12:131 [FREE Full text] [CrossRef] [Medline]
- Thabrew H, D'Silva S, Darragh M, Goldfinch M, Meads J, Goodyear-Smith F. Comparison of YouthCHAT, an electronic composite psychosocial screener, with a clinician interview assessment for young people: randomized controlled trial. J Med Internet Res 2019 Dec 03;21(12):e13911 [FREE Full text] [CrossRef] [Medline]
- Rath B, Swenshon S, Haase K, Szeles T, Jung C, Jacobi F, et al. Using a mobile application to detect health needs among children and adolescents who are newly arrived migrants in Europe. J Public Health (Oxf) 2019 Dec 20;41(4):840-849. [CrossRef] [Medline]
- Svedberg P, Arvidsson S, Larsson I, Carlsson I, Nygren JM. Barriers and enablers affecting successful implementation of the electronic health service Sisom: multicenter study of child participation in pediatric care. J Med Internet Res 2019 Nov 15;21(11):e14271 [FREE Full text] [CrossRef] [Medline]
- Thompson LA, Martinko T, Budd P, Mercado R, Schentrup AM. Meaningful use of a confidential adolescent patient portal. J Adolesc Health 2016 Feb;58(2):134-140. [CrossRef] [Medline]
- Boyce S, Nyangara F, Kamunyori J. A mixed-methods quasi-experimental evaluation of a mobile health application and quality of care in the integrated community case management program in Malawi. J Glob Health 2019 Jun;9(1):010811 [FREE Full text] [CrossRef] [Medline]
- McCulloh RJ, Fouquet SD, Herigon J, Biondi EA, Kennedy B, Kerns E, et al. Development and implementation of a mobile device-based pediatric electronic decision support tool as part of a national practice standardization project. J Am Med Inform Assoc 2018 Sep 01;25(9):1175-1182 [FREE Full text] [CrossRef] [Medline]
- Alawna M, Unver B, Yuksel E. The reliability of a smartphone goniometer application compared with a traditional goniometer for measuring ankle joint range of motion. J Am Podiatr Med Assoc 2019 Jan;109(1):22-29. [CrossRef] [Medline]
- Singh H, Mallaiah R, Yadav G, Verma N, Sawhney A, Brahmachari SK. iCHRCloud: web and mobile based child health imprints for smart healthcare. J Med Syst 2017 Nov 29;42(1):14. [CrossRef] [Medline]
- Gálvez JA, Lockman JL, Schleelein LE, Simpao AF, Ahumada LM, Wolf BA, et al. Interactive pediatric emergency checklists to the palm of your hand - How the Pedi Crisis App traveled around the world. Paediatr Anaesth 2017 Aug;27(8):835-840. [CrossRef] [Medline]
- Han X, Scheetz J, Keel S, Liao C, Liu C, Jiang Y, et al. Development and validation of a smartphone-based visual acuity test (vision at home). Transl Vis Sci Technol 2019 Jul;8(4):27 [FREE Full text] [CrossRef] [Medline]
- Thabtah F. An accessible and efficient autism screening method for behavioural data and predictive analyses. Health Informatics J 2019 Dec;25(4):1739-1755 [FREE Full text] [CrossRef] [Medline]
- Angues RV, Suits A, Palmer VS, Okot C, Okot RA, Atonywalo C, et al. A real-time medical cartography of epidemic disease (Nodding syndrome) using village-based lay mHealth reporters. PLoS Negl Trop Dis 2018 Jun;12(6):e0006588 [FREE Full text] [CrossRef] [Medline]
- Binotti M, Cavallin F, Ingrassia PL, Pejovic NJ, Monzani A, Genoni G, et al. Heart rate assessment using NeoTapAdvancedSupport: a simulation study. Arch Dis Child Fetal Neonatal Ed 2019 Jul;104(4):440-442. [CrossRef] [Medline]
- Estai M, Winters J, Kanagasingam Y, Shiikha J, Checker H, Kruger E, et al. Validity and reliability of remote dental screening by different oral health professionals using a store-and-forward telehealth model. Br Dent J 2016 Oct 07;221(7):411-414. [CrossRef] [Medline]
- Detsomboonrat P, Pisarnturakit PP. Development and evaluation: the satisfaction of using an oral health survey mobile application. Telemed J E Health 2019 Jan;25(1):55-59. [CrossRef] [Medline]
- Matin SB, Wallingford A, Xu S, Ng N, Ho A, Vanosdoll M, et al. Feasibility of a mobile health tool for mothers to identify neonatal illness in rural Uganda: acceptability study. JMIR Mhealth Uhealth 2020 Feb 18;8(2):e16426 [FREE Full text] [CrossRef] [Medline]
- Kim M, Yune S, Chang S, Jung Y, Sa SO, Han HW. The fever coach mobile app for participatory influenza surveillance in children: usability study. JMIR Mhealth Uhealth 2019 Oct 17;7(10):e14276 [FREE Full text] [CrossRef] [Medline]
- van Karnebeek CD, Houben RF, Lafek M, Giannasi W, Stockler S. The treatable intellectual disability APP www. treatable-id. org: a digital tool to enhance diagnosis and care for rare diseases. Orphanet J Rare Dis 2012 Jul 23;7:47 [FREE Full text] [CrossRef] [Medline]
- Heida A, Dijkstra A, Kobold AM, Rossen J, Kindermann A, Kokke F, et al. Efficacy of home telemonitoring versus conventional follow-up: a randomized controlled trial among teenagers with inflammatory bowel disease. J Crohns Colitis 2018 Mar 28;12(4):432-441. [CrossRef] [Medline]
- Li C, Huang S, Su X, Zhang T, Jiang K. Monitoring of home recovery using the 317-nursing mobile application following day-case surgery in children: perspectives from both nurses and patients. Medicine (Baltimore) 2019 Aug;98(31):e16639 [FREE Full text] [CrossRef] [Medline]
- Padidar P, Shaker M, Amoozgar H, Khorraminejad-Shirazi M, Hemmati F, Najib KS, et al. Detection of neonatal jaundice by using an android OS-based smartphone application. Iran J Pediatr 2019 Feb 02 (forthcoming). [CrossRef]
- Rath B, Maltezou HC, Papaevangelou V, Papagrigoriou-Theodoridou MA, Alchikh M, Myles P, PEDSIDEA Network. Partnering for enhanced digital surveillance of influenza-like disease and the effect of antivirals and vaccines (PEDSIDEA). Influenza Respir Viruses 2019 Jul;13(4):309-318 [FREE Full text] [CrossRef] [Medline]
- Jiam NT, Hoon AH, Hostetter CF, Khare MM. IIAM (important information about me): a patient portability profile app for adults, children and families with neurodevelopmental disabilities. Disabil Rehabil Assist Technol 2017 Aug;12(6):599-604. [CrossRef] [Medline]
- The rural and remote implications of a national e-health strategy. National Health Rural Alliance. 2009. URL: https://www.ruralhealth.org.au/sites/default/files/position-papers/position-paper-09-07-23.pdf [accessed 2021-12-19]
- Curtin J. A digital divide in rural and regional Australia? Current Issues Briefs - Parliament of Australia. 2001. URL: https://www.aph.gov.au/About_Parliament/Parliamentary_Departments/Parliamentary_Library/Publications_Archive/CIB/cib0102/02CIB01#top [accessed 2021-12-19]
- Henry L. Bridging the urban-rural digital divide and mobilizing technology for poverty eradication: challenges and gaps. GSM Association. URL: https://www.un.org/development/desa/dspd/wp-content/uploads/sites/22/2019/03/Henry-Bridging-the-Digital-Divide-2019.pdf [accessed 2021-12-19]
- Walker J. Adding to the list of rural inequalities-Digital technology exclusion. Aust J Rural Health 2019 Oct 30;27(5):440-441. [CrossRef] [Medline]
- Christensen H, Hickie IB. Using e-health applications to deliver new mental health services. Med J Aust 2010 Jun 07;192(S11):53-56. [CrossRef] [Medline]
- Griffiths KM, Farrer L, Christensen H. The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust 2010 Jun 07;192(S11):4-11. [CrossRef] [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Wu YP, Steele RG, Connelly MA, Palermo TM, Ritterband LM. Commentary: pediatric eHealth interventions: common challenges during development, implementation, and dissemination. J Pediatr Psychol 2014 Jul;39(6):612-623 [FREE Full text] [CrossRef] [Medline]
- Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan 18;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]
Abbreviations
NFP: not-for-profit |
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
Edited by R Kukafka, G Eysenbach; submitted 26.11.20; peer-reviewed by L Ospina-Pinillos, A Čuš; comments to author 14.01.21; revised version received 15.04.21; accepted 14.10.21; published 12.05.22
Copyright©Elizabeth Stewart, Alyssa Milton, Hannah Frances Yee, Michael Jae Song, Anna Roberts, Tracey Davenport, Ian Hickie. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 12.05.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.