This paper is in the following e-collection/theme issue:

Digital Health Reviews (1174) eHealth Literacy / Digital Literacy (351) Theoretical Frameworks and Concepts (230) mHealth for Wellness, Behavior Change and Prevention (2728) Web-based and Mobile Health Interventions (3092)

Published on in Vol 23, No 6 (2021): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/23473, first published .
Application of the eHealth Literacy Model in Digital Health Interventions: Scoping Review

Application of the eHealth Literacy Model in Digital Health Interventions: Scoping Review

Application of the eHealth Literacy Model in Digital Health Interventions: Scoping Review

Authors of this article:

Mariam El Benny1 Author Orcid Image ;   Tamar Kabakian-Khasholian1 Author Orcid Image ;   Fadi El-Jardali2 Author Orcid Image ;   Marco Bardus1 Author Orcid Image
Article Authors Cited by (43) Tweetations (23) Metrics

    Review

    1Department of Health Promotion and Community Health, Faculty of Health Sciences, American University of Beirut, Beirut, Lebanon

    2Department of Health Management and Policy, Faculty of Health Sciences, American University of Beirut, Beirut, Lebanon

    Corresponding Author:

    Marco Bardus, BA, MA, PhD

    Department of Health Promotion and Community Health

    Faculty of Health Sciences

    American University of Beirut

    PO-Box 11-0236, Riad-El-Solh

    Beirut, 1107 2020

    Lebanon

    Phone: 961 1 350 0000 ext 4694

    Email: marco.bardus@gmail.com


    Background: Digital health interventions (DHIs) are increasingly being adopted globally to address various public health issues. DHIs can be categorized according to four main types of technology: mobile based, web based, telehealth, and electronic health records. In 2006, Norman and Skinner introduced the eHealth literacy model, encompassing six domains of skills and abilities (basic, health, information, scientific, media, and computer) needed to effectively understand, process, and act on health-related information. Little is known about whether these domains are assessed or accounted for in DHIs.

    Objective: This study aims to explore how DHIs assess and evaluate the eHealth literacy model, describe which health conditions are addressed, and which technologies are used.

    Methods: We conducted a scoping review of the literature on DHIs, based on randomized controlled trial design and reporting the assessment of any domain of the eHealth literacy model. MEDLINE, CINAHL, Embase, and Cochrane Library were searched. A duplicate selection and data extraction process was performed; we charted the results according to the country of origin, health condition, technology used, and eHealth literacy domain.

    Results: We identified 131 unique DHIs conducted in 26 different countries between 2001 and 2020. Most DHIs were conducted in English-speaking countries (n=81, 61.8%), delivered via the web (n=68, 51.9%), and addressed issues related to noncommunicable diseases (n=57, 43.5%) or mental health (n=26, 19.8%). None of the interventions assessed all six domains of the eHealth literacy model. Most studies focused on the domain of health literacy (n=96, 73.2%), followed by digital (n=19, 14.5%), basic and media (n=4, 3%), and information and scientific literacy (n=1, 0.7%). Of the 131 studies, 7 (5.3%) studies covered both health and digital literacy.

    Conclusions: Although many selected DHIs assessed health or digital literacy, no studies comprehensively evaluated all domains of the eHealth literacy model; this evidence might be overlooking important factors that can mediate or moderate the effects of these interventions. Future DHIs should comprehensively assess the eHealth literacy model while developing or evaluating interventions to understand how and why interventions can be effective.

    J Med Internet Res 2021;23(6):e23473

    doi:10.2196/23473

    Keywords

    eHealth literacy; digital health interventions; consumer health information; scoping review; mHealth; mobile phone 


    Digital Health Interventions

    In the last 20 years, digital health or eHealth has emerged as an important research field. At the intersection of medical informatics, public health, and business, eHealth refers to the use of “health services and information delivered or enhanced through the Internet and related technologies” [1]. Technologies such as web based, mobile based, telehealth, and electronic health records (EHRs) have become widely adopted in the so-called digital health interventions (DHIs). DHIs can be defined as “health services delivered electronically through formal or informal care. DHIs can range from electronic medical records used by providers to mobile health (mHealth) apps used by consumers” [2]. The World Health Organization has recently produced a classification of DHIs, identifying four main types: clients, health care providers, health systems, and data services [3]. On PubMed, as of August 3, 2020, the number of records mentioning eHealth or DHIs in their title or abstract has consistently increased over the past 20 years, starting from 65 in 2000 to 11,395 in 2019, reaching a total of 6720.

    Some systematic reviews and meta-analyses have described the effectiveness of DHIs in addressing various public health problems, such as somatic diseases [4], or health literacy and health outcomes [5]. Nevertheless, it is still unclear what makes DHIs superior to nondigital interventions or what components of these interventions facilitate positive outcomes reported [6]. In addition, it is unclear whether DHIs are effective because of their content or the manner in which they are delivered. Regarding the content of interventions, some systematic reviews have focused on exploring the way people process and understand information available on the internet [7,8]. In fact, with so many resources and information available on the internet, patients and users enrolled in DHIs may face challenges in understanding and making sense of the information they receive. Some research has focused on problems related to the ability to process information derived from web-based sources or delivered through technologies.

    The eHealth Literacy Model

    In 2006, Norman and Skinner [9] proposed a conceptual model that encompasses six different domains of literacy required to process information from technology sources: traditional literacy, health literacy, information literacy, scientific literacy, media literacy, and computer literacy. According to Norman and Skinner [9], traditional or basic functional literacy includes simple and primitive literacy skills, including the ability to read and understand text and the ability to speak and write in a certain language. Information literacy includes the ability to know how knowledge is structured and how information can be used in a certain way that informs other people. Media literacy is the capability to critique a media subject and place information in different contexts. Health literacy, coined in the 1970s, can be generally defined as “the degree to which individuals can obtain, process, understand, and communicate about health-related information needed to make informed health decisions” (as reported by Berkman et al [10]). According to Norman and Skinner [9], health literacy is the ability to perform basic reading and numerical tasks required to function in the health care environment; patients with adequate health literacy can read, understand, and act on health care information. More recent evolutions of the concept include a variety of competencies and skills, including knowledge, motivation, and competencies related to accessing, understanding, appraising, and applying health-related information in health care, disease prevention, and health promotion settings [11]. Several systematic reviews have analyzed the relationship between health literacy and a variety of health outcomes, indicating that a good level of health literacy is generally associated with positive health outcomes in various health domains, such as vaccination [12], noncommunicable diseases (NCDs) such as chronic kidney disease or coronary artery disease, heart failure [13-15], oral health [16], quality of life [17], and excess body weight [18]. Some other review evidence has shown how interventions promoting critical health literacy [19] could be very beneficial for the community [20] or among specific segments of the population, such as adolescents [21] or older adults [22].

    Strictly related to the concept of eHealth is computer or technology literacy, which is the capability to use new technologies and software and the ability to access electronic health information [9]. Recent conceptualizations expand this domain to look at the ability to process information, to engage with patients’ own health, at the motivation and ability to engage with digital devices, at feeling safe and in control, at having access to health care and technological systems that work, and at meeting digital services that suit individuals’ needs [23]. Norman and Skinner [24] have developed a scale to assess eHealth literacy, called eHealth literacy scale (eHEALS), which has been one of the most adopted and cited, with 449 citations on the Journal of Medical Internet Research page and more than 1320 results on Google Scholar (as of August 3, 2020). The last domain of the eHealth literacy model, scientific literacy, involves the ability to allocate health-related findings in the right context by systematically understanding the “nature, aims, methods, applications, limitations, and politics” of building knowledge [9]. Several systematic reviews have analyzed the relationship between health literacy in mHealth apps and interventions [5,7,8,25,26], generally reporting positive associations among health literacy, digital literacy, and health outcomes. Other reviews have specifically examined how technology can affect health literacy in health programs [27-29].

    According to the developers of the eHealth literacy model, the six domains can be grouped into two main categories: analytic (traditional, media, and information) and context-specific (health, scientific, and computer). The analytical category refers to a set of competencies that can be applied to a wide range of information sources, whereas context-specific categories include competencies that can only be applied to a specific problem in a specific context [9]. For example, the ability of a person living with type 2 diabetes to process information related to diabetes is different from their ability to process information related to vaccines, mental health, or other chronic conditions. Similarly, the ability to use a mobile phone to call someone does not necessarily translate into the ability to use a mobile app, navigate a website, or evaluate the information retrieved while searching on the internet.

    Related Work and Study Aims

    Arguably, researchers developing DHIs should always take into account the domains of computer or technology literacy and health literacy, as these are potential pathways for more effective and equitable interventions [30]. Health literacy can be viewed as both an outcome and a mediator in interventions intended to improve health outcomes [31]. Technologies or delivery modes can also be seen as interacting or moderating factors [32], depending on the type of technology used to deliver an intervention on a specific health topic. DHIs can be developed to improve health literacy (outcome) or they can be developed to improve clinical outcomes in which one or more dimensions of the eHealth literacy model are considered as mediators or moderators of the effects of the intervention. Researchers developing DHIs could then assume that people enrolling in these interventions should have good levels of functional, scientific, media, and information literacy to understand how to write or read information they are exposed to.

    However, to what extent are these assumptions tenable? In other words, is the eHealth literacy model purely conceptual or does it find a concrete application in DHIs? To the best of our knowledge, there are no systematic reviews that specifically discuss the application of the complete eHealth literacy model in DHIs. When we were developing the search strategies for this project, we searched for existing systematic reviews in PubMed and PROSPERO databases with the keyword eHealth literacy and identified only four systematic reviews [33-36]. However, all these reviews have focused on the domain of digital literacy, looking at specific health outcomes in specific segments of the population, such as people living with HIV [33], underserved populations in the United States [34], older adults [35], or college students [36]. Therefore, this scoping review aims to identify and describe DHIs that assess any domain of the eHealth literacy model and to identify which domains are assessed and evaluated the most. We considered DHIs that were developed to improve clinical outcomes or that were aimed at different literacies, according to the eHealth literacy model. In other words, we considered interventions that looked at eHealth literacy either as an outcome or as a mediator of intervention effects, as long as the domains of the eHealth literacy model were assessed.


    Overview

    We followed the scoping review framework by Arksey and O’Malley [37], which encompasses five stages: (1) identification of the initial research questions; (2) identification of relevant studies; (3) study selection; (4) charting the data; and (5) collating, summarizing, and reporting the results. The stages are described further in the following sections.

    Stage 1: Identifying the Research Question

    The main review question, based on the eHealth literacy model, was “To what extent are DHIs assessing the 6 domains of the eHealth literacy model?” More specifically, we wanted to answer the following research questions: What domains of the eHealth literacy model (ie, computer, health, traditional, media, information, and science literacy) are assessed and reported in the literature? What health conditions have been investigated? What technologies are used?

    Stage 2: Identifying Relevant Studies

    We searched four electronic databases that cover most of the medical and public health literature: MEDLINE, CINAHL, Embase, and Cochrane Library.

    We used a predefined search strategy, encompassing keywords and medical subject headings to cover three main concepts: eHealth literacy model, digital health, and the study design for interventions. The eHealth literacy model concept entailed terms such as health literacy, literacy, computer literacy, information literacy, basic, functional, scientific, media, information, computer, health, eHealth, literacy, literate, illiteracy, and illiterate. The second concept, digital health, expanded on the above and entailed keywords, such as telemedicine, internet, mobile, phone, digital, medium or media, mHealth, eHealth, telemedicine, and computer, based on other systematic reviews recently conducted by one of the authors [6,38,39]. The third concept, that is, the research design, entailed a predefined set of keywords and operands that Cochrane has developed to identify randomized controlled trials (RCTs); this is because we wanted to identify the best level of evidence available [40]. The search strategy used for MEDLINE is provided in Multimedia Appendix 1. Database searches were completed, and references were retrieved on January 24, 2020.

    In addition, we used the reference list of identified systematic reviews on the topic to identify other potentially relevant studies.

    Stage 3: Study Selection

    We followed the Joanna Briggs’ Institute’s PCC (Population-Concept-Context) framework [41,42] to define our inclusion criteria, as it applies to scoping reviews. We included studies that discussed DHIs (concepts) and reported the assessment of at least one domain of the eHealth literacy model (context). In this context, we conceived the dimensions of the eHealth literacy model as either outcomes or mediators of DHIs. The assessment of the different types of literacy was considered a sufficient indicator for DHIs considering such dimensions as outcomes or mediators of intervention effects. We did not restrict the results to any population, with the idea of inductively categorizing the results according to health condition, hence defining the population of reference in the analytical phase.

    The screening process consisted of two stages: title and abstract as well as full-text screening. The first stage involved 2 reviewers (MEB and MB) and one research assistant, who independently screened all records identified by the searches. This task was completed using a web-based application for systematic reviews, Rayyan [43]. The interrater reliability was excellent (agreement 96%; Cohen κ=0.834; Gwet AC1=0.950). All records with disagreement among the 3 reviewers were automatically included in the full-text screening stage. The full-text screening stage was completed by the first author with the help of a research assistant and verified by the fourth author. All disagreements were resolved through discussion.

    Stage 4: Charting the Data

    For each retrieved record, 2 authors (MEB and MB) extracted the following information into a Microsoft Excel spreadsheet: first author name, year of publication, article title, journal, and number of trial registry (if available), principal investigator name (if available), country of the first author or of the principal investigator (if available). This information was used to identify and map articles pertaining to the same study. In the full-text stage, we also extracted text to verify whether the record included a digital component, was based on a randomized controlled design, focused on specific health conditions, and measured and reported results related to one of the domains of the eHealth literacy model (health, computer, basic or functional, information, media, and scientific literacy).

    When multiple records were available for one study, we chose the country of origin of the first author or of the principal investigator listed in the study protocol; we chose the year of publication of the first published article available.

    On the basis of the information extracted, we categorized studies according to the domains of the eHealth literacy model (ie, health, computer, basic or functional, information, media, and scientific literacy). We also inductively categorized the studies according to the health conditions described. When multiple conditions were reported, we categorized the study as having multiple conditions. Finally, we inductively categorized the interventions according to four main types of technology: (1) mobile-based, including mobile apps, text messages, and interactive voice response, exclusively designed for mobile or other handheld devices; (2) web-based, including those designed for being accessed via computer, explicitly labeled as web- or internet-based, online, and e-learning, delivered through bespoke websites or social media outlets, such as social networking sites (eg, Facebook or Twitter)—social media are web-based apps that can be accessed via different devices connected to the internet, including smartphones [39,44,45]; (3) telehealth, comprising telerehabilitation, telemedicine, or other interventions focused on distributing services and information via electronic information and telecommunication devices [46]; (4) EHRs, focusing on EHRs that are defined as “a repository of patient data in digital form, stored and exchanged securely, and accessible by multiple authorized users” [47]. Telehealth and EHR interventions use the internet to connect various devices, including tablets and mobile phones, yet they represent a different type of delivery mode and format: EHRs. We labeled interventions using a combination of the modes described earlier, as reported in other studies [6,48]. When studies reported a combination of the abovementioned categories, we categorized the study as a hybrid.

    The first author and a research assistant independently completed the classifications; in case of inconsistencies or disagreements between the classifications, the fourth author acted as a third reviewer and resolved the disagreements through discussion. All the authors agreed with the final categorization.

    Stage 5: Collating, Summarizing, and Reporting the Results

    We performed a descriptive analysis of the characteristics of the included papers and reported the results by year of publication, the country of origin of the study authors, eHealth literacy domain, health condition, and type of technology used.


    Search Results

    The electronic database search yielded 4135 records. The selection process is summarized in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram shown in Figure 1. Briefly, after removing duplicates, the titles and abstracts of 3138 records were independently screened by 3 reviewers. During the title and abstract screening, we excluded 2661 records that were deemed irrelevant. The remaining 477 records were assessed for eligibility in the full text. Scanning the reference lists of two relevant systematic reviews [5,49] allowed us to identify seven other eligible records. We evaluated the eligibility of 484 records that were screened in full text. Of these, 326 records were excluded for the following reasons: 48 did not discuss DHIs (wrong context); 72 reported on digital interventions but did not use an RCT or randomized clinical trial design (wrong study design); 193 records discussed DHIs but did not report any type of literacy (no relevant outcome assessed or reported); 4 were duplicate records; and for the remaining 9 records, we could not retrieve a PDF file. The list of excluded references is provided in Multimedia Appendix 2. Overall, we included 158 records: 79 records reported concluded interventions and 79 records reported protocols of ongoing studies, without reporting results. These were included because they described the assessment of some domains of the eHealth literacy model. The 158 records described a total of 131 unique studies.

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram. RCT: randomized controlled trial.
    View this figure

    Characteristics of the Included Studies

    Publication Year and Geographic Distribution

    As shown in Figure 2, most of the selected studies were conducted in the last 4 years (86/131, 65.6%), followed by an exponential trend, peaking in 2019 (30/131, 22.9%) and ranging from 2001 to 2020.

    The studies were conducted in 26 countries (Table 1). Most studies were conducted across 3 countries (81/131, 61.8%), including the United States (43/131, 32.8%), Australia (28/131, 21.3%), and the United Kingdom (10/131, 7.6%). Approximately one-third of the studies (38/131, 29%) were conducted in European countries such as the United Kingdom (10/38, 26%); Germany (8/38, 21%); Denmark (5/38, 13%); Sweden (4/38, 11%); the Netherlands (3/38, 8%); Norway (2/38, 5%); and Belgium, Finland, Luxemburg, Ireland, Slovakia, and Switzerland (1/38, 3% each). Asian countries were represented by Iran (4/16, 25%); Turkey (3/16, 19%); Hong Kong (2/16, 13%); Singapore (2/16, 13%); Japan (2/16, 13%); and Jordan, Malaysia, and Pakistan (1/16, 6% each). Overall, only 0.8% (1/131) of studies were conducted in Africa (South Africa) and 22.1% (29/131) in Oceania (New Zealand: 1/29, 3%; Australia: 28/29, 97%).

    In the following sections, we have reported the results according to our research objectives, whereas a table with the detailed characteristics of the selected studies is provided in Multimedia Appendix 3.

    ‎
    Figure 2. Distribution of selected studies (N=131) by year of publication.
    View this figure
    Table 1. Distribution of included studies by country (N=131).
    CountryStudies, n (%)
    United States43 (32.8)
    Australia28 (21.3)
    United Kingdom10 (7.6)
    Germany8 (6.1)
    Denmark5 (3.8)
    Sweden4 (3.1)
    Iran4 (3.1)
    Netherlands3 (2.2)
    Turkey3 (2.2)
    Brazil2 (1.5)
    Canada2 (1.5)
    Hong Kong2 (1.5)
    Japan2 (1.5)
    Norway2 (1.5)
    Singapore2 (1.5)
    Belgium1 (0.7)
    Finland1 (0.7)
    Ireland1 (0.7)
    Jordan1 (0.7)
    Luxemburg1 (0.7)
    Malaysia1 (0.7)
    New Zealand1 (0.7)
    Pakistan1 (0.7)
    Slovakia1 (0.7)
    South Africa1 (0.7)
    Switzerland1 (0.7)
    Domains of the eHealth Literacy Model Assessed

    Figure 3 presents the years of publication of the included studies grouped by domain of the eHealth literacy model. In total, 2.2% (3/131) of the included studies were published before 2006, when the seminal publications of the eHealth literacy model appeared [9,24]. These studies included an assessment of computer literacy. Studies published in 2008 mostly reported assessments of health literacy.

    Of the 131 studies included, none assessed or measured all six domains of the eHealth literacy model. Most of the studies (124/131, 94.6%) focused on one of the six domains of the eHealth literacy model; only 5.3% (7/131) of studies reported the assessment of two domains, namely health literacy and digital or computer literacy. Most of the studies that reported on one literacy domain (124/131, 94.7%) focused on health literacy (95/124, 76.6%), followed by digital or computer literacy (19/124, 15.3%), basic or functional literacy (4/124, 3.2%), media literacy (4/124, 3.2%), information literacy (1/124, 0.8%), and scientific literacy (1/124, 0.8%).

    ‎
    Figure 3. Distribution of studies (N=131) by eHealth literacy model domain.
    View this figure
    Health Conditions Addressed and Technologies Used

    Table 2 provides a summary of the selected studies grouped by technology category and health condition category.

    A large number of studies (61/131, 46.5%) discussed interventions addressing NCDs, such as hypertension, obesity, end-stage kidney disease, type 2 diabetes, chronic kidney disease, heart disease (vascular disease, cerebrovascular disorders, ischemic heart disease, coronary artery disease, and heart failure), fibromyalgia syndrome, and asthma. Of these 61 NCD-focused studies, 3 (5%) also discussed mental health topics, and 1 (2%) covered sexual and reproductive health. The second most covered category of health conditions was mental health (26/131, 19.8%), including depression, eating disorders, mental and behavioral disorders, anxiety, and suicide prevention. Other topics included health education (16/131, 12.2%), such as health promotion, health communication, patient provider communication and literacy, aging and maternal and infant health (4/131, 3.0% of studies), sexual and reproductive health, and substance use (3/131, 2.2% of studies each). The remaining 11.4% (15/131) studies covered a variety of health topics.

    Table 2. Number of studies by health condition category and type of technology used.
    Health condition and technology usedWeb based (n=68), n (%)Mobile based (n=40), n (%)Telehealth (n=10), n (%)EHRsa (n=5), n (%)Hybrid (n=8), n (%)Total (N=131), n (%)
    NCDsb22 (32.4)19 (47.5)6 (60)3 (60)6 (75)56 (42.7)
    NCDs—mental health1 (1.5)2 (5)0 (0)0 (0)0 (0)3 (2.3)
    NCDs—sexual and reproductive health0 (0)0 (0)0 (0)0 (0)1 (12.5)1 (0.8)
    Mental health21 (30.9)4 (10)1 (10)0 (0)0 (0)26 (19.8)
    Aging2 (2.9)1 (2.5)1 (10)0 (0)0 (0)4 (3.1)
    Health education topics9 (13.2)4 (10)1 (10)2 (40)0 (0)16 (12.2)
    Maternal and infant health2 (2.9)2 (5)0 (0)0 (0)0 (0)4 (3.1)
    Sexual and reproductive health2 (2.9)1 (2.5)0 (0)0 (0)0 (0)3 (2.3)
    Substance use2 (2.9)1 (2.5)0 (0)0 (0)0 (0)3 (2.3)
    Other health topics7 (10.3)6 (15)1 (10)0 (0)1 (12.5)15 (11.5)

    aEHR: electronic health record.

    bNCD: noncommunicable disease.

    With regard to the technologies used, most studies included web-based interventions (68/131, 51.9%), followed by mobile-based (40/131, 30.5%), telehealth (10/131, 7.6%) EHRs (5/131, 3.8%), and hybrid interventions (8/131, 6.1%). Examples of web-based technology included e-learning portals for specialized training [50,51], experimental websites, and social media platforms [52-54], which are used to deliver motivational or informational campaigns. Mobile-based interventions included health apps [55-57], SMS text messaging or WhatsApp [58], games [59,60], and interactive voice response [61,62]. Telehealth interventions included rehabilitation programs [63,64] or remote counseling [65]. Hybrid interventions included combinations of mobile apps and EHRs [55-57], SMS text messaging and EHRs [66], or a mix of web- and mobile-based technologies [67].

    Among web-based interventions (n=68), most focused on NCDs (22/68, 32%), mental health (21/68, 31%), and health education topics (9/68, 13%). Mobile-based interventions (n=40) followed a similar pattern, with approximately half of the studies focusing on NCDs (19/40, 48%) or other health topics (6/40, 15%). Most telehealth (6/10, 60%), EHR (3/5, 60%), and hybrid (6/8, 75%) interventions focused on NCDs.


    Principal Findings

    This is the first scoping review examining the extent to which DHIs have assessed, accounted for, and reported any of the six domains of the eHealth literacy model by Norman and Skinner [9]. We identified a sizable literature discussing DHIs developed in 26 countries, spanning two decades. The eHealth literacy model [9] and eHEALS [24] date back to 2006, but we included 3 studies that were published before that year and all assessed computer literacy. This might indicate that attention toward the ability to use technology was a research interest in the early 2000s. However, this interest has not grown exponentially and concomitantly with the growth of DHIs. It is interesting to observe that the assessment of digital literacy has grown only after 2015, but it has remained below the assessment of health literacy, which was the domain assessed the most over time. There is no clear explanation for these trends. A bibliometric analysis of the studies cited in the seminal papers mentioned above could reveal the connections between publications and demonstrate when the eHealth literacy model has received more citations.

    Most of the evidence comes from the Global North, that is, from English-speaking countries including the United States, Australia, and the United Kingdom. A few studies have been conducted in countries of the Global South, such as Africa, Latin America, or South East Asia. This finding is consistent with that reported in a recent scoping review on digital health innovations [68] and in a recent bibliometric analysis of research on mHealth apps [69], which showed a predominance of articles published in the United States, the United Kingdom, Australia, and Canada. Publication bias and limited evidence from developing countries or the Global South has been previously reported in the literature [70-72], yet there seems to be a lack of evidence on DHIs from Africa, the Middle East, South America, or Southeast Asia. There may be various reasons for this absence of evidence. First, research on digital technologies might not have reached an advanced stage to produce interventions with the highest level of evidence (ie, RCTs). Second, the existing digital divide might persist in many countries, both low- and high-income countries [30]; however, mobile phones and telemedicine are becoming more widely adopted [46,73]. Third, researchers based in low- and middle-income countries (LMICs) may be published in languages other than English or might have limited English language proficiency, but this latter assumption does not seem to be grounded in evidence [74,75]. Another reason might be that researchers in LMICs might choose to publish in journals that are not indexed in the databases we searched. Alternatively, researchers in LMICs might not have the possibility to publish their results because of a lack of funding for open access publications or because editors demonstrate publication bias [72]. Regardless of the reasons, we call for digital health researchers based in countries of the Global South to publish more study protocols and diffuse intervention results; we also call the international community of editors and publishing houses to incentivize or support research published from these underrepresented countries, so that stronger conclusions can be drawn from a truly global evidence base.

    Domains of the eHealth Literacy Model Assessed

    Our findings showed that none of the 131 selected DHIs conducted in the last 20 years accounted for or assessed all six domains of the eHealth literacy model. Although these interventions were included because they assessed at least one domain of the model, only 5.3% (7/131) of studies included the assessment of more than one domain. These 7 studies assessed only digital and health literacy. Our study also shows that most DHIs have assessed and evaluated health literacy [19] among intervention participants, which is an important factor that can determine the health outcomes of a study [21,31,32]. Although the focus on health literacy in DHIs is consistent with some literature reviews combining the study of health literacy identified through our searches [5,22,26,34,35,49], it is somewhat surprising that none of the other four domains of the eHealth literacy model were concomitantly addressed.

    There are numerous explanations for these findings. First, researchers specialized in DHIs might not be familiar with or might have ignored the original model, even though the seminal paper by Norman and Skinner [9] and the paper describing the eHEALS [24] are highly cited (as of October 17, 2020, Google Scholar showed 1128 and 1450 citations, respectively). Second, researchers might have decided to focus on other domains of the model while making implicit or explicit assumptions about the levels of literacy in other domains. For example, the limited evidence related to the assessment of the domains of scientific, information, media, and functional literacy might be based on the assumption that digital literacy instruments, such as the popular eHEALS [24], include questions related to the use of information on the internet as a medium of search information; hence, these could be associated with media and information literacy domains. However, there exist several instruments that specifically assess media literacy [76,77], scientific literacy [78,79], and information literacy [80,81]. Moreover, Norman and Skinner [9] did not consider overlapping elements when they developed the eHealth literacy model, which considers the six domains as distinct and separate.

    Although intervention designers should aim to develop content that is understood by people with low functional literacy [82,83], this fact should be proven or verified by the same intervention designers. One way to do so is to assess functional literacy or to report the level of literacy rather than to just develop the content of the intervention through formal readability and usability testing. The fact that other domains of the eHealth literacy model were not always conducted raises concerns about the generalizability of such interventions across the eHealth literacy spectrum. DHIs tend to attract tech-savvy, healthy volunteers who have access to technology and who might have different sociodemographic and psychological profiles compared with people who belong to vulnerable segments of the population and do not have access to technology [30].

    Another important finding was that few identified DHIs assessed digital literacy (n=26). Not assessing digital literacy is based on the assumption that all participants are equally able to use technology and are able to make sense of the information delivered. This assumption might not be tenable in all contexts, and it does not allow researchers to understand whether participants appropriately received the intervention. In other words, health literacy is context-specific and varies according to different situations and topics. Arguably, health and digital literacy might act as moderators of intervention effects and not including these factors might underestimate or overestimate intervention effects [84].

    The limited assessment of digital literacy in DHIs also raises some ethical considerations in terms of equity and social justice, as these interventions tend to attract highly educated, healthy, and digitally literate individuals who have easy access to technology, leaving out less-educated and poorer segments of the population, who may be most in need of the interventions themselves [30,85]. This selection bias isolates segments of the population that are traditionally difficult to reach [86,87], yet it is important to acknowledge that the results of DHIs might be less generalizable than interventions that do not use technology.

    Another reason for the absence of a comprehensive and accurate assessment of the six domains of the eHealth literacy model might be due to the fact that this assessment will be unfeasible and daunting for the participants. Holding constant the basic or functional literacy (ie, numeracy and ability to read), assessing all six domains using existing scales for media, scientific, health, digital, and information literacy would require longer questionnaires that will take more time to complete, which might discourage participation in these studies. For example, one of the most used instruments to assess digital literacy is the relatively short (8 items) eHEALS [24]. However, for context-specific domains such as health literacy [9], there are many more instruments available, which vary in length and complexity [25,88]. A recent review identified 43 different instruments [89], and the Health Literacy Toolshed database included 200 measures [90]. Similar issues of measurement pertain to the assessment of literacy in a digital world [91], including media literacy [92]. Nevertheless, we urge digital health researchers to find ways to assess and evaluate the different domains of the eHealth literacy model, so that they can gain a better understanding of the study participants’ characteristics, abilities, and needs. If measuring all domains might appear unfeasible, we suggest that DHI researchers prioritize the assessment of digital literacy—using the short eHEALS [24]—and health literacy, which is context specific, according to the model by Norman and Skinner [9]. Once the health topic or context is defined (mental health, breast cancer, etc), the choice of a short, yet valid instrument to assess health literacy in that context would become easier. As digital health and health literacy can change due to the intervention itself, we recommend assessing these constructs before and after the intervention. Finally, media, scientific, and traditional literacy are analytical skills that are not specific to any context; it would be easier for researchers to routinely assess these domains before the start of any intervention.

    Health Conditions Addressed and Technologies Used

    This scoping review showed that the selected DHIs published in the last 20 years focused mostly on NCDs, delivered via web- or mobile-based platforms. This is consistent with the findings of a few recent scoping reviews focusing on research on DHIs for behavior change [93,94] or in a recent bibliometric analysis of mHealth apps [69]. Although most DHIs have covered NCDs and mental health, there are many avenues for digital health. Further systematic reviews could be developed to specifically qualify and quantify the effectiveness of DHIs delivered via web or mobile phones in reducing NCDs and mental health issues. These systematic reviews could also anticipate sensitivity analyses based on the modes of delivery, length of the interventions, or the assessment of eHealth literacy model domains. This scoping review provides a valuable map of the evidence and sets the research agenda for DHIs in the coming years.

    Strengths and Limitations

    To the best of our knowledge, this is the first scoping review that systematically examined evidence pertaining to the application of the eHealth literacy model by Norman and Skinner [9] in DHIs. We looked at the highest quality of evidence, following a predetermined search strategy and a systematic approach to appraise the literature, without restricting our searches to specific periods, populations, countries, or health conditions. Nevertheless, this study has some limitations that are common to many other systematic or scoping reviews. These limitations include the fact that we looked only at peer-reviewed articles available in English. It is possible that some evidence on the use of the eHealth literacy model could have been reported in non–peer-reviewed or gray literature. Another limitation is related to the use of an RCT filter and focus on the RCT study design. Although RCTs provide the highest level of evidence, according to Grading of Recommendations Assessment, Development and Evaluation standards [40], it might be possible that some relevant research entailed the use of other types of study designs.

    Conclusions

    This review suggests that future DHIs should focus more on the assessment of the eHealth literacy domains when developing a DHI, especially the domains that are assessed the least, such as scientific, media, basic, and information literacy. Even though assessing all domains of the eHealth literacy model might be unfeasible, it would allow researchers to account for factors that might moderate or mediate the effects of the interventions on the targeted health outcomes.

    Future systematic reviews should be conducted to examine the effects of DHIs on various health outcomes identified in this review by anticipating subgroup or sensitivity analyses comparing different types of intervention, delivery modes, and most importantly different levels of health literacy or digital literacy.

    Acknowledgments

    The authors wish to thank Mrs Layal Hneiny, a medical librarian at the American University of Beirut, for her help during the development of the search strategies and Ms Fatma Barakat and Ms Dalia Sarieldine for their help with the screening and data selection of included articles.

    Authors' Contributions

    This study is the result of the Integrative Learning Experience carried out by MEB as a partial fulfillment of the Master of Public Health, offered by the Faculty of Health Sciences, American University of Beirut, Lebanon. TKK and FEJ acted as advisors and second readers of the project, respectively; MB acted as the project’s preceptor and supervisor. MB and MEB conceived and designed the review and MB coordinated it. TKK and FEJ provided intellectual feedback in the development of the study. MEB and MB were involved in developing the search strategy, extracted the data, completed the analyses, and interpreted the data. MEB and MB drafted the manuscript. All authors have reviewed and approved the final version of the manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    MEDLINE search strategy.

    DOCX File , 25 KB
    ‎
    Multimedia Appendix 2

    Records excluded during the full-text screening phase with full citations.

    DOCX File , 74 KB
    ‎
    Multimedia Appendix 3

    Characteristics of included studies with references.

    DOCX File , 265 KB
    1. Eysenbach G. What is e-health? J Med Internet Res 2001;3(2):E20 [FREE Full text] [CrossRef] [Medline]
    2. Soobiah C, Cooper M, Kishimoto V, Bhatia RS, Scott T, Maloney S, et al. Identifying optimal frameworks to implement or evaluate digital health interventions: a scoping review protocol. BMJ Open 2020 Aug 13;10(8):e037643 [FREE Full text] [CrossRef] [Medline]
    3. Classification of digital health interventions v1.0 : a shared language to describe the uses of digital technology for health. World Health Organization. 2018.   URL: https://tinyurl.com/ybd4kedp [accessed 2021-05-17]
    4. Elbert NJ, van Os-Medendorp H, van Renselaar W, Ekeland AG, van Roijen LH, Raat H, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res 2014 Apr 16;16(4):e110 [FREE Full text] [CrossRef] [Medline]
    5. Jacobs RJ, Lou JQ, Ownby RL, Caballero J. A systematic review of eHealth interventions to improve health literacy. Health Informatics J 2016 Jun;22(2):81-98 [FREE Full text] [CrossRef] [Medline]
    6. Bardus M, Smith JR, Samaha L, Abraham C. Mobile phone and web 2.0 technologies for weight management: a systematic scoping review. J Med Internet Res 2015 Nov 16;17(11):e259 [FREE Full text] [CrossRef] [Medline]
    7. Car J, Lang B, Colledge A, Ung C, Majeed A. Interventions for enhancing consumers' online health literacy. Cochrane Database Syst Rev 2011 Jun 15(6):CD007092 [FREE Full text] [CrossRef] [Medline]
    8. Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res 2015 May 07;17(5):e112 [FREE Full text] [CrossRef] [Medline]
    9. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res 2006 Jun 16;8(2):e9 [FREE Full text] [CrossRef] [Medline]
    10. Berkman ND, Davis TC, McCormack L. Health literacy: what is it? J Health Commun 2010;15 Suppl 2:9-19. [CrossRef] [Medline]
    11. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, (HLS-EU) Consortium Health Literacy Project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012 Jan 25;12:80 [FREE Full text] [CrossRef] [Medline]
    12. Lorini C, Santomauro F, Donzellini M, Capecchi L, Bechini A, Boccalini S, et al. Health literacy and vaccination: a systematic review. Hum Vaccin Immunother 2018 Feb 01;14(2):478-488 [FREE Full text] [CrossRef] [Medline]
    13. Cajita MI, Cajita TR, Han H. Health literacy and heart failure: a systematic review. J Cardiovasc Nurs 2016;31(2):121-130 [FREE Full text] [CrossRef] [Medline]
    14. Taylor DM, Fraser S, Dudley C, Oniscu GC, Tomson C, Ravanan R, ATTOM investigators. Health literacy and patient outcomes in chronic kidney disease: a systematic review. Nephrol Dial Transplant 2018 Sep 01;33(9):1545-1558. [CrossRef] [Medline]
    15. Ghisi GL, Chaves GS, Britto RR, Oh P. Health literacy and coronary artery disease: a systematic review. Patient Educ Couns 2018 Feb;101(2):177-184. [CrossRef] [Medline]
    16. Firmino RT, Ferreira FM, Paiva SM, Granville-Garcia AF, Fraiz FC, Martins CC. Oral health literacy and associated oral conditions: a systematic review. J Am Dent Assoc 2017 Aug;148(8):604-613. [CrossRef] [Medline]
    17. Zheng M, Jin H, Shi N, Duan C, Wang D, Yu X, et al. The relationship between health literacy and quality of life: a systematic review and meta-analysis. Health Qual Life Outcomes 2018 Oct 16;16(1):201 [FREE Full text] [CrossRef] [Medline]
    18. Michou M, Panagiotakos DB, Costarelli V. Low health literacy and excess body weight: a systematic review. Cent Eur J Public Health 2018 Sep;26(3):234-241 [FREE Full text] [CrossRef] [Medline]
    19. Chinn D. Critical health literacy: a review and critical analysis. Soc Sci Med 2011 Jul;73(1):60-67. [CrossRef] [Medline]
    20. de Wit L, Fenenga C, Giammarchi C, di Furia L, Hutter I, de Winter A, et al. Community-based initiatives improving critical health literacy: a systematic review and meta-synthesis of qualitative evidence. BMC Public Health 2017 Jul 20;18(1):40 [FREE Full text] [CrossRef] [Medline]
    21. Fleary SA, Paasche-Orlow MK, Joseph P, Freund KM. The relationship between health literacy, cancer prevention beliefs, and cancer prevention behaviors. J Cancer Educ 2019 Oct;34(5):958-965 [FREE Full text] [CrossRef] [Medline]
    22. Chesser AK, Woods NK, Smothers K, Rogers N. Health literacy and older adults: a systematic review. Gerontol Geriatr Med 2016;2:A [FREE Full text] [CrossRef] [Medline]
    23. Norgaard O, Furstrand D, Klokker L, Karnoe A, Batterham R, Kayser L, et al. The e-health literacy framework: a conceptual framework for characterizing e-health users and their interaction with e-health systems. Knowl Manage E-Learn Int J 2015;7(4):540. [CrossRef]
    24. Norman CD, Skinner HA. eHeals: the eHealth literacy scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [CrossRef] [Medline]
    25. Collins SA, Currie LM, Bakken S, Vawdrey DK, Stone PW. Health literacy screening instruments for eHealth applications: a systematic review. J Biomed Inform 2012 Jun;45(3):598-607 [FREE Full text] [CrossRef] [Medline]
    26. Kim H, Xie B. Health literacy in the eHealth era: a systematic review of the literature. Patient Educ Couns 2017 Jun;100(6):1073-1082. [CrossRef] [Medline]
    27. Dominick G, Friedman D, Hoffman-Goetz L. Do we need to understand the technology to get to the science? A systematic review of the concept of computer literacy in preventive health programs. Health Edu J 2010 Mar 15;68(4):296-313. [CrossRef]
    28. Hur I, Lee R, Schmidt J. How healthcare technology shapes health literacy? A systematic review. Healthcare Information Systems and Technology (Sighealth). 2015.   URL: https://aisel.aisnet.org/amcis2015/HealthIS/GeneralPresentations/3/ [accessed 2021-05-17]
    29. Hæsum LK, Ehlers L, Hejlesen O. Influence of health literacy on outcomes using telehomecare technology: a systematic review. Health Edu J 2015 Jan 09;75(1):72-83. [CrossRef]
    30. Azzopardi-Muscat N, Sørensen K. Towards an equitable digital public health era: promoting equity through a health literacy perspective. Eur J Public Health 2019 Oct 01;29(Supplement_3):13-17 [FREE Full text] [CrossRef] [Medline]
    31. Paasche-Orlow MK, Wolf MS. The causal pathways linking health literacy to health outcomes. Am J Health Behav 2007;31 Suppl 1:19-26. [CrossRef] [Medline]
    32. Neter E, Brainin E. Association between health literacy, ehealth literacy, and health outcomes among patients with long-term conditions. Eur Psychol 2019 Jan;24(1):68-81. [CrossRef]
    33. Han H, Hong H, Starbird LE, Ge S, Ford AD, Renda S, et al. Ehealth literacy in people living with HIV: systematic review. JMIR Public Health Surveill 2018 Sep 10;4(3):e64 [FREE Full text] [CrossRef] [Medline]
    34. Chesser A, Burke A, Reyes J, Rohrberg T. Navigating the digital divide: a systematic review of eHealth literacy in underserved populations in the United States. Inform Health Soc Care 2016;41(1):1-19. [CrossRef] [Medline]
    35. Watkins I, Xie B. eHealth literacy interventions for older adults: a systematic review of the literature. J Med Internet Res 2014 Nov 10;16(11):e225 [FREE Full text] [CrossRef] [Medline]
    36. Stellefson M, Hanik B, Chaney B, Chaney D, Tennant B, Chavarria EA. eHealth literacy among college students: a systematic review with implications for eHealth education. J Med Internet Res 2011 Dec 01;13(4):e102 [FREE Full text] [CrossRef] [Medline]
    37. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005 Feb;8(1):19-32. [CrossRef]
    38. Bardus M, Smith JR, Samaha L, Abraham C. Mobile and Web 2.0 interventions for weight management: an overview of review evidence and its methodological quality. Eur J Public Health 2016 Aug;26(4):602-610 [FREE Full text] [CrossRef] [Medline]
    39. Bardus M, Rassi RE, Chahrour M, Akl EW, Raslan AS, Meho LI, et al. The use of social media to increase the impact of health research: systematic review. J Med Internet Res 2020 Jul 06;22(7):e15607 [FREE Full text] [CrossRef] [Medline]
    40. Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg 2011 Jul;128(1):305-310 [FREE Full text] [CrossRef] [Medline]
    41. Aromataris E, Munn Z, editors. JBI manual for evidence synthesis. In: Joanna Briggs Institute Reviewer's Manual. Australia: JBI - The University of Adelaide; 2020.
    42. Munn Z, Peters MD, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018 Nov 19;18(1):143 [FREE Full text] [CrossRef] [Medline]
    43. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
    44. Constantinides E, Fountain SJ. Web 2.0: Conceptual foundations and marketing issues. J Direct Data Digit Mark Pract 2008 Jan 4;9(3):231-244. [CrossRef]
    45. Bardus M. The web 2.0 and social media technologies for pervasive health communication: are they effec. Stud Commun Sci 2011;11(1):119-136. [CrossRef]
    46. Zobair K, Sanzogni L, Sandhu K, Islam M. Telemedicine adoption opportunities and challenges in the developing world. Eval Chall Opport Healthcare Ref 2020:193. [CrossRef]
    47. Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 2008 May;77(5):291-304. [CrossRef] [Medline]
    48. Brørs G, Pettersen TR, Hansen TB, Fridlund B, Hølvold LB, Lund H, et al. Modes of e-Health delivery in secondary prevention programmes for patients with coronary artery disease: a systematic review. BMC Health Serv Res 2019 Jun 10;19(1):364 [FREE Full text] [CrossRef] [Medline]
    49. Carter MC, Burley VJ, Cade JE. Handheld electronic technology for weight loss in overweight/obese adults. Curr Obes Rep 2014 Sep;3(3):307-315. [CrossRef] [Medline]
    50. Taylor-Rodgers E, Batterham PJ. Evaluation of an online psychoeducation intervention to promote mental health help seeking attitudes and intentions among young adults: randomised controlled trial. J Affect Disord 2014 Oct 15;168:65-71. [CrossRef] [Medline]
    51. Jorm AF, Kitchener BA, Fischer J, Cvetkovski S. Mental health first aid training by e-learning: a randomized controlled trial. Aust N Z J Psychiatry 2010 Dec;44(12):1072-1081. [CrossRef] [Medline]
    52. Tse CK, Bridges SM, Srinivasan DP, Cheng BS. Social media in adolescent health literacy education: a pilot study. JMIR Res Protoc 2015 Mar 09;4(1):e18 [FREE Full text] [CrossRef] [Medline]
    53. Feinkohl I, Flemming D, Cress U, Kimmerle J. The impact of personality factors and preceding user comments on the processing of research findings on deep brain stimulation: a randomized controlled experiment in a simulated online forum. J Med Internet Res 2016 Mar 03;18(3):e59 [FREE Full text] [CrossRef] [Medline]
    54. Hui A, Wong PW, Fu K. Evaluation of an online campaign for promoting help-seeking attitudes for depression using a facebook advertisement: an online randomized controlled experiment. JMIR Ment Health 2015;2(1):e5 [FREE Full text] [CrossRef] [Medline]
    55. Schougaard LM, Mejdahl CT, Petersen KH, Jessen A, de Thurah A, Sidenius P, et al. Effect of patient-initiated versus fixed-interval telePRO-based outpatient follow-up: study protocol for a pragmatic randomised controlled study. BMC Health Serv Res 2017 Jan 26;17(1):83 [FREE Full text] [CrossRef] [Medline]
    56. Navaneethan SD, Jolly SE, Schold JD, Arrigain S, Nakhoul G, Konig V, et al. Pragmatic randomized, controlled trial of patient navigators and enhanced personal health records in CKD. Clin J Am Soc Nephrol 2017 Sep 07;12(9):1418-1427 [FREE Full text] [CrossRef] [Medline]
    57. Redfern J, Usherwood T, Harris MF, Rodgers A, Hayman N, Panaretto K, et al. A randomised controlled trial of a consumer-focused e-health strategy for cardiovascular risk management in primary care: the Consumer Navigation of Electronic Cardiovascular Tools (CONNECT) study protocol. BMJ Open 2014 Jan 31;4(2):e004523 [FREE Full text] [CrossRef] [Medline]
    58. Khaleghi M, Shokravi FA, Peyman N, Moridi M. Evaluating the effect of educational interventions on health literacy through social networking services to promote students' quality of life. Korean J Fam Med 2019 May;40(3):188-193 [FREE Full text] [CrossRef] [Medline]
    59. Parisod H, Pakarinen A, Axelin A, Löyttyniemi E, Smed J, Salanterä S. Feasibility of mobile health game "Fume" in supporting tobacco-related health literacy among early adolescents: a three-armed cluster randomized design. Int J Med Inform 2018 May;113:26-37. [CrossRef] [Medline]
    60. Sanchez R, Brown E, Kocher K, DeRosier M. Improving children's mental health with a digital social skills development game: a randomized controlled efficacy trial of adventures aboard the S.S. GRIN. Games Health J 2017 Feb;6(1):19-27. [CrossRef] [Medline]
    61. Cyan R, Hussain H, Shehki I, Ishaq Z, Khan T, Rothenberg R. Affordable technology for saving maternal and infant lives: moving on with solutions. Ann Glob Health 2016;82(3):382. [CrossRef]
    62. Kamal AK, Muqeet A, Farhat K, Khalid W, Jamil A, Gowani A, et al. Using a tailored health information technology- driven intervention to improve health literacy and medication adherence in a Pakistani population with vascular disease (Talking Rx) - study protocol for a randomized controlled trial. Trials 2016 Mar 05;17(1):121 [FREE Full text] [CrossRef] [Medline]
    63. Rezvani F, Härter M, Dirmaier J. Promoting a home-based walking exercise using telephone-based health coaching and activity monitoring for patients with intermittent claudication (TeGeCoach): protocol for a randomized controlled trial. Psychother Psych Med 2018 Aug 06;68(08):e43. [CrossRef]
    64. Dinesen B, Dittmann L, Gade JD, Jørgensen CK, Hollingdal M, Leth S, et al. "Future Patient" telerehabilitation for patients with heart failure: protocol for a randomized controlled trial. JMIR Res Protoc 2019 Sep 19;8(9):e14517 [FREE Full text] [CrossRef] [Medline]
    65. Mudiyanselage SB, Stevens J, Watts JJ, Toscano J, Kotowicz MA, Steinfort CL, et al. Personalised telehealth intervention for chronic disease management: a pilot randomised controlled trial. J Telemed Telecare 2019 Jul;25(6):343-352. [CrossRef] [Medline]
    66. Sexual Health Empowerment for Women's Health. Clinical Trials. 2019.   URL: https://clinicaltrials.gov/ct2/show/NCT03984695 [accessed 2021-05-17]
    67. Duncan M, Vandelanotte C, Kolt GS, Rosenkranz RR, Caperchione CM, George ES, et al. Effectiveness of a web- and mobile phone-based intervention to promote physical activity and healthy eating in middle-aged males: randomized controlled trial of the ManUp study. J Med Internet Res 2014 Jun 12;16(6):e136 [FREE Full text] [CrossRef] [Medline]
    68. Iyawa G, Herselman M, Botha A. A scoping review of digital health innovation ecosystems in developed and developing countries. In: Proceedings of the IST-Africa Week Conference (IST-Africa). 2017 Presented at: IST-Africa Week Conference (IST-Africa); May 30- June 2, 2017; Windhoek, Namibia. [CrossRef]
    69. Peng C, He M, Cutrona SL, Kiefe CI, Liu F, Wang Z. Theme trends and knowledge structure on mobile health apps: bibliometric analysis. JMIR Mhealth Uhealth 2020 Jul 27;8(7):e18212 [FREE Full text] [CrossRef] [Medline]
    70. Singh D. Publication bias - a reason for the decreased research output in developing countries. Afr J Psych 2006 Oct 02;9(3):153-155. [CrossRef]
    71. Mulimani P. Publication bias towards Western populations harms humanity. Nat Hum Behav 2019 Oct 10;3(10):1026-1027. [CrossRef] [Medline]
    72. Yousefi-Nooraie R, Shakiba B, Mortaz-Hejri S. Country development and manuscript selection bias: a review of published studies. BMC Med Res Methodol 2006 Aug 01;6:37 [FREE Full text] [CrossRef] [Medline]
    73. Adelakun O, Garcia R. Technical factors in telemedicine adoption in extreme resource-poor countries. In: Global Health and Volunteering Beyond Borders. Switzerland: Springer; 2019:83-101.
    74. Hyland K. Academic publishing and the myth of linguistic injustice. J Second Lang Writ 2016 Mar;31:58-69. [CrossRef]
    75. Hyland K. Participation in publishing: the demoralizing discourse of disadvantage. In: Habibie P, Hyland K, editors. Novice Writers and Scholarly Publication. Switzerland: Palgrave Macmillan; 2018:13-33.
    76. Ashley S, Maksl A, Craft S. Developing a news media literacy scale. Journal Mass Commun Educ 2013 Jan 02;68(1):7-21. [CrossRef]
    77. Eristi B, Erdem C. Development of a media literacy skills scale. Contemp Educ Technol 2017;8(3):A. [CrossRef]
    78. Bybee R, McCrae B, Laurie R. PISA 2006: an assessment of scientific literacy. J Res Sci Teach 2009 Oct;46(8):865-883. [CrossRef]
    79. Harlen W. The assessment of scientific literacy in the OECD/PISA project. Stud Sci Educ 2001 Jan;36(1):79-103. [CrossRef]
    80. Kurbanoglu SS, Akkoyunlu B, Umay A. Developing the information literacy self‐efficacy scale. J Doc 2006 Nov;62(6):730-743. [CrossRef]
    81. Çoklar AN, Yaman ND, Yurdakul IK. Information literacy and digital nativity as determinants of online information search strategies. Comp Hum Behav 2017 May;70:1-9. [CrossRef]
    82. Mackert M, Kahlor L, Tyler D, Gustafson J. Designing e-health interventions for low-health-literate culturally diverse parents: addressing the obesity epidemic. Telemed J E Health 2009 Sep;15(7):672-677 [FREE Full text] [CrossRef] [Medline]
    83. Mackert M, Champlin SE, Holton A, Muñoz II, Damásio MJ. eHealth and health literacy: a research methodology review. J Comput-Mediat Comm 2014 Apr 12;19(3):516-528. [CrossRef]
    84. Pelikan JM, Ganahl K, Roethlin F. Health literacy as a determinant, mediator and/or moderator of health: empirical models using the European Health Literacy Survey dataset. Glob Health Promot 2018 Nov 14:57-66. [CrossRef] [Medline]
    85. Bardus M, Hassan FA. eHealth for obesity prevention among low-income populations: is research is promoting health for all? Marco Bardus. Eur J Public Health 2016;26(1):77. [CrossRef]
    86. Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med 2007 May;32(5 Suppl):119-126. [CrossRef] [Medline]
    87. Atienza AA, Hesse BW, Baker TB, Abrams DB, Rimer BK, Croyle RT, et al. Critical issues in eHealth research. Am J Prev Med 2007 May;32(5 Suppl):71-74 [FREE Full text] [CrossRef] [Medline]
    88. Haun J, Luther S, Dodd V, Donaldson P. Measurement variation across health literacy assessments: implications for assessment selection in research and practice. J Health Commun 2012;17 Suppl 3:141-159. [CrossRef] [Medline]
    89. Duell P, Wright D, Renzaho AM, Bhattacharya D. Optimal health literacy measurement for the clinical setting: a systematic review. Patient Educ Couns 2015 Nov;98(11):1295-1307. [CrossRef] [Medline]
    90. Health Literacy Tool Shed.   URL: https://healthliteracy.bu.edu/ [accessed 2021-05-17]
    91. Jones-Kavalier BR, Flannigan SL. Connecting the digital dots : literacy of the 21st century. Educause Q. 2006.   URL: https:/​/er.​educause.edu/​-/​media/​files/​articles/​2006/​4/​eqm0621.​pdf?la=en&hash=072FD518E173153E68975F73DA646847D9736291 [accessed 2021-05-17]
    92. Ptaszek G. Media literacy outcomes, measurement. In: The International Encyclopedia of Media Literacy. Hoboken, New Jersey, United States: Wiley; 2019.
    93. Joseph-Shehu EM, Ncama BP, Mooi N, Mashamba-Thompson TP. The use of information and communication technologies to promote healthy lifestyle behaviour: a systematic scoping review. BMJ Open 2019 Oct 28;9(10):e029872 [FREE Full text] [CrossRef] [Medline]
    94. Taj F, Klein MC, van Halteren A. Digital health behavior change technology: bibliometric and scoping review of two decades of research. JMIR Mhealth Uhealth 2019 Dec 13;7(12):e13311 [FREE Full text] [CrossRef] [Medline]

    ‎
    DHI: digital health intervention
    eHEALS: eHealth literacy scale
    EHR: electronic health record
    LMIC: low- and middle-income country
    mHealth: mobile health
    NCD: noncommunicable disease
    PCC: Population-Concept-Context
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    RCT: randomized controlled trial

    Edited by G Eysenbach; submitted 13.08.20; peer-reviewed by J Dubin, Z Wang, O El-Gayar; comments to author 30.09.20; revised version received 20.10.20; accepted 14.04.21; published 03.06.21

    Copyright

    ©Mariam El Benny, Tamar Kabakian-Khasholian, Fadi El-Jardali, Marco Bardus. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 03.06.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.