This paper is in the following e-collection/theme issue:

Engagement with and Adherence to Digital Health Interventions, Law of Attrition (491) Human Factors and Usability Case Studies (787)

Published on in Vol 22, No 2 (2020): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/11613, first published .
Adherence to Electronic Health Tools Among Vulnerable Groups: Systematic Literature Review and Meta-Analysis

Adherence to Electronic Health Tools Among Vulnerable Groups: Systematic Literature Review and Meta-Analysis

Adherence to Electronic Health Tools Among Vulnerable Groups: Systematic Literature Review and Meta-Analysis

Authors of this article:

Jelena Arsenijevic1 Author Orcid Image ;   Lars Tummers1 Author Orcid Image ;   Niels Bosma2 Author Orcid Image
Article Authors Cited by (46) Tweetations (38) Metrics

    Review

    1Utrecht University School of Governance, Faculty of Law Economics and Governance, Utrecht, Netherlands

    2Utrecht University School of Economics, Faculty of Law Economics and Governance, Utrecht, Netherlands

    *these authors contributed equally

    Corresponding Author:

    Jelena Arsenijevic, PhD

    Utrecht University School of Governance

    Faculty of Law Economics and Governance

    Bijlhouwerstraat 6

    Utrecht

    Netherlands

    Phone: 31 643005169

    Email: j.arsenijevic@uu.nl


    Background: Electronic health (eHealth) tools are increasingly being applied in health care. They are expected to improve access to health care, quality of health care, and health outcomes. Although the advantages of using these tools in health care are well described, it is unknown to what extent eHealth tools are effective when used by vulnerable population groups, such as the elderly, people with low socioeconomic status, single parents, minorities, or immigrants.

    Objective: This study aimed to examine whether the design and implementation characteristics of eHealth tools contribute to better use of these tools among vulnerable groups.

    Methods: In this systematic review, we assessed the design and implementation characteristics of eHealth tools that are used by vulnerable groups. In the meta-analysis, we used the adherence rate as an effect size measure. The adherence rate is defined as the number of people who are repetitive users (ie, use the eHealth tool more than once). We also performed a meta-regression analysis to examine how different design and implementation characteristics influenced the adherence rate.

    Results: Currently, eHealth tools are continuously used by vulnerable groups but to a small extent. eHealth tools that use multimodal content (such as videos) and have the possibility for direct communication with providers show improved adherence among vulnerable groups.

    Conclusions: eHealth tools that use multimodal content and provide the possibility for direct communication with providers have a higher adherence among vulnerable groups. However, most of the eHealth tools are not embedded within the health care system. They are usually focused on specific problems, such as diabetes or obesity. Hence, they do not provide comprehensive services for patients. This limits the use of eHealth tools as a replacement for existing health care services.

    J Med Internet Res 2020;22(2):e11613

    doi:10.2196/11613

    Keywords

    eHealth; digital health; disparities in health care; meta-analysis 


    Background

    Amra is a fictional 56-year-old Turkish migrant who has lived in Germany for more than 20 years. Although she recognized the first signs of menopause, she felt ashamed to visit her male general practitioner (GP) and talk about this. In addition, her German is not good. She discovered through her network of Turkish women that there is an app called Intelligent Health Assistant that can be downloaded on her mobile phone. This app can help her find information about menopause in both Turkish and German. Furthermore, the app allows her to make an appointment with a female doctor [1]. The above example shows how innovative communication technologies such as electronic health (eHealth) tools can be used to provide better information of, and access to, health care services for vulnerable groups such as migrants [1].

    eHealth tools are increasingly being applied in health care [2]. They are known by different names, such as eHealth, informational communication technologies in health, consumer health information technologies, mobile health, Web-based health platforms, or telemedicine [3]. Usually, they are computer- and Web-based tools that are intended to improve quality of health care, health outcomes, access to health care services, and patients’ quality of life [4]. Examples of eHealth tools include patient portals, Web-based platforms that offer health care tools, or mobile phone apps. eHealth tools can use different technologies such as Web platforms developed for that purpose or social media platforms such as Facebook. Some of them are specifically developed for smartphones, whereas others can be used on any digital device [5]. Different services can be provided by eHealth tools—for example, making appointments, checking the results of laboratory tests, or participation in Web-based prevention programs. The first eHealth tools were developed in the United States. Today, many governments in Europe also advocate the use of eHealth tools within health care systems [6]. Different stakeholders are involved in their development. Some eHealth tools are developed in cooperation with health care providers. Others (also known as consumer eHealth) are developed by for-profit and nonprofit parties—small entrepreneurs or big companies—and are available on the open market [7].

    It is asserted that eHealth tools have advantages compared with traditional delivery of health care services [8]. One of the potential advantages of using eHealth tools is that they can facilitate better patient-provider interactions. Of particular importance is the direct patient-provider interaction through eHealth tools that eliminates the need for physical appointments. It is assumed that such interactions can enhance the active participation of patients and lead to a more patient-centered care [9]. Furthermore, these tools mostly use encrypted Web platforms or apps that can capture personal data. This secures privacy for patients. In addition, with eHealth tools, users do not need to make an appointment to communicate with health care providers. Thus, users have quicker access to health care providers [10].

    Although the advantages of using these tools in health care are well described, it is unknown to what extent these tools are effective when used by vulnerable groups. Vulnerable population groups are defined as social groups that have an increased risk for adverse health outcomes [11,12]. Vulnerable population groups include people with low socioeconomic status, older adults, single parents, minorities, or immigrants [5,13,14]. These groups tend to have lower health outcomes and experience more difficulties in accessing health care services compared with the general population [15]. Most of these difficulties are related to social injustice and can be improved by efficient health policies or by adopting innovative health tools such as eHealth tools [12]. Previous studies have shown that the use of eHealth tools among vulnerable groups can have double-folded effects [16]. In some cases, eHealth tools improve access to health care. In our fictional example of Amra, it helped her and catered to her current needs. In her case, the mobile phone app provided improved access to adequate information and health care services. However, innovative tools do not always have positive effects among vulnerable groups. In some cases, these tools can increase the disparities that exist between vulnerable groups and the general population [17]. For example, older adults who are not familiar with internet technology may not be able to make appointments via an electronic patient portal [18]. eHealth tools may then reduce access to health care for these groups.

    However, information about the effectiveness of eHealth tools among vulnerable groups is still inconsistent. Previous studies have shown that effectiveness of eHealth tools among vulnerable groups is influenced by the level of adherence [19,20]. The term adherence was initially used for medication, but it is also used in other health areas [21]. Adherence is defined by the World Health Organization (WHO) as the “extent to which a person’s behavior—taking medication, following a diet, and/or executing lifestyle changes-corresponds with agreed recommendations from a health care provider” [22]. In the case of eHealth tools, there are many challenges in applying this WHO definition [21-24]. In 2005, Eysenbach was the first to notice that although in the case of medication adherence we often know what optimal dosage is, this is not always the case for eHealth tools [21,25]. Some authors have proposed the concept of intended use or use as it is designed [22,23]. However, this provides no justification for the level of intended use. Others argue that the use of all components of eHealth tools by all population groups might not be necessary. Some groups might achieve their personal goals by using only a few components [24]. Furthermore, different eHealth tools might require different intended uses to be effective in changing health outcomes [24,26]. For example, to change their lifestyle, users might be engaged with eHealth tools once per day for extended periods, whereas to maintain good self-management of chronic diseases, users need to be engaged several times per day [27,28]. This means that adherence can be influenced by users’ characteristics as well as the characteristics of the goal of the eHealth tool. On this basis, different metrics of adherence are proposed—some authors propose measures such as the number of log-ins or the number of characters that are typed every time a person is logged in or the number of Web pages accessed [22]. Others propose the use of different measures such as the attrition rate or the dropout rate.

    In previous studies, adherence to eHealth tools was compared among different population groups, including vulnerable groups [18,29,30]. Some of the studies report this percentage at the end of the intervention period, without reporting dropout rates across population groups. Not surprisingly, most of these studies concluded that the percentage of users from a vulnerable population is lower than that among the general population [31,32]. However, this does not imply that vulnerable groups did not achieve the intended use.

    In this study, we used the method proposed by Sieverink et al [23] as operationalization category C level—“Assigned when the intended use of the technology was provided and justified using theory, evidence, or rationale.” We examined the number of repeated users for the eHealth tool after a period of time that is justified to be relevant for this eHealth tool.

    Another drawback of the previous studies that assessed failure in use of eHealth tools among vulnerable groups was that the focus was typically limited to generic characteristics such as low health literacy, low education levels, and lack of access to fast internet [3,33,34]. However, the design and implementation characteristics of eHealth tools can also play a role in their effective use among vulnerable groups [8]. Previous findings have shown that design characteristics such as the type of technology used (mobile app or Web-based platform), use of multimodal content (use of videos, games, or quizzes), or the possibility of direct interaction between patients and providers can increase the use of eHealth tools among vulnerable groups [3]. Different vulnerable groups have different preferences regarding the type of technology used. Some vulnerable groups such as migrants or low-income single mothers prefer the use of mobile phones, whereas others such as chronically sick or older patients seem to prefer Web-based platforms [3,35,36]. Multimodal content facilitates the use of eHealth tools for vulnerable groups that have problems with health literacy (ability to understand, proceed, and make decisions with health information). Videos or games are less language saturated and can be understood and used by people with low health literacy [37]. To overcome the problem of a digital divide (lack of knowledge on how to use the internet) [38] and/or health literacy [39], eHealth tools sometimes use direct interaction between patients and providers. Direct interaction makes personalized information available to the patients, which consequently leads to a better understanding in patients [40]. On the basis of previous literature, we have also identified implementation characteristics that can lead to improved use of eHealth tools among vulnerable groups. One of these characteristics is the possibility to let eHealth tools be used by vulnerable groups exclusively or to introduce eHealth tools that are developed for the general population but can be easily adopted by vulnerable groups. The possibility of training related to the use of eHealth tools is also important for vulnerable groups. Reluctance to use eHealth tools may stem from feelings of incompetence in vulnerable groups. Training can help them overcome this problem [13].

    Objectives

    On the basis of the above-mentioned information, this study had two goals. First, we aimed to identify the level of adherence toward eHealth tools among vulnerable groups. To this end, we conducted a systematic literature review and meta-analysis. Second, we aimed to establish how different design and implementation characteristics influence the level of adherence. To this end, we conducted a meta-regression. Identifying potentially successful designs for eHealth tools can help include these tools as a regular part of health care service delivery. Furthermore, if eHealth tools are adopted by vulnerable groups, they could improve access to health care services and even replace some of the existing services [41].


    Reporting Standards

    A systematic literature review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) strategy [42]. In addition, we used PRISMA recommendations for a replicable meta-analysis (see Multimedia Appendix 1).

    Inclusion Criteria

    We included studies that (1) examined the use of Web-based innovative technologies among at least one vulnerable group (older adults, chronically sick, minorities, people with low socioeconomic status, and migrants), (2) were published in peer-reviewed journals in English after 2007, (3) focused on people aged 18 years or older, and (4) reported the level of patient/user participation (adherence). Regarding the design, we included studies with the following designs: randomized controlled trial (RCT and similar designs such as pragmatic RCT), prospective longitudinal studies, pre- and postdesign studies, and cohort studies.

    Exclusion Criteria

    We excluded studies with a qualitative research design, case studies, opinion papers, literature reviews or theoretical views, studies that assessed the use of Web-based technologies to address the education or decision-making process among medical providers, studies that examined new medical devices and their technical characteristics based on Web-based apps, studies that assessed psychometric instruments that are used to evaluate Web-based apps, and studies that evaluated Web-based population surveys.

    Search Strategy, Study Selection, and Data Extraction

    First, we conducted an electronic search in the following databases: PubMed, Web of Knowledge, EBSCO, and CINAHL. All databases were searched from January 5, 2017, to January 5, 2018. To develop the search strategies, we checked two main sets of keywords: (1) eHealth tools and corresponding synonyms (Web-based information technologies, social media, internet based, electronic-records, Facebook, etc) and (2) health disparities and corresponding synonyms (disparity in health, vulnerable groups, or inequity). For both sets of keywords, we also checked the thesaurus and Medical Subject Headings terms. Second, we developed a search strategy for each database. The detailed strategy for PubMed is presented in Textbox 1. After the initial selection of studies, we checked their reference lists for additional literature. A publication from the reference list (bibliography) was included in the review after applying the same inclusion and exclusion criteria. Third, we conducted a forward search by looking up the studies that cited the included studies. For this purpose, we used PubMed. Fourth, we used literature review studies to check whether we included studies that have been identified in previous literature reviews. We used a PRISMA flowchart to present the search strategy.

    Search string used for PubMed.

    String used for PubMed: ((((((((((e-Health[Title/Abstract] OR eHealth[Title/Abstract]) OR ((“health”[MeSH Terms] OR “health”[All Fields]) AND (“Information (Basel)”[Journal] OR “information”[All Fields]) AND technologies[Title/Abstract])) OR patient portals[Title/Abstract]) OR telemedicine[Title/Abstract]) OR “social media”[MeSH Terms]) OR Facebook[Title/Abstract]) OR Twitter[Title/Abstract]) OR Web 2.0[Title/Abstract]) OR “internet”[MeSH Terms]) AND ((“health”[MeSH Terms] OR “health”[All Fields]) AND disparities[All Fields])) OR vulnerable [All Fields] OR disadvantaged [All Fields] and migrants [MESH]OR immigrants [MESH] OR low income [Title/Abstract] OR older adults [Title/Abstract]))

    Textbox 1. Search string used for PubMed.

    Study Selection and Characteristics of the Selected Studies

    On the basis of the search strategy, we identified 473 publications. We presented the selection process through a PRISMA flowchart (Figure 1). After applying filters for English language and duration from 2007 to 2017, we were left with 429 publications. In the next step, we checked the titles and abstracts, resulting in 318 excluded studies (mostly studies addressing telemedicine, using providers as participants, or using data on an organizational level). Thereafter, we screened the remaining 111 publications. Among them were 13 literature reviews [4,34,36,43-51] and 21 opinion papers [6-9,13-15,52-65]. These were all excluded. We also excluded 28 studies that examined only sociodemographic characteristics of eHealth users and 12 studies that were design papers. In addition, we excluded eight studies because they were qualitative studies that used focus group methods to gather data. In total, we included 27 studies based on our inclusion and exclusion criteria [3,33,35,66-89].

    For conducting search in the other three databases, we used combinations of all two keywords. The articles that were found within the other databases and met our inclusion and exclusion criteria were the same as those already identified with PubMed.

    The summarized description of all selected articles is presented in Table 1. The detailed description of all included articles is presented in Multimedia Appendix 2.

    ‎
    Figure 1. Searching strategy for PubMed I.
    View this figure
    Table 1. Summary of the study characteristics (N=27).
    Study characteristicsValueStudy
    Year of publication, n (%)

    		2001 (4)Kim et al [66]

    		20102 (7)Sarkar et al [67], Kerr et al [68]

    		20112 (7)Ancker et al [33], Goel et al [69]

    		20135 (19)Ronda et al [35], Nazi et al [81], Osborn et al [70], Joseph et al [72], Ryan et al [88]

    		20142 (7)Steinberg et al [86], Herring et al [75]

    		20156 (22)Campbell et al [74], Foster et al [3], Billings et al [76], Smith et al [77], Levy et al [78], Jhamb et al [79]

    		20165 (19)Joseph et al [73], Gordon and Hornbrook [80], Aalbers et al [84], Cavallo et al [85], Bickmore et al [87]

    		20174 (15)Cullen et al [71], Ernsting et al [82], Arcury et al [83], Buis et al [89]
    Country, n (%)

    		United States23 (85)Kim et al [66], Sarkar et al [67], Ancker et al [33], Goel et al [69], Osborn et al [70], Cullen et al [71], Joseph et al [72], Joseph et al [73], Campbell et al [74], Herring et al [75], Billings et al [76], Smith et al [77], Levy et al [78], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81], Foster et al [3], Arcury et al [83], Cavallo et al [85], Steinberg et al [86], Bickmore et al [87], Ryan et al [88], Buis et al [89]

    		Other4 (15)Kerr et al [68], Ronda et al [35], Ernsting et al [82], Aalbers et al [84]
    Design of the study, n (%)

    		Cohort study7 (26)Kim et al [66], Sarkar et al [67], Kerr et al [68], Smith et al [77], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81]

    		Randomized controlled trial12 (44)Ronda et al [35], Osborn et al [70], Cullen et al [71], Joseph et al [73], Herring et al [75], Billings et al [76], Levy et al [78], Ernsting et al [82], Steinberg et al [86], Bickmore et al [87], Ryan et al [88], Buis et al [89]

    		One group pre- to postdesign5 (19)Joseph et al [72], Campbell et al [74], Foster et al [3], Aalbers et al [84], Cavallo et al [85]

    		Longitudinal studies3 (11)Ancker et al [33], Goel et al [69], Arcury et al [83]
    Sample size, n (%)

    		N>10017 (62)Kim et al [66], Sarkar et al [67], Kerr et al [68], Ancker et al [33], Goel et al [69], Ronda et al [35], Cullen et al [71], Smith et al [77], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81], Ernsting et al [82], Arcury et al [83], Aalbers et al [84], Cavallo et al [85], Steinberg et al [86], Buis et al [89]

    		N<10010 (37)Osborn et al [70], Joseph et al [72], Joseph et al [73], Campbell et al [74], Herring et al [75], Billings et al [76], Levy et al [78], Foster et al [3], Bickmore et al [87], Ryan et al [88]
    Area of health care where electronic health tool is applied, n (%)

    		Primary care5 (18.5)Ancker et al [33], Nazi et al [81], Ernsting et al [82], Arcury et al [83], Bickmore et al [87]

    		Diabetes4 (14.8)Sarkar et al [67], Ronda et al [35], Levy et al [78], Ryan et al [88]

    		Cardiovascular diseases2 (7.4)Kerr et al [68], Buis et al [89]

    		Obesity8 (29.6)Kim et al [66], Cullen et al [71], Joseph et al [72], Joseph et al [73], Herring et al [75], Aalbers et al [84], Cavallo et al [85], Steinberg et al [86]

    		Other chronic diseases4 (14.8)Osborn et al [70], Campbell et al [74], Smith et al [77], Jhamb et al [79]

    		Reproductive health2 (7.4)Goel et al [69], Gordon and Hornbrook [80]

    		Nursing home2 (7.4)Billings et al [76], Foster et al [3]
    Target population, n (%)

    		Minorities12 (44.4)Kim et al [66], Cullen et al [71], Joseph et al [72], Joseph et al [73], Campbell et al [74], Billings et al [76], Foster et al [3], Arcury et al [83], Steinberg et al [86], Bickmore et al [87], Ryan et al [88], Buis et al [89]

    		Low-income people5 (18.5)Ancker et al [33], Herring et al [75], Levy et al [78], Ernsting et al [82], Cavallo et al [85]

    		Older adults4 (14.8)Goel et al [69], Smith et al [77], Gordon and Hornbrook [80], Aalbers et al [84]

    		Chronically sick6 (22.5)Sarkar et al [67], Kerr et al [68], Ronda et al [35], Osborn et al [70], Jhamb et al [79], Nazi et al [81]
    Quality score of the studies, mean (SD)21.07 (2.90); minimum: 17.00, maximum: 31.00All

    Quality Assessment

    To assess the quality of the included studies, we used the quality assessment proposed by Zingg et al [90]. This tool is known as Integrated quality Criteria for the Review of Multiple Study designs (ICROMS). ICROMS allows us to calculate the quality scores for articles with different study designs such as RCTs, cohort studies, or controlled before-and-after studies. It consists of a clear and transparent scoring system accompanied by a decision matrix for each of the indicators that is related to the quality of the article. Each indicator gets a score of 2 if the criteria for the indicator are met, 0 if this is not the case, and 1 if it is unknown whether the criteria were met. In total, 33 indicators are grouped in seven dimensions, namely, clear aims and justification, managing bias in sampling or between groups, managing bias in outcome measurements and blinding, managing bias in follow-up, managing bias in other study aspects, analytical rigor, and managing bias in reporting/ethical considerations. The score depends on the design of the study.

    Data Extraction and Outcome Measures

    In accordance with the PRISMA guidelines, we extracted the following characteristics for each study: year of publication, country of origin, study design, target population, area of health care where eHealth tool is applied, and quality of the study. To calculate the adherence level, we also extracted the total sample size (N), the sample size for those who used eHealth more than once (n2), and the sample size for those who are registered but did not use eHealth tools more than once—uptake (n1). We also calculated the probability of continuous users (intended adherence; P2=n2/N) and probability of one-time users (P1=n1/N). These data are presented in Multimedia Appendix 2.

    We also extracted the following design characteristics: the possibility to have direct contact with a medical provider, use of multimodal content (videos, games, and quizzes), and the type of technology used (patient portal, Web-based portal, or mobile app). Next, we extracted the following implementation characteristics: target group addressed by eHealth tools, whether the eHealth tool is exclusive for the target group or can be used among the whole population (inclusive), and the possibility of training. These data are presented in Multimedia Appendix 2. All data were extracted by 1 researcher.

    Data Synthesis and Analysis

    To assess the adherence among vulnerable population groups, we conducted a random-effects meta-analysis. We calculated the ratio between the probability of nonusers (people who did not use eHealth tools or those who used eHealth tools once, usually during registration) and the probability of continuous users (intended adherence). People who used eHealth tools only once, when they were registered, are similar to nonusers. They might be registered by their health care providers or family members, but they had never activated and used their account. We calculated the probability of nonusers as P1=n1/N, where N is the total sample, and n1 is the number of people who used eHealth tools only once or did not use it at all.

    If the study reported the number of nonusers, we compared the repetitive users with nonusers. Next, we calculated the probability of continuous users as P2=n2/N, where n2 is the number of users who used eHealth tools as it was designed and in a way that was justified to be relevant for this eHealth tool. Thereafter, we calculated the estimate of the effect size measure—risk ratio (RR=P2/P1) and made the logarithm transformation log(RR). Logarithm transformation was usually used when the included studies had a different research design [91]. For the visual representation of the results, we used a funnel plot (see Figure 2). Between studies, heterogeneity was assessed through the I2 statistic (with a value higher than 75% considered as large).

    We performed a meta-regression to assess the extent to which different design and implementation characteristics influence the adherence rate among vulnerable population groups. In the meta-regression, the dependent variable was the size of the effect estimates from the individual studies. As explanatory variables, we included design and implementation characteristics such as the type of technology used for the eHealth tool (patient portal, Web-based tool, or mobile app), the presence of multimodal content (yes/no), the availability of training for the use of eHealth tools, and direct interaction with a medical doctor (yes/no). The quality of the study was used as a covariate. The results of the meta-analysis (effect size measures for adherence) might be saturated with different sampling methods and different study designs. This can lead to heterogeneity in effect size measures. Meta-regression can also help explore the reasons for heterogeneity. In the meta-regression output, heterogeneity between the studies was measured through the I2 statistic (with a value higher than 75% considered as large). The proportion of between-study variance explained by the model was calculated through tau squared. For both meta-analysis and meta-regression, we extracted data from 27 studies.

    ‎
    Figure 2. Results from meta-analysis-effect size adherence rate.
    View this figure

    Publication Bias Tests

    It has been shown that studies that report statistically significant results or clinically relevant results are published more often [92]. This can lead to publication bias—that is, effect sizes of studies included in the meta-analysis differ from the general effect size when considering all studies [93]. To test for publication bias related to standardized adherence, we applied the Begg and Mazumdar rank correlation tests and the Egger test. The results from the publication bias test are presented in the Results section.


    Main Study Characteristics

    We included 27 studies. Table 1 presents the characteristics of the included studies. Most studies were from the United States (23/27, 85%) and were published in the period 2013 to 2017 (22/27, 82%). In addition, most studies had an RCT design (12/27, 44%). However, it is also worth mentioning that most RCTs were derived from larger cohort studies. This means that randomization has been conducted between registered and repeated users. Furthermore, most studies were related to primary care or health promotion (eg, addressing the problem of obesity). The studies related to primary care were mostly associated with patient portals, such as kp.org portals from different states in the United States, the MyChart portal from the United Kingdom, or My Health at Vanderbilt (also in the United States), that aimed to provide better access to primary care for chronically sick users. eHealth tools that address the problem of obesity were usually Web portals. They presented extensions of already existing health promotion interventions: these interventions were not delivered in community centers; these were delivered through Web-based portals. This was, for instance, the case with the Muévete Alabama study that aimed to decrease obesity among Latinas in the United States [94]. Our results showed that most studies targeted minorities (12/27, 44%). In addition, more than half of the studies used a sample size of more than 100. The mean value of quality score was 21. This can be described as a middle-quality score. Most studies had the lowest score on the dimension managing bias in sampling or between groups. Our results also showed that some design characteristics, such as type of technology, were related to some characteristics of the included studies.

    Design and Implementation Characteristics of Electronic Health Tools

    In Table 2, we summarize the design and implementation characteristics of the eHealth tools that are used by vulnerable population groups.

    The number of studies related to patient portals and Web-based platforms was quite high (22/27, 82%), whereas there were fewer studies that evaluated mobile apps (5/27, 19%). Our results also showed that almost all eHealth tools (23/27, 86%) provided the possibility for direct communication with the provider. Conversely, the number of eHealth tools that used multimodal content was small (10/27, 37%). The studies that used multimodal content were usually Web-based portals that provide videos or games. One example is a Dutch study that aimed to improve the lifestyle of older adults [78]. Among the implementation characteristics, the possibility of training for the use of the eHealth tool was rare—only 5 (5/27, 19%) studies reported it. The number of eHealth tools exclusively made for vulnerable groups was similar to the number of tools that can be applied to a general population (13/27, 48% vs 14/27, 52%).

    Our results also showed that some design characteristics, such as the type of technology, were related to the design. Most studies with an RCT design were related to the use of Web-based platforms, whereas those with a cohort design were related to the use of patient portals. Patient portals were related to primary care services or nursing homes, whereas Web-based platforms were mostly related to the problems of obesity. Table 3 presents these results.

    Table 2. Design and implementation characteristics (N=27).
    Design and implementation characteristicsValue, n (%)Study
    Design characteristics

    		Type of technology used


    		Web-based platforms12 (44)Kerr et al [68], Cullen et al [71], Joseph et al [72], Joseph et al [73], Campbell et al [74], Billings et al [76], Arcury et al [83], Aalbers et al [84], Cavallo et al [85], Steinberg et al [86], Bickmore et al [87], Ryan et al [88]


    		Patient portals10 (37)Kim et al [66], Sarkar et al [67], Ancker et al [33], Goel et al [69], Ronda et al [35], Osborn et al [70], Smith et al [77], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81]


    		Mobile app5 (19)Herring et al [75], Levy et al [78], Foster et al [3], Ernsting et al [82], Buis et al [89]

    		Use of multimodal content (yes=1; no=0)


    		Yes10 (37)Cullen et al [71], Joseph et al [72], Joseph et al [73], Campbell et al [74], Billings et al [76], Ernsting et al [82], Aalbers et al [84], Cavallo et al [85], Steinberg et al [86], Bickmore et al [87]


    		No17 (63)Kim et al [66], Sarkar et al [67], Kerr et al [68], Ancker et al [33], Goel et al [69], Ronda et al [35], Osborn et al [70], Herring et al [75], Smith et al [77], Levy et al [78], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81], Foster et al [3], Arcury et al 112], Ryan et al [88], Buis et al [89]

    		Possibility of direct interaction with provider (yes=1; no=0)


    		Yes23 (86)Kim et al [66], Sarkar et al [67], Kerr et al [68], Ancker et al [33], Goel et al [69], Ronda et al [35], Osborn et al [70], Cullen et al [71], Joseph et al [72], Joseph et al [73], Herring et al [75], Smith et al [77], Levy et al [78], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81], Foster et al [3], Ernsting et al [82], Arcury et al [83], Aalbers et al [84], Cavallo et al [85], Bickmore et al [87], Ryan et al [88], Buis et al [89]


    		No4 (15)Campbell et al [74], Billings et al [76], Steinberg et al [86], Ernsting et al [82]
    Implementation characteristics

    		Type of target group


    		Minorities12 (44)Kim et al [66], Cullen et al [71], Joseph et al [72], Joseph et al [73], Campbell et al [74], Billings et al [76], Foster et al [3], Arcury et al [83], Steinberg et al [86], Bickmore et al [87], Ryan et al [88], Buis et al [89]


    		Low-income people5 (19)Ancker et al [33], Herring et al [75], Levy et al [78], Ernsting et al [82], Cavallo et al [85]


    		Older adults4 (15)Goel et al [69], Smith et al [77], Gordon and Hornbrook [80], Aalbers et al [84]


    		Chronically sick6 (23)Sarkar et al [67], Kerr et al [68], Ronda et al [35], Osborn et al [70], Jhamb et al [79], Nazi et al [81]

    		Exclusive or inclusive for target group


    		Exclusive14 (52)Kim et al [66], Sarkar et al [67], Ronda et al [35], Cullen et al [71], Joseph et al [73], Herring et al [75], Levy et al [78], Nazi et al [81], Foster et al [3], Aalbers et al [84], Steinberg et al [86], Bickmore et al [87], Ryan et al [88], Buis et al [89]


    		Inclusive13 (48)Kerr et al [68], Ancker et al [33], Goel et al [69], Osborn et al [70], Joseph et al [72], Campbell et al [74], Billings et al [76], Smith et al [77], Jhamb et al [79], Gordon and Hornbrook [80], Ernsting et al [82], Arcury et al [83], Cavallo et al [85]

    		Possibility of training


    		Yes5 (19)Kim et al [66], Sarkar et al [67], Kerr et al [68], Joseph et al [72], Bickmore et al [87]


    		No22 (82)Ancker et al [33], Goel et al [69], Ronda et al [35], Osborn et al [70], Cullen et al [71], Joseph et al, 2016. [73], Campbell et al [74], Herring et al [75], Billings et al [76], Smith et al [77], Levy et al [78], Jhamb et al [79], Gordon and Hornbrook [80], Nazi et al [81], Foster et al [3], Ernsting et al [82], Arcury et al [83], Aalbers et al [84], Cavallo et al [85], Steinberg et al [86], Ryan et al [88], Buis et al [89]
    Table 3. Type of technology used and study characteristics (N=27).
    Type of technology usedStudy designs, n (%)Area of health care where electronic health tool is applied, n (%)Target population, n (%)
    Web-based platform (n=11)RCTa, 6 (22); others, 5 (19)Obesity, 6 (22); others, 5 (19)Minorities, 8 (30); others 3 (11)
    Patient portals (n=11)Cohort, 6 (22); others, 5 (19)General practice, 4 (15); others, 7 (26)Chronically sick, 5 (19); elderly, 3 (11); minorities, 2 (8); low-income, 1 (4)
    Mobile apps (n=5)RCT, 3 (11); others, 2 (7)—bLow-income people, 3 (11); others, 2 (7)

    aRCT: randomized controlled trial.

    bMissing data.

    Adherence to Electronic Health Tools Among Vulnerable Groups—Results From Meta-Analysis

    To examine the extent to which vulnerable population groups adopted eHealth tools, we conducted meta-analyses. Results from the meta-analysis on adherence effect size measures showed that the difference in proportion between intended adherers and only registered users was 7% (95% CI −0.23 to 0.38). They showed that users from vulnerable groups adopted eHealth tools for continuous use. However, the difference between registered and repetitive users was still small. In Figure 2, the middle value on the axis should be 0.5 instead of the standard—0. The reason was that we examined the difference in users who registered once but not in continuous users and repetitive users. This means that all users had a chance to potentially use the eHealth tool. I2 tests show high between-study heterogeneity.

    Design and Implementation Characteristics of Electronic Health Tools and Adherence

    To examine how different design and implementation characteristics influence the adherence rate, we applied meta-regression. The results from meta-regression (Table 4) showed that studies that evaluated eHealth tools with multimodal content and direct patient-provider interaction reported a higher adherence rate. This means that the use of multimodal content and the possibility of having direct contact with providers seem to increase the adoption of eHealth tools among vulnerable groups, although endogeneity is clearly a potential cause for concern.

    Table 4. Results from meta-regression with adherence as an effect size measure.
    Independent variablesBeta coefficientSEP value
    Patient portal technology (yes=1, no=0)1.370.73.07
    Mobile app technology (yes=1, no=0)1.750.75.13
    Exclusive tool (yes=1, no=1).510.44.25
    Multimodal content (yes=1, no=0)2.49a0.72.00
    Training for using eHealth tool (yes=1, no=0)−.510.56.38
    Interaction with health providers (yes=1, no=0)1.23a0.55.03
    Quality score of included study (minimum=0, maximum=31).490.78.53
    Constant−3.72b1.86.06
    Adjusted R238.80—c—
    Τ21.086——
    I299.84——

    aP≤.05.

    bP≤.10.

    cNot applicable.

    Publication Bias Test

    To estimate the between-study heterogeneity, we applied the Begg and Egger tests. The Begg test estimated the rank correlation between the effect size measure and its variance, and it is more appropriate because we used log (RR) as an estimate effect size measure. We also presented the graph for the Egger test because it is the most often reported test for publication biases [93]. Our results are presented in Table 5. The Egger test graph is shown in Figure 3.

    The Begg test showed that there was no rank correlation between effect size measure and its variance. This means that there was no evidence of publication bias for this effect size. The results from the Egger test were consistent with that of the Begg test. The regression line shows that their publication bias does not seem to be present here (Figure 4).

    Table 5. Results from the Begg correlation test.
    Begg correlation testValue
    Adjusted Kendall score (P-Q)−29
    Standard deviation of score47.97
    Number of studies27
    z score−0.60
    Pr>|z|0.545
    ‎
    Figure 3. Funnel plot corresponding to Begg’s test (pseudo 95% confidence limits).
    View this figure
    ‎
    Figure 4. Regression line related to the Egger test.
    View this figure

    Principal Findings

    Our first goal was to identify the level of adherence related to eHealth tools among vulnerable groups. As the adherence of eHealth tools is a precursor for their effectiveness, we hope that our results can help to identify the potentially effective tools for vulnerable groups. In this study, we compared the proportion of people who showed intended adherence with those who did not use eHealth tools. Our results show that the pooled level of intended adherence toward eHealth tools is 7% (95% CI −0.23 to 0.38), which implies that some people from vulnerable population groups used eHealth tools over time. However, the very small percentage (7%) implies that the number of adherers can be improved. This is consistent with the findings from previous studies [95]. They reported that the use of the internet is generally lower among vulnerable groups [94]. The small percentage (7%) in this study can be related to the high level of heterogeneity. In this review, we included studies with different designs (eg, longitudinal and RCT). This, among other factors, led to heterogeneity in the estimation of adherence levels. Furthermore, the difference in adherence levels can be observed among different vulnerable groups. In the United States, migrants show higher adherence levels than people from low-income groups or older adults when they use eHealth tools [40,53]. In this study, we included not only studies that involved different vulnerable groups but also those that addressed different health outcomes. This can also be an explanation for high heterogeneity.

    Our second goal was to identify the design and implementation characteristics that influence the level of adherence within vulnerable groups. The results from the meta-regression show that design characteristics of eHealth tools, multimodal content and possibility of having a direct contact with the provider, are predictors of a higher adherence level. These two characteristics are assumed to mitigate the problems of health literacy and the digital divide among vulnerable groups. These results are particularly observed among eHealth tools that target minorities—one example is an eHealth tool for increasing knowledge on diabetes among African Americans [40]. The presence of multimodal content could increase the intrinsic motivation of participants and enable them to understand basic messages without language barriers. Furthermore, the use of multimodal content exceeds borders: eHealth tools are not only storage rooms for health information but also tools to learn how to do things or how to change health behavior. Direct interaction with providers without actual visits can save time. This is particularly important for single parents or people with low income and several jobs [40,83].

    The low adherence among vulnerable groups and the fact that some design characteristics can improve adherence might imply that people from vulnerable groups will adhere to eHealth tools more if these tools are designed in accordance with their needs. For example, people diagnosed with high blood pressure might adhere more to Web-based portals if the portal shows a video on how to change your lifestyle instead of posting a text about healthy diets [28,96]. This is related not only to language barriers but also to the comprehension of health information. Joint dysfunctionality is another potential issue with low adherence. Joint dysfunctionality occurs when eHealth tools do not connect all health services. For example, participants may use both patient portals to refill their medications and Web-based tools to decrease their weight. However, these two tools and their data may not necessarily be connected. In case they are not connected, it may negatively affect adherence for both tools; the inclusion of both tools in daily routine may be perceived as too burdensome. Adherence is one of the precursors for effectiveness of eHealth tools. Our results suggest that, although small, adherence among vulnerable groups does exist, but it develops over time. This implies that eHealth tools do have the potential to decrease disparities among vulnerable groups.

    The results from the systematic review also show that some users, although registered, never use eHealth tools. This can be explained by the fact that users might be registered by their provider. For example, GPs in The Bronx (the United States) usually register their patients to a patient portal during the regular appointment [33]. However, the registered patients never use patient portals or Web-based platforms. In other words, participants interested in eHealth tools register and continue as active users. Those without an interest in eHealth tools might be registered but without continuous use. This way eHealth tools attract a specific share of users among the vulnerable groups, and these users are consistently using the app. However, this creates the problem of how to attract new users within this population. Recent studies show that participants from vulnerable groups use eHealth tools less than other population groups [4]. One way to overcome this problem is to use inclusive tools that cover different population groups. This includes tools that are used by both younger and older users or by people from different social statuses. Another way is to capitalize on social ties and networks to expand the number of users [72]. For example, some eHealth tools allow for the use of encrypted chat groups for family members or for people with the same ethnical background.

    Furthermore, our results show that design characteristics such as the type of technology (Web-based platform, mobile phones, or patient portals) have different patterns to address vulnerable groups. The most common types of technology used for health purposes are patient portals and Web-based platforms. They are different in design and purpose. Patient portals are characterized by direct interaction between the patient and the provider. They focus mostly on older adults or the chronically sick. Kp.org, a patient portal from the United States, is such an example. They also provide training for their users. For example, patients in nursing homes receive training for computer use and navigation through the portal [66]. This way, patient portals try to overcome problems associated with the digital divide. Conversely, Web-based platforms are usually drivers for tools that were developed before as paper-and-pencil version for general population groups [97]. Furthermore, most Web-based platforms are related to obesity. This is also related to the fact that the United States has the highest rates of obesity in the world and that most of our studies come from the United States [98]. Most Web-based platforms have a clear theoretical background and a clear evaluation plan. Web-based platforms usually benefit from multimodal content—they use videos or games to improve the adherence of their users [84]. This way they also overcome the problems of health literacy. They usually focus on one specific problem—obesity or diabetes—without connecting it to other aspects of patients’ health status. They are not always directly connected to other electronical data within the health care system. Conversely, patient portals are embodied within health care systems, but they also do not cover all aspects of health care. Usually, patient portals are developed for certain health care providers (certain hospitals or insurance companies). One of the examples is a patient portal for veterans in the US army known as My HealtheVet. This portal was created to address the special needs of veterans, and it is adjusted for specialized providers. The information from this portal is not connected with health care services outside of veterans’ clinics. It is also difficult to generalize the experience from this portal to that of similar eHealth tools [13]. If patient portals were linked to all providers and allowed patients to store information from different types of services, adherence to them might improve. For example, they do not always include prevention measures or possible therapeutic advice [41]. This can be important to improve effectiveness.

    Our results also show that Web-based platforms are usually developed as exclusive tools for vulnerable groups—for example, for the gay population or Hispanic minorities [17,99,100]. This can be double sided as these groups might feel stigmatized in comparison with the general population with similar problems. Mobile phones are favored among certain vulnerable groups such as minorities that are trendsetters in their use [17]. However, our results show that only a small number of mobile health apps have been evaluated. One of the reasons might be that mobile phone apps are usually produced by small entrepreneurs. Their distribution does not require legal or ethical approval. In addition, they are very often not directly connected to health care systems [82].

    Limitations

    The results of the meta-analysis related to the adherence of eHealth tools show a high level of heterogeneity. This was expected as we included different vulnerable groups, different eHealth tools, and different diseases that these tools address. Furthermore, we included studies with different designs such as RCTs, cohort studies, or observational studies. It would be useful to run meta-analyses related to adherence for each of the designs or for each of the vulnerable groups. Heterogeneity in our meta-analysis can be due to some eHealth tools being specifically related to certain health care institutions and that they cannot be applied in other institutions. This is important for the adherence rate—people who move from one nursing home to another cannot use the same eHealth tool anymore. Patient portals related to specific nursing homes are exemplary for this situation [77,78]. In addition, results based on users from one institution are difficult to extrapolate to the population level. This is emphasized by the lack of clear patterns for evaluating eHealth tools or deciding on outcome measures related to their effectiveness [9].

    In this study, we also used meta-regression. We are aware that the number of included studies is small (N=27). This decreases the power of our analysis and might lead to biases. Furthermore, endogeneity is an issue.

    Despite our efforts to perform all subsequent steps in the searching process carefully, we might have missed some relevant studies. This might be because of our definition of vulnerable groups and the ambiguities in the terminology of eHealth. In addition, the small number of included studies did not allow us to identify design and implementation characteristics per vulnerable group. In other words, we could not determine which design characteristics suits which group the best.

    As we focused only on studies that have a reported adherence rate, this means that we excluded studies that evaluated eHealth tools using different measures. For example, some studies from low- and middle-income countries evaluate eHealth tools using only health outcomes or subjective measures such as quality of life or user satisfaction [50].

    Although most European countries and the United States do have legal regulations about the use of eHealth tools, there is still a concern about the data collected via eHealth tools. In this study, we did not pay attention to legal and ethical considerations related to eHealth tools. This can be an interesting avenue for future research.

    In the United States, many eHealth tools are funded through the federal government [54]. For example, the US government aims to spend US $38 billion in 10 years to develop eHealth for making health care more accessible. However, many end users (patients or medical providers) also pay for eHealth tools. Furthermore, many of the tools are funded by small entrepreneurs. In this study, we did not examine the source of funding and mechanisms of financing. Future research might benefit from including these characteristics.

    Conclusions

    In conclusion, the use of eHealth among vulnerable population groups is still minimal. One way to improve adherence among vulnerable groups is to design eHealth tools with multimodal content. In addition, enabling direct communication between users and medical providers can improve access to health information among vulnerable groups. Future research should focus on evaluation studies on eHealth tools and health outcomes related to them, in addition to user satisfaction. Furthermore, future research should pay attention to defining intended adherence for different vulnerable groups and related eHealth tools. Providing eHealth tools that connect different health services would potentially improve the use not only among vulnerable groups but also in the general population. Although previous studies have emphasized that eHealth tools can be used to replace regular services, this can only be possible if eHealth tools are actively used.

    To the best of our knowledge, this is the first study to synthesize the influence of design and implementation characteristics on adherence. Our results show that multimodal content—video and games—can be an incentive for use among vulnerable groups. In addition, direct communication with health care providers may increase adherence. However, the evidence is preliminary as it is based on cross-sectional analysis. These results are useful for the design of future eHealth tools. In this study, we assessed the level of intended adherence. However, we did not assess the effectiveness of eHealth tools. In other words, we did not assess the extent to which eHealth tools help vulnerable groups improve their health outcomes. Future research should also focus on the effectiveness of eHealth tools among vulnerable groups.

    Acknowledgments

    LT acknowledges funding from De Nederlandse Organisatie voor Wetenschappelijk Onderzoek Grant 016.VIDI.185.017. Furthermore, he acknowledges that this work was supported by the National Research Foundation of Korea Grant, funded by the Korean Government (NRF-2017S1A3A2067636). This study is also supported by Resilient Societies, a multidisciplinary research theme within the Faculty of Law, Economics and Governance.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Preferred Reporting Items for Systematic Reviews and Meta-Analyses Checklist.

    DOCX File , 16 KB
    ‎
    Multimedia Appendix 2

    The characteristics of studies included in meta-analysis.

    DOCX File , 33 KB
    1. Samkange-Zeeb F, Ernst SA, Klein-Ellinghaus F, Brand T, Reeske-Behrens A, Plumbaum T, et al. Assessing the acceptability and usability of an internet-based intelligent health assistant developed for use among Turkish migrants: results of a study conducted in Bremen, Germany. Int J Environ Res Public Health 2015 Dec 3;12(12):15339-15351 [FREE Full text] [CrossRef] [Medline]
    2. Auerbach AD. Evaluating digital health tools-prospective, experimental, and real world. JAMA Intern Med 2019 Jun 1;179(6):840-841. [CrossRef] [Medline]
    3. Foster J, Miller L, Isbell S, Shields T, Worthy N, Dunlop AL. mHealth to promote pregnancy and interconception health among African-American women at risk for adverse birth outcomes: a pilot study. Mhealth 2015;1:20 [FREE Full text] [CrossRef] [Medline]
    4. Latulippe K, Hamel C, Giroux D. Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res 2017 Apr 27;19(4):e136 [FREE Full text] [CrossRef] [Medline]
    5. Saberi P, Johnson MO. Correlation of internet use for health care engagement purposes and HIV clinical outcomes among HIV-positive individuals using online social media. J Health Commun 2015 Jun 29;20(9):1026-1032 [FREE Full text] [CrossRef] [Medline]
    6. Bacigalupe G, Askari SF. E-Health innovations, collaboration, and healthcare disparities: developing criteria for culturally competent evaluation. Fam Syst Health 2013 Sep;31(3):248-263. [CrossRef] [Medline]
    7. Lang A, Mertes A. e-Health policy and deployment activities in Europe. Telemed J E Health 2011 May;17(4):262-268. [CrossRef] [Medline]
    8. World Health Organization. Atlas eHealth Country Profiles: Based on the Findings of the Second Global Survey on eHealth. Geneva: World Health Organization; 2011.
    9. Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med 2007 May;32(5 Suppl):S119-S126. [CrossRef] [Medline]
    10. Guendelman S, Broderick A, Mlo H, Gemmill A, Lindeman D. Listening to communities: mixed-method study of the engagement of disadvantaged mothers and pregnant women with digital health technologies. J Med Internet Res 2017 Jul 5;19(7):e240 [FREE Full text] [CrossRef] [Medline]
    11. Flaskerud JH, Winslow BJ. Conceptualizing vulnerable populations health-related research. Nurs Res 1998;47(2):69-78. [CrossRef] [Medline]
    12. Braveman P, Gruskin S. Defining equity in health. J Epidemiol Community Health 2003 Apr;57(4):254-258 [FREE Full text] [CrossRef] [Medline]
    13. Nazi KM, Hogan TP, Wagner TH, McInnes DK, Smith BM, Haggstrom D, et al. Embracing a health services research perspective on personal health records: lessons learned from the VA My HealtheVet system. J Gen Intern Med 2010 Jan;25(Suppl 1):62-67 [FREE Full text] [CrossRef] [Medline]
    14. Baur C. An analysis of factors underlying e-health disparities. Camb Q Healthc Ethics 2008;17(4):417-428. [CrossRef] [Medline]
    15. Viswanath K, Kreuter MW. Health disparities, communication inequalities, and eHealth. Am J Prev Med 2007 May;32(5 Suppl):S131-S133 [FREE Full text] [CrossRef] [Medline]
    16. Gorman JR, Clapp JD, Calac D, Kolander C, Nyquist C, Chambers CD. Creating a culturally appropriate web-based behavioral intervention for American Indian/Alaska Native women in Southern California: the healthy women healthy Native nation study. Am Indian Alsk Native Ment Health Res 2013;20(1):1-15. [CrossRef] [Medline]
    17. Marcus BH, Hartman SJ, Larsen BA, Pekmezi D, Dunsiger SI, Linke S, et al. Pasos Hacia La Salud: a randomized controlled trial of an internet-delivered physical activity intervention for Latinas. Int J Behav Nutr Phys Act 2016 May 28;13:62 [FREE Full text] [CrossRef] [Medline]
    18. Kontos E, Blake KD, Chou WS, Prestin A. Predictors of eHealth usage: insights on the digital divide from the Health Information National Trends Survey 2012. J Med Internet Res 2014 Jul 16;16(7):e172 [FREE Full text] [CrossRef] [Medline]
    19. Mohr DC, Cuijpers P, Lehman K. Supportive Accountability: A Model for Providing Human Support to Enhance Adherence to eHealth Interventions. J Med Internet Res 2011 Mar 10;13(1):e30. [CrossRef]
    20. Morawski K, Ghazinouri R, Krumme A, McDonough J, Durfee E, Oley L, et al. Rationale and design of the Medication adherence Improvement Support App For Engagement-Blood Pressure (MedISAFE-BP) trial. Am Heart J 2017 Apr;186:40-47. [CrossRef] [Medline]
    21. Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    22. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug 5;13(3):e52 [FREE Full text] [CrossRef] [Medline]
    23. Sieverink F, Kelders SM, van Gemert-Pijnen JE. Clarifying the concept of adherence to eHealth technology: systematic review on when usage becomes adherence. J Med Internet Res 2017 Dec 6;19(12):e402 [FREE Full text] [CrossRef] [Medline]
    24. Beintner I, Vollert B, Zarski A, Bolinski F, Musiat P, Görlich D, et al. Adherence reporting in randomized controlled trials examining manualized multisession online interventions: systematic review of practices and proposal for reporting standards. J Med Internet Res 2019 Aug 15;21(8):e14181 [FREE Full text] [CrossRef] [Medline]
    25. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res 2018 Jun 6;20(6):e199 [FREE Full text] [CrossRef] [Medline]
    26. Alkhaldi G, Modrow K, Hamilton F, Pal K, Ross J, Murray E. Promoting engagement with a digital health intervention (HeLP-Diabetes) using email and text message prompts: mixed-methods study. Interact J Med Res 2017 Aug 22;6(2):e14 [FREE Full text] [CrossRef] [Medline]
    27. Pal K, Eastwood SV, Michie S, Farmer A, Barnard ML, Peacock R, et al. Computer-based interventions to improve self-management in adults with type 2 diabetes: a systematic review and meta-analysis. Diabetes Care 2014 Jun;37(6):1759-1766. [CrossRef] [Medline]
    28. McLean G, Band R, Saunderson K, Hanlon P, Murray E, Little P, DIPSS co-investigators. Digital interventions to promote self-management in adults with hypertension systematic review and meta-analysis. J Hypertens 2016 Apr;34(4):600-612 [FREE Full text] [CrossRef] [Medline]
    29. Lorence D, Park H. Group disparities and health information: a study of online access for the underserved. Health Informatics J 2008 Mar;14(1):29-38. [CrossRef] [Medline]
    30. Wangberg SC, Sørensen T, Andreassen HK. Using the internet to support exercise and diet: a stratified Norwegian survey. Med 2 0 2015 Aug 26;4(2):e3 [FREE Full text] [CrossRef] [Medline]
    31. Ronis SD, McConnochie KM, Wang H, Wood NE. Urban telemedicine enables equity in access to acute illness care. Telemed J E Health 2017 Feb;23(2):105-112. [CrossRef] [Medline]
    32. Perzynski A, Roach MJ, Shick S, Callahan B, Gunzler D, Cebul R, et al. Patient portals and broadband internet inequality. J Am Med Inform Assoc 2017 Sep 1;24(5):927-932 [FREE Full text] [CrossRef] [Medline]
    33. Ancker JS, Barrón Y, Rockoff ML, Hauser D, Pichardo M, Szerencsy A, et al. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med 2011 Oct;26(10):1117-1123 [FREE Full text] [CrossRef] [Medline]
    34. Jacobs RJ, Lou JQ, Ownby RL, Caballero J. A systematic review of eHealth interventions to improve health literacy. Health Informatics J 2016 Jun;22(2):81-98. [CrossRef] [Medline]
    35. Ronda MC, Dijkhorst-Oei L, Gorter KJ, Beulens JW, Rutten GE. Differences between diabetes patients who are interested or not in the use of a patient Web portal. Diabetes Technol Ther 2013 Jul;15(7):556-563 [FREE Full text] [CrossRef] [Medline]
    36. Chou WS, Prestin A, Lyons C, Wen K. Web 2.0 for health promotion: reviewing the current evidence. Am J Public Health 2013 Jan;103(1):e9-18. [CrossRef] [Medline]
    37. Mangone ER, Lebrun V, Muessig KE. Mobile phone apps for the prevention of unintended pregnancy: a systematic review and content analysis. JMIR Mhealth Uhealth 2016 Jan 19;4(1):e6 [FREE Full text] [CrossRef] [Medline]
    38. Feng Y, Xie W. Digital divide 2.0: the role of social networking sites in seeking health information online from a longitudinal perspective. J Health Commun 2015;20(1):60-68. [CrossRef] [Medline]
    39. Bodie GD, Dutta MJ. Understanding health literacy for strategic health marketing: eHealth literacy, health disparities, and the digital divide. Health Mark Q 2008;25(1-2):175-203. [CrossRef] [Medline]
    40. Moussa M, Sherrod D, Choi J. An e-health intervention for increasing diabetes knowledge in African Americans. Int J Nurs Pract 2013 Sep;19(Suppl 3):36-43. [CrossRef] [Medline]
    41. Krist AH, Aycock RA, Etz RS, Devoe JE, Sabo RT, Williams R, et al. MyPreventiveCare: implementation and dissemination of an interactive preventive health record in three practice-based research networks serving disadvantaged patients--a randomized cluster trial. Implement Sci 2014 Dec 11;9:181 [FREE Full text] [CrossRef] [Medline]
    42. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009 Aug 18;151(4):264-9, W64. [CrossRef] [Medline]
    43. Kruse CS, Bouffard S, Dougherty M, Parro JS. Telemedicine use in rural Native American communities in the era of the ACA: a systematic literature review. J Med Syst 2016 Jun;40(6):145 [FREE Full text] [CrossRef] [Medline]
    44. Bennett GG, Steinberg DM, Stoute C, Lanpher M, Lane I, Askew S, et al. Electronic health (eHealth) interventions for weight management among racial/ethnic minority adults: a systematic review. Obes Rev 2014 Oct;15(Suppl 4):146-158. [CrossRef] [Medline]
    45. Huxley CJ, Atherton H, Watkins JA, Griffiths F. Digital communication between clinician and patient and the impact on marginalised groups: a realist review in general practice. Br J Gen Pract 2015 Dec;65(641):e813-e821 [FREE Full text] [CrossRef] [Medline]
    46. Hudnut-Beumler J, Po'e E, Barkin S. The use of social media for health promotion in Hispanic populations: a scoping systematic review. JMIR Public Health Surveill 2016 Jul 11;2(2):e32 [FREE Full text] [CrossRef] [Medline]
    47. Heitkemper E, Mamykina L, Travers J, Smaldone A. Do health information technology self-management interventions improve glycemic control in medically underserved adults with diabetes? A systematic review and meta-analysis. J Am Med Inform Assoc 2017 Sep 1;24(5):1024-1035 [FREE Full text] [CrossRef] [Medline]
    48. Marcin JP, Shaikh U, Steinhorn RH. Addressing health disparities in rural communities using telehealth. Pediatr Res 2016 Jan;79(1-2):169-176. [CrossRef] [Medline]
    49. Coughlin S, Prochaska J, Williams LB, Besenyi G, Heboyan V, Goggans S, et al. Patient web portals, disease management, and primary prevention. Risk Manag Healthc Policy 2017;10:33-40 [FREE Full text] [CrossRef] [Medline]
    50. Njoroge M, Zurovac D, Ogara EA, Chuma J, Kirigia D. Assessing the feasibility of eHealth and mHealth: a systematic review and analysis of initiatives implemented in Kenya. BMC Res Notes 2017 Feb 10;10(1):90 [FREE Full text] [CrossRef] [Medline]
    51. Wald HS, Dube CE, Anthony DC. Untangling the Web--the impact of Internet use on health care and the physician-patient relationship. Patient Educ Couns 2007 Nov;68(3):218-224. [CrossRef] [Medline]
    52. McAuley A. Digital health interventions: widening access or widening inequalities? Public Health 2014 Dec;128(12):1118-1120. [CrossRef] [Medline]
    53. López L, Grant RW. Closing the gap: eliminating health care disparities among Latinos with diabetes using health information technology tools and patient navigators. J Diabetes Sci Technol 2012 Jan 1;6(1):169-176 [FREE Full text] [CrossRef] [Medline]
    54. Lyles CR, Fruchterman J, Youdelman M, Schillinger D. Legal, practical, and ethical considerations for making online patient portals accessible for all. Am J Public Health 2017 Oct;107(10):1608-1611. [CrossRef] [Medline]
    55. Rodriquez EJ, Pérez-Stable EJ. The time is now for eHealth research with Latinos. Am J Public Health 2017 Nov;107(11):1705-1707. [CrossRef] [Medline]
    56. Khan M, Hiratsuka VY, Dillard D, Robinson R, Mau M. Availability and deployment of telemedicine/telehealth technologies in rural Alaska. Fed Pract 2012 Dec;29(12):19-21 [FREE Full text] [Medline]
    57. Geraghty AW, Torres LD, Leykin Y, Pérez-Stable EJ, Muñoz RF. Understanding attrition from international Internet health interventions: a step towards global eHealth. Health Promot Int 2013 Sep;28(3):442-452 [FREE Full text] [CrossRef] [Medline]
    58. Fan L, Thomas M, Deitrick GE, Polomano RC. Awareness and action for eliminating health care disparities in pain care: web-based resources. J Pain Palliat Care Pharmacother 2008;22(3):243-250. [CrossRef] [Medline]
    59. McCartney M. How do we know whether medical apps work? Br Med J 2013 Mar 20;346:f1811. [CrossRef] [Medline]
    60. Baatar T, Suldsuren N, Bayanbileg S, Seded K. Telemedicine support of maternal and newborn health to remote provinces of Mongolia. Stud Health Technol Inform 2012;182:27-35. [Medline]
    61. Schneider RB, Biglan KM. The promise of telemedicine for chronic neurological disorders: the example of Parkinson's disease. Lancet Neurol 2017 Jul;16(7):541-551. [CrossRef] [Medline]
    62. Yellowlees P, Marks S, Hilty D, Shore JH. Using e-health to enable culturally appropriate mental healthcare in rural areas. Telemed J E Health 2008 Jun;14(5):486-492 [FREE Full text] [CrossRef] [Medline]
    63. Merrell RC, Doarn CR. Telemedicine for vulnerable populations. Telemed J E Health 2014 Oct;20(10):885-886. [CrossRef] [Medline]
    64. Piette J, Lun K, Moura L, Fraser H, Mechael P, Powell J, et al. Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here? Bull World Health Organ 2012 May 1;90(5):365-372 [FREE Full text] [CrossRef] [Medline]
    65. Apter AJ. Can patient portals reduce health disparities? A perspective from asthma. Ann Am Thorac Soc 2014 May;11(4):608-612. [CrossRef] [Medline]
    66. Kim E, Stolyar A, Lober WB, Herbaugh AL, Shinstrom SE, Zierler BK, et al. Challenges to using an electronic personal health record by a low-income elderly population. J Med Internet Res 2009 Oct 27;11(4):e44 [FREE Full text] [CrossRef] [Medline]
    67. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, Lopez A, et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010;15(Suppl 2):183-196 [FREE Full text] [CrossRef] [Medline]
    68. Kerr C, Murray E, Noble L, Morris R, Bottomley C, Stevenson F, et al. The potential of web-based interventions for heart disease self-management: a mixed methods investigation. J Med Internet Res 2010 Dec 2;12(4):e56 [FREE Full text] [CrossRef] [Medline]
    69. Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med 2011 Oct;26(10):1112-1116 [FREE Full text] [CrossRef] [Medline]
    70. Osborn CY, Mayberry LS, Wallston KA, Johnson KB, Elasy TA. Understanding patient portal use: implications for medication management. J Med Internet Res 2013 Jul 3;15(7):e133 [FREE Full text] [CrossRef] [Medline]
    71. Cullen KW, Thompson D, Chen T. Outcome evaluation of family eats. Health Educ Behav 2017 Feb;44(1):32-40. [CrossRef] [Medline]
    72. Joseph R, Pekmezi DW, Lewis T, Dutton G, Turner LW, Durant NH. Physical activity and social cognitive theory outcomes of an internet-enhanced physical activity intervention for African American female college students. J Health Dispar Res Pract 2013;6(2):1-18 [FREE Full text] [Medline]
    73. Joseph RP, Pekmezi D, Dutton GR, Cherrington AL, Kim Y, Allison JJ, et al. Results of a culturally adapted internet-enhanced physical activity pilot intervention for overweight and obese young adult African American women. J Transcult Nurs 2016 Mar;27(2):136-146 [FREE Full text] [CrossRef] [Medline]
    74. Campbell AN, Turrigiano E, Moore M, Miele GM, Rieckmann T, Hu M, et al. Acceptability of a web-based community reinforcement approach for substance use disorders with treatment-seeking American Indians/Alaska Natives. Community Ment Health J 2015 May;51(4):393-403 [FREE Full text] [CrossRef] [Medline]
    75. Herring SJ, Cruice JF, Bennett GG, Davey A, Foster GD. Using technology to promote postpartum weight loss in urban, low-income mothers: a pilot randomized controlled trial. J Nutr Educ Behav 2014;46(6):610-615 [FREE Full text] [CrossRef] [Medline]
    76. Billings DW, Leaf SL, Spencer J, Crenshaw T, Brockington S, Dalal RS. A randomized trial to evaluate the efficacy of a web-based HIV behavioral intervention for high-risk African American women. AIDS Behav 2015 Jul;19(7):1263-1274 [FREE Full text] [CrossRef] [Medline]
    77. Smith SG, O'Conor R, Aitken W, Curtis LM, Wolf MS, Goel MS. Disparities in registration and use of an online patient portal among older adults: findings from the LitCog cohort. J Am Med Inform Assoc 2015 Jul;22(4):888-895 [FREE Full text] [CrossRef] [Medline]
    78. Levy N, Moynihan V, Nilo A, Singer K, Bernik LS, Etiebet M, et al. The Mobile Insulin Titration Intervention (MITI) for insulin adjustment in an urban, low-income population: randomized controlled trial. J Med Internet Res 2015 Jul 17;17(7):e180 [FREE Full text] [CrossRef] [Medline]
    79. Jhamb M, Cavanaugh KL, Bian A, Chen G, Ikizler TA, Unruh ML, et al. Disparities in electronic health record patient portal use in nephrology clinics. Clin J Am Soc Nephrol 2015 Nov 6;10(11):2013-2022 [FREE Full text] [CrossRef] [Medline]
    80. Gordon NP, Hornbrook MC. Differences in access to and preferences for using patient portals and other eHealth technologies based on race, ethnicity, and age: a database and survey study of seniors in a large health plan. J Med Internet Res 2016 Mar 4;18(3):e50 [FREE Full text] [CrossRef] [Medline]
    81. Nazi KM, Hogan TP, McInnes DK, Woods SS, Graham G. Evaluating patient access to Electronic Health Records: results from a survey of veterans. Med Care 2013 Mar;51(3 Suppl 1):S52-S56. [CrossRef] [Medline]
    82. Ernsting C, Dombrowski SU, Oedekoven M, O Sullivan JL, Kanzler M, Kuhlmey A, et al. Using smartphones and health apps to change and manage health behaviors: a population-based survey. J Med Internet Res 2017 Apr 5;19(4):e101 [FREE Full text] [CrossRef] [Medline]
    83. Arcury TA, Quandt SA, Sandberg JC, Miller DP, Latulipe C, Leng X, et al. Patient portal utilization among ethnically diverse low income older adults: observational study. JMIR Med Inform 2017 Nov 14;5(4):e47 [FREE Full text] [CrossRef] [Medline]
    84. Aalbers T, Qin L, Baars MA, de Lange A, Kessels RP, Rikkert MG. Changing behavioral lifestyle risk factors related to cognitive decline in later life using a self-motivated eHealth intervention in dutch adults. J Med Internet Res 2016 Jun 17;18(6):e171 [FREE Full text] [CrossRef] [Medline]
    85. Cavallo DN, Sisneros JA, Ronay AA, Robbins CL, Jilcott Pitts SB, Keyserling TC, et al. Assessing the feasibility of a web-based weight loss intervention for low-income women of reproductive age: a pilot study. JMIR Res Protoc 2016 Feb 26;5(1):e30 [FREE Full text] [CrossRef] [Medline]
    86. Steinberg DM, Levine EL, Lane I, Askew S, Foley PB, Puleo E, et al. Adherence to self-monitoring via interactive voice response technology in an eHealth intervention targeting weight gain prevention among Black women: randomized controlled trial. J Med Internet Res 2014 Apr 29;16(4):e114 [FREE Full text] [CrossRef] [Medline]
    87. Bickmore TW, Utami D, Matsuyama R, Paasche-Orlow MK. Improving access to online health information with conversational agents: a randomized controlled experiment. J Med Internet Res 2016 Jan 4;18(1):e1 [FREE Full text] [CrossRef] [Medline]
    88. Ryan JG, Schwartz R, Jennings T, Fedders M, Vittoria I. Feasibility of an internet-based intervention for improving diabetes outcomes among low-income patients with a high risk for poor diabetes outcomes followed in a community clinic. Diabetes Educ 2013;39(3):365-375. [CrossRef] [Medline]
    89. Buis LR, Artinian NT, Schwiebert L, Yarandi H, Levy PD. Text messaging to improve hypertension medication adherence in African Americans: BPMED intervention development and study protocol. JMIR Res Protoc 2015 Jan 2;4(1):e1 [FREE Full text] [CrossRef] [Medline]
    90. Zingg W, Castro-Sanchez E, Secci F, Edwards R, Drumright L, Sevdalis N, et al. Innovative tools for quality assessment: integrated quality criteria for review of multiple study designs (ICROMS). Public Health 2016 Apr;133:19-37. [CrossRef] [Medline]
    91. Higgins JP, White IR, Anzures-Cabrera J. Meta-analysis of skewed data: combining results reported on log-transformed or raw scales. Stat Med 2008 Dec 20;27(29):6072-6092 [FREE Full text] [CrossRef] [Medline]
    92. Huang J, Maassen van den Brink H, Groot W. A meta-analysis of the effect of education on social capital. Eco Educ Rev 2009;28(4):454-464. [CrossRef]
    93. Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J 1997 Sep 13;315(7109):629-634 [FREE Full text] [CrossRef] [Medline]
    94. Torrens E, Walker SM. Demographic characteristics of Australian health consumers who were early registrants for opt-in personally controlled electronic health records. HIM J 2017 Apr 27;46(3):127-133. [CrossRef]
    95. Makai P, Perry M, Robben SH, Schers HJ, Heinen MM, Rikkert MG, et al. Evaluation of an eHealth intervention in chronic care for frail older people: why adherence is the first target. J Med Internet Res 2014 Jun 23;16(6):e156 [FREE Full text] [CrossRef] [Medline]
    96. Debon R, Coleone JD, Bellei EA, De Marchi AC. Mobile health applications for chronic diseases: a systematic review of features for lifestyle improvement. Diabetes Metab Syndr 2019;13(4):2507-2512. [CrossRef] [Medline]
    97. Carter E, Nunlee-Bland G, Callender C. A patient-centric, provider-assisted diabetes telehealth self-management intervention for urban minorities. Perspect Health Inf Manag 2011 Jan 1;8:1b [FREE Full text] [Medline]
    98. Flegal KM, Carroll MD, Kit BK, Ogden CL. Prevalence of obesity and trends in the distribution of body mass index among US adults, 1999-2010. J Am Med Assoc 2012 Feb 1;307(5):491-497. [CrossRef] [Medline]
    99. Benitez TJ, Cherrington AL, Joseph RP, Keller C, Marcus B, Meneses K, et al. Using web-based technology to promote physical activity in Latinas: results of the Muévete Alabama pilot study. Comput Inform Nurs 2015 Jul;33(7):315-324 [FREE Full text] [CrossRef] [Medline]
    100. Hall EW, Sanchez TH, Stein AD, Stephenson R, Zlotorzynska M, Sineath RC, et al. Use of videos improves informed consent comprehension in web-based surveys among internet-using men who have sex with men: a randomized controlled trial. J Med Internet Res 2017 Mar 6;19(3):e64 [FREE Full text] [CrossRef] [Medline]

    ‎
    eHealth: electronic health
    GP: general practitioner
    ICROMS: Integrated quality Criteria for the Review of Multiple Study designs
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analysis
    RCT: randomized controlled trial
    RR: risk ratio
    WHO: World Health Organization

    Edited by G Eysenbach; submitted 13.08.18; peer-reviewed by D López López, A Nguyen, D Chisolm, H Zeeb, R Jacobs; comments to author 01.04.19; revised version received 26.05.19; accepted 26.09.19; published 06.02.20

    Copyright

    ©Jelena Arsenijevic, Lars Tummers, Niels Bosma. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 06.02.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.