This paper is in the following e-collection/theme issue:

E-Health Policy and Health Systems Innovation (207) Adoption and Change Management of eHealth Systems (670) Health Services in Resource-Poor Settings and LMICs (371) eHealth Infrastructures (306)

Published on in Vol 23, No 6 (2021): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/23715, first published .
eHealth Implementation Issues in Low-Resource Countries: Model, Survey, and Analysis of User Experience

eHealth Implementation Issues in Low-Resource Countries: Model, Survey, and Analysis of User Experience

eHealth Implementation Issues in Low-Resource Countries: Model, Survey, and Analysis of User Experience

Authors of this article:

Norman Archer1 Author Orcid Image ;   Cynthia Lokker2 Author Orcid Image ;   Maryam Ghasemaghaei1 Author Orcid Image ;   Deborah DiLiberto3 Author Orcid Image
Article Authors Cited by (15) Tweetations (4) Metrics

    Original Paper

    1Information Systems, DeGroote School of Business, McMaster University, Hamilton, ON, Canada

    2Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

    3Global Health, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

    *all authors contributed equally

    Corresponding Author:

    Norman Archer, PhD

    Information Systems

    DeGroote School of Business

    McMaster University

    1280 Main St. West

    Hamilton, ON, L8S 4M4

    Canada

    Phone: 1 905 525 9140 ext 23944

    Email: archer@mcmaster.ca


    Background: The implementation of eHealth in low-resource countries (LRCs) is challenged by limited resources and infrastructure, lack of focus on eHealth agendas, ethical and legal considerations, lack of common system interoperability standards, unreliable power, and shortage of trained workers.

    Objective: The aim of this study is to describe and study the current situation of eHealth implementation in a small number of LRCs from the perspectives of their professional eHealth users.

    Methods: We developed a structural equation model that reflects the opinions of professional eHealth users who work on LRC health care front lines. We recruited country coordinators from 4 LRCs to help recruit survey participants: India, Egypt, Nigeria, and Kenya. Through a web-based survey that focused on barriers to eHealth implementation, we surveyed 114 participants. We analyzed the information using a structural equation model to determine the relationships among the constructs in the model, including the dependent variable, eHealth utilization.

    Results: Although all the model constructs were important to participants, some constructs, such as user characteristics, perceived privacy, and perceived security, did not play a significant role in eHealth utilization. However, the constructs related to technology infrastructure tended to reduce the impact of concerns and uncertainties (path coefficient=−0.32; P=.001), which had a negative impact on eHealth utilization (path coefficient=−0.24; P=.01). Constructs that were positively related to eHealth utilization were implementation effectiveness (path coefficient=0.45; P<.001), the countries where participants worked (path coefficient=0.29; P=.004), and whether they worked for privately or publicly funded institutions (path coefficient=0.18; P<.001). As exploratory research, the model had a moderately good fit for eHealth utilization (adjusted R2=0.42).

    Conclusions: eHealth success factors can be categorized into 5 groups; our study focused on frontline eHealth workers’ opinions concerning 2 of these groups: technology and its support infrastructure and user acceptance. We found significant disparities among the responses from different participant groups. Privately funded organizations tended to be further ahead with eHealth utilization than those that were publicly funded. Moreover, participant comments identified the need for more use of telemedicine in remote and rural regions in these countries. An understanding of these differences can help regions or countries that are lagging in the implementation and use of eHealth technologies. Our approach could also be applied to detailed studies of the other 3 categories of success factors: short- and long-term funding, organizational factors, and political or legislative aspects.

    J Med Internet Res 2021;23(6):e23715

    doi:10.2196/23715

    Keywords

    eHealth; low-resource countries; eHealth implementation effectiveness; end user survey; eHealth utilization 


    Background

    eHealth is the cost-effective and secure use of information communication technology in support of health and health-related fields, including health-care services, health surveillance, health literature, and health education, knowledge and research [1].

    Properly implemented eHealth has the potential to scale up the delivery of health care to people in low-resource countries (LRCs) [2]. Quality of health care has been found to contribute the most to the success of these eHealth interventions, whereas cost contributes the most to intervention failures [3]. The challenges faced when implementing eHealth vary among countries. For eHealth solutions to succeed in LRCs, an organized approach must be used to address these challenges.

    A 2016 World Health Organization report [1] indicates significant progress in eHealth planning:

    More than half of WHO Member States now have an eHealth strategy, and 90% of eHealth strategies reference the objectives of UHC [Universal Health Coverage] or its key elements. It is becoming mainstream for countries to have policies for managing information. When well-articulated, eHealth strategies should enable the interoperability needed to support people-centred health services for everyone, and the move from disease silos to resilient health systems which can deliver UHC.

    Although the World Health Organization claims that more than half of their member states have an eHealth strategy, actual implementation of their strategies is not always followed. This is clear from published studies and concerns expressed by many researchers [3-7]. For example, Kiberu et al [4] suggest that although many sub-Saharan African countries are evaluating eHealth as a means of improving health care accessibility, several are engaged in the proof-of-concept stage of unsustainable pilot projects. There are no national guidelines in many LRCs for the secure management of individual digital health information and services, placing personal data at risk. Implementation issues with standards and interoperability can create barriers to the use of eHealth and its spread across regions or nations to support the full realization of potential health system benefits. The potential for eHealth to reduce health care costs and enable access to better quality health care is limited, often due to inadequate funding, inadequate infrastructure causing power blackouts, poor internet connectivity, and an unskilled eHealth workforce. However, regardless of having to work in such difficult circumstances, progress is being made in implementing eHealth in many LRCs. For example, a few of the many research papers that have been published by LRC researchers include critical issues such as eHealth being used to combat infant mortality in rural and remote regions of India [8] and Nigeria [9], open-source electronic health record systems that support interoperable links among them have been installed in Kenya [10], and telecom policies developed to encourage optimal digital network implementation to support eHealth in Egypt [11].

    Objective

    The objective of this study is to create and validate a model of the factors that influence the successful implementation of eHealth in LRCs, based on eHealth challenges identified in a survey of LRC eHealth end users. A recent categorization [12], modified by Ahmed et al [13], has synthesized eHealth success factors into 5 categories: (1) technology and its support infrastructure, (2) user acceptance, (3) short- and long-term funding, (4) organizational factors, and (5) political or legislative aspects. Our study focuses on factors 1 and 2, which are likely to be of interest and intimately familiar to the end user participants we surveyed.


    Construct Development for the eHealth Implementation Issues Model

    Figure 1 shows the model used in this study. The following discussion describes the details of the model’s development and related hypotheses (shown in small rectangles in the diagram), including the background of the model constructs. The study based on this model was implemented through a web-based questionnaire detailed in Multimedia Appendix 1.

    ‎
    Figure 1. Structural equation model of eHealth implementation issues in low-resource countries.
    View this figure
    Task Characteristics

    Tasks are the activities that support the organization. Information systems facilitate completion of the organization’s tasks. Compatibility with work processes or work styles and task difficulty are often used to characterize tasks [14]. Research has shown [15] that if a system is more aligned with the needs of the users, there is a greater chance of system success. A fit-variability model [16] (related to the task-technology fit model by Goodhue [15]) showed that different stakeholders may perceive eHealth viability and fit of the same eHealth services very differently and that there can be discrepancies between organizational viability and individual fit of specific eHealth services. Relevant information from the study by Goodhue and Thompson [15] was used to derive the task characteristics construct for the questionnaire (Multimedia Appendix 1). This leads to:

    • Hypothesis 1a: Task characteristics will positively affect eHealth usability.
    User Characteristics

    “User characteristics are attitudes, perceptions, and demographics that are specific to the individual users of the information system” [14]. A survey of 465 medical professionals in northwest Nigerian hospitals [7] showed that the majority of the participants had a good level of literacy for implementing and working with new eHealth systems. They found statistically significant positive correlations between intention to use eHealth and attitude toward eHealth, perceived usefulness toward eHealth, information technology experience and eHealth, and technical infrastructure for eHealth. The user characteristics construct for the questionnaire (Multimedia Appendix 1) was adapted from the study by Zayyad and Toycan [7]. This leads to:

    • Hypothesis 1b: User characteristics will positively affect eHealth usability.
    • Hypothesis 2a: User characteristics will negatively affect concerns and uncertainties.
    Perceived Technology Infrastructure

    Information communications and technology is a key component of any eHealth program. For an eHealth program to succeed, technology must be available to support a database that is always directly and easily accessible to practitioners for health record input, retrieval, analysis, and sharing within the patient’s circle of care. The infrastructure must include stable and reliable digital communications network hardware and software to support health record access and sharing through point-of-care devices used by health care providers. Highly reliable electrical power must also be available at all times to drive the components of the infrastructure [4,6,7,17]. The relevant perceived technology infrastructure construct for the questionnaire (Multimedia Appendix 1) was adapted from the study by Zayyad and Toycan [7] as follows:

    • Hypothesis 1c: Perceived technology infrastructure will positively affect eHealth usability.
    • Hypothesis 2b: Perceived technology infrastructure will negatively affect concerns and uncertainties.
    Perceived Privacy

    A study of eHealth in Bangladesh found that privacy concerns by patients [18] did not have a significant impact on provider concerns about eHealth use. An explanation is that poor eHealth privacy and security considerations resulted in eHealth systems being judged by patients as failures [19]. This is the opposite of the findings from users of eHealth (clinicians) who may tend to believe that privacy and security are less important when an eHealth system provides superior health care. The perceived privacy construct for the questionnaire (Multimedia Appendix 1) was derived from the study by Wilkowska and Ziefle [20], as follows:

    • Hypothesis 2c: Perceived privacy will positively affect concerns and uncertainties.
    Perceived Security

    Security and privacy issues are related in eHealth systems; it is not possible to manage privacy without a secure system. A survey of professionals working in Nigerian hospitals addressed the professionals’ intentions to use eHealth [7]. A question related to security concerns showed a nonsignificant correlation of this issue with the intention to use eHealth. The perceived security construct of the questionnaire (Multimedia Appendix 1) was derived from the study by Wilkowska and Ziefle [20] as follows:

    • Hypothesis 2d: Perceived security will positively affect concerns and uncertainties.
    eHealth Usability

    Usability represents an important yet often overlooked factor impacting the implementation and meaningful use of eHealth systems. Without usable systems, doctors, medical technicians, nurses, administrative staff, and other users would have great difficulty in realizing the potential benefits of eHealth systems. The usability of technical systems has been studied in the information systems field, beginning with the landmark work by Davis [21]. This is a key measure of an eHealth system and directly reflects how users may react positively to its use. In general, if the system is built to perform specific user tasks, its usability will be greater. The eHealth usability construct for the questionnaire (Multimedia Appendix 1) was adapted from the study by Davis [21]. The foregoing leads to the following:

    • Hypothesis 3a: eHealth usability will positively affect eHealth utilization.
    Concerns and Uncertainties About eHealth

    A review of empirical research classifying eHealth implementations as successes or failures [3] found that quality of health care was most often mentioned as contributing to the success of eHealth interventions. This review found that costs were most frequently mentioned as contributing to failure, although workflow issues were also mentioned in most of the articles reviewed. Workflow issues could lead to disagreement among the affected clinicians, increasing uncertainties, and the potential for failure of the eHealth system. The concerns and uncertainties about eHealth construct of the questionnaire (Multimedia Appendix 1) adapted ideas expressed by Aranda-Jan et al [22] in a systematic review of what does not work in African eHealth projects, leading to the following hypotheses:

    • Hypothesis 1d: Concerns and uncertainties about eHealth systems will negatively affect eHealth usability.
    • Hypothesis 3b: Concerns and uncertainties about eHealth systems will negatively affect eHealth utilization.
    Perceived Implementation Effectiveness

    Underlying factors affect health care professionals’ decisions to implement eHealth technology applications in LRCs [7]. These include the perceived usefulness, belief, willingness, and attitude of health care professionals. Our study implicitly reflects these factors in terms of survey feedback from users who have chosen to implement eHealth in their workplaces. The technological capability of eHealth systems is one of the key factors that influence the successful implementation of a technology [17]. Technological success factors include functional and nonfunctional requirements, interoperability, and user interface design. The long-term sustainability of a system depends on the economic, social, and organizational sustainability in which the technology is embedded. The perceived eHealth implementation effectiveness construct of the questionnaire (Multimedia Appendix 1) was developed from ideas expressed by Rezai-Rad et al [23] and is stated as:

    • Hypothesis 3c: Perceived implementation effectiveness will positively affect eHealth utilization.
    eHealth System Utilization

    Utilization of an eHealth system is a measure of how popular the system is with the users and if it will be sustainable and worth the operating cost in the long run. This was measured in our study by eHealth system utilization, a one-indicator formative construct Q10 (Multimedia Appendix 1) that lists possible eHealth utilization levels by the participant’s organization.

    Table 1 summarizes the reference sources mentioned above that were used to create the eHealth implementation model.

    Table 1. Summary of sources for eHealth implementation model constructs.
    TitleConstructTypeStudy
    Perceived task characteristicsValidatedReflectiveGoodhue and Thompson [15]
    Individual characteristicsNewReflectiveZayyad and Toycan [7]
    Perceived technology infrastructureNewReflectiveZayyad and Toycan [7]
    Perceived eHealth privacyValidatedReflectiveWilkowska and Ziefle [20]
    Perceived eHealth securityValidatedReflectiveWilkowska and Ziefle [20]
    eHealth usabilityValidatedReflectiveDavis [21]
    Concerns and uncertainties about eHealthNewReflectiveAranda-Jan et al [22]
    eHealth implementation effectivenessNewReflectiveRezai-Rad et al [23]
    eHealth utilizationNewFormative (1-indicator variable)N/Aa

    aN/A: not applicable (as this construct is developed in this study).

    Implementation

    Overview

    This study was approved by the McMaster University Research Ethics Board. In addition to its own approval process, the board contacted eHealth authorities in each of the 4 LRCs to ensure that the research proposal was acceptable. The study proceeded in 2 phases: I and II. An individual country coordinator with experience in eHealth implementations was recruited for each of the 4 countries involved in the study. They received nominal reimbursement for managing the recruitment of eHealth expert consultants for phase I and survey participants for phase II in their respective countries. eHealth consultants in the phase 1 study also received a nominal sum. Participants were not paid for completing the survey (phase II). In the 2 countries where 35 or more participants were recruited, a random draw prize was awarded to 1 participant in each country.

    Phase I

    On the basis of a detailed review of the relevant eHealth literature, we developed a draft questionnaire. Our study focused on eHealth users and support staff in the 4 representative LRCs: Kenya (East Africa), Nigeria (West Africa), India (South Asia), and Egypt (North Africa). Consultations with the experts on the phase I questionnaire informed the design of the final questionnaire and highlighted the importance of the factors identified from the literature. A key finding was that the model was too broad, including a number of strategic concerns that individual participants, as clinicians or other end users, were less likely to be directly concerned with. These strategic concerns (3 of the 5 categories [12] referenced earlier in this paper: short- and long-term funding, organizational factors, and political or legislative aspects) made the questionnaire too long for busy eHealth users to be willing to complete. Therefore, we reduced the scope of the issues covered to the first 2 categories (technology and its support infrastructure and user acceptance). The final questionnaire was based on the more limited model shown in Figure 1. Each reflective construct in the model included at least three indicator variables, which were presented on a 7-point Likert scale. The resulting questionnaire is provided in Multimedia Appendix 1, and the information, consent, and invitation to participate messages to the survey participants are provided in Multimedia Appendix 2. The web-based version, developed using Qualtrics software[24], took about 15 minutes for the participants to complete.

    Phase II

    A convenience sampling survey of eHealth users in the 4 LRCs was arranged by the relevant country coordinators, who recruited suitable participants. Participants were from public and private institutions in rural and urban areas and had varying levels of eHealth experience. Details of the survey process are provided in Multimedia Appendix 3. From 177 invitations to participate, 114 (64.4% overall response rate) valid responses were completed from India (39/114, 34.2%), Egypt (52/114, 45.6%), Kenya (11/114, 9.6%), and Nigeria (12/114, 10.5%). Statistical data were analyzed using partial least squares with Smart PLS3 software [25].


    Participant Demographics

    Participant demographics are detailed in Table 2, including country comparisons. In the table, the Total column represents the number of participants from each country who completed the survey successfully. All data are presented as absolute values and percentages. Percentages in the total column on the righthand side sum to 100% for the categories presented in each of the 5 table divisions that were also used, along with country, as control variables (occupation, employer, eHealth experience, urban or rural experience, and employment experience).

    Table 2. Participant demographics.
    CharacteristicsCountry, n (%)Total (N=114), n (%)

    		India (n=39)Egypt (n=52)Kenya (n=11)Nigeria (n=12)
    Occupation

    		Physicians8 (20.5)20 (38.4)1 (9.1)1 (8.3)81 (71.1)

    		Nurses1 (2.5)1 (1.9)1 (9.1)0 (0)8 (7)

    		Allied health personnel4 (10.2)3 (5.8)1 (9.1)1 (8.3)25 (21.9)
    Employer

    		Work in privately funded health care10 (25.4)10 (19.2)1 (9.1)1 (8.3)61 (53.5)

    		Work in publicly funded health care3 (7.7)14 (26.9)1 (9.1)1 (8.3)53 (46.5)
    eHealth experience

    		No previous experience with eHealth4 (10.2)7 (13.4)0 (0)0 (0)26 (22.8)

    		2 or more years of experience with eHealth6 (15.3)19 (36.5)2 (18.2)1 (8.3)88 (77.2)
    Urban versus rural experience

    		eHealth experience only in urban settings1 (2.5)10 (19.2)1 (9.1)0 (0)28 (24.6)

    		eHealth experience only in rural settings0 (0)0 (0)0 (0)0 (0)2 (1.8)

    		eHealth experience in both rural and urban settings12 (31.6)14 (26.9)1 (9.1)1 (8.3)84 (73.6)
    Employment experience

    		Predominant eHealth experience in clinics0 (0)6 (11.5)1 (9.1)1 (8.3)26 (22.7)

    		Predominant eHealth experience in education1 (2.5)8 (15.3)0 (0)1 (8.3)23 (20.6)

    		Predominant eHealth experience in technology support3 (7.7)3 (5.8)0 (0)0 (0)16 (13.7)

    		Predominant eHealth experience in training0 (0)5 (9.6)0 (0)0 (0)12 (10.5)

    		Predominant eHealth experience in monitoring and evaluation1 (2.5)7 (13.4)1 (9.1)0 (0)20 (18)

    		Predominant eHealth experience in administration1 (2.5)2 (3.8)1 (9.1)0 (0)10 (8.6)

    		Predominant eHealth experience in planning0 (0)3 (5.8)0 (0)0 (0)7 (5.9)

    eHealth Implementation Model Results

    Figure 2 shows the results calculated from the structural equation model, which was run with bootstrapping using 1000 subsamples. The calculated path coefficients for the proposed hypotheses, shown in Figure 2, are listed in Table 3. Hypotheses H1a, H2b, H3b, and H3c were supported, whereas the remaining hypotheses (H1b, H1c, H1d, H2a, H2c, H2d, and H3a) were not supported (all with P>.05).

    Control variables (for the demographic categories in Table 2) were also run against the model, and those with significant results are included in Figure 2. These are (1) participant’s country and (2) employer private or public funded. The results for these control variables are shown in the lower part of Table 3.

    The composite reliabilities and average variance extracted (AVE) for the reflective constructs are shown in Table 4. The composite reliability of a construct measures the reliability of the indicator variables included in the construct. All the composite reliabilities were above the accepted lower limit of 0.70 [26]. The AVE results measure the fit of the internal structure of the model. AVE is slightly below the accepted lower limit of 0.50 for concerns and uncertainties, but all the other values meet the lower limit within the rounding error; therefore, with this exception, the model has convergent validity. The heterotrait ratio of correlations [27], shown in Table 5, assesses the discriminant validity in the model. The resulting maximum value of 0.79 is below the 0.85 threshold, so discriminant validity is established. The adjusted R2 values for usability, concerns and uncertainties, and eHealth utilization are listed in Table 6.

    Participants’ responses to the eHealth utilization construct were analyzed according to the extent to which the participants indicated that eHealth was used in their organization. The results, stratified by country, are shown as absolute values and percentages in Table 7.

    ‎
    Figure 2. Model results for eHealth implementation issues.
    View this figure
    Table 3. Calculated path coefficients and significance.
    RelationshipPath coefficientP value
    Hypothesis

    		H1a: Task characteristics→aeHealth usability0.33<.001

    		H1b: User characteristics→eHealth usability−0.03.87

    		H1c: Perceived technology infrastructure→eHealth usability0.10.42

    		H1d: Concerns and uncertainties about eHealth→eHealth usability0.01.97

    		H2a: User characteristics→concerns and uncertainties−0.08.47

    		H2b: Perceived technology infrastructure→concerns and uncertainties−0.32<.001

    		H2c: Perceived privacy→concerns and uncertainties0.20.17

    		H2d: Perceived security→concerns and uncertainties0.02.93

    		H3a: Usability→eHealth utilization0.02.88

    		H3b: Concerns and uncertainties→eHealth utilization−0.24.01

    		H3c: Perceived implementation effectiveness→eHealth utilization0.45<.001
    Significant control variables

    		Country of participant→eHealth utilization0.29.004

    		Private or public funding→eHealth utilization0.18.009

    aArrows represent the directional relationships of the coefficients.

    Table 4. Composite reliabilities and average variance extracted for reflective constructs.
    ConstructComposite reliabilityAVEa
    Concerns and uncertainties0.740.42
    Perceived implementation effectiveness0.890.73
    Perceived privacy0.880.71
    Perceived security0.720.49
    Perceived technology infrastructure0.860.68
    Perceived usability0.850.59
    Task characteristics0.780.48
    User characteristics0.890.72

    aAVE: average variance extracted.

    Table 5. Discriminant analysis via heterotrait-monotrait ratio of correlations.
    ConstructsConcerns and uncertaintyPerceived implementation effectivePerceived privacyPerceived securityPerceived technology infrastructureTask characteristicsUsabilityUser characteristics
    Perceived implementation effectiveness0.36—a——————
    Perceived privacy0.360.16——————
    Perceived security0.170.140.79—————
    Perceived technology infrastructure0.460.540.040.10————
    Task characteristics0.330.770.250.230.57———
    Usability0.190.320.270.240.250.48——
    User characteristics0.250.530.140.100.440.480.15—
    eHealth utilization0.350.530.020.180.440.550.330.24

    aNot applicable.

    Table 6. Adjusted R2 from model calculations.
    Latent variableAdjusted R2
    Usability0.12
    Concerns and uncertainties0.13
    eHealth utilization0.42
    Table 7. Participants’ responses to the question “indicate to what extent eHealth is used in your organization.”
    Extent of eHealth use in my organizationTotal (N=114), n (%)Egypt (n=52), n (%)India (n=39), n (%)Nigeria (n=12), n (%)Kenya (n=11), n (%)
    Never3 (2.6)3 (5.8)0 (0)0 (0)0 (0)
    To a very small extent22 (19.3)12 (23.1)9 (23.1)0 (0)1 (9.1)
    To a small extent24 (21.1)12 (23.1)4 (10.2)5 (41.6)3 (27.3)
    To a moderate extent35 (30.7)14 (26.9)15 (38.5)4 (33.3)2 (18.2)
    To a fairly great extent17 (14.9)8 (15.4)6 (15.4)2 (16.7)1 (9.1)
    To a great extent6 (5.3)3 (5.8)2 (5.1)0 (0)1 (9.1)
    To a very great extent7 (6.1)0 (0)3 (7.7)1 (8.3)3 (27.3)

    Principal Findings

    The focus of our study was on providers and support staff with applied experience in eHealth and the extent of use they perceived of eHealth in their organizations. The structural equation model we developed resulted in an estimate of the extent of eHealth utilization in their organizations (adjusted R2=0.42). This result is in the range of a moderately good result [28] for an exploratory study.

    In the model, only task characteristics contributed significantly to usability (Figure 2). When the model was run without control variables in place, it gave an adjusted R2 of 0.310 for eHealth utilization, with the path coefficient usability→eHealth utilization significant at 0.18 (P=.01). Other path coefficients changed very little, but this one became nonsignificant at a value of 0.015 (P=.88) when the 2 significant control variables (country and public or private employment) were included in the final model. This result tells us that there was variability among the opinions of the participants about eHealth utilization that depended upon their home country and their employer (private or public).

    eHealth Utilization

    eHealth utilization in participant organizations was dealt with in question 10 of Multimedia Appendix 1. Its raw data results were analyzed and are presented in Table 7. These results show a similar distribution of engagement level of eHealth in health care organizations in each of the 4 countries. The median use of eHealth by the LRC participant organizations was to a moderate extent, as indicated by calculations from the raw data in response to the question in Multimedia Appendix 1the extent of eHealth use in your organization. This suggests a generally favorable reaction to the applicability of eHealth to health care organization work. Although the sample sizes of the Nigerian and Kenyan responses were too small for statistical comparisons, there was little difference in their average results from the Egyptian and Indian responses.

    Privacy and Security

    Privacy and its related supporting functionality, that is, security, were not found to be significant to the model construct eHealth concerns and uncertainties and, thus, to eHealth utilization, which is in agreement with similar studies [7,18]. Although this may be the case for eHealth users such as our participants, patients themselves in other LRC studies were found to be concerned about privacy violations through secondary or unauthorized access [19,29]. However, the lack of significance in our model did not mean that privacy and security were unimportant to the participants. From the raw data for question 4 in Multimedia Appendix 1, the overall result was a median value of 7 (you strongly agree) and a mean value of 6.3 (you agree) for the 3 positive statements in the questionnaire about the relevance of privacy, and similar results for the 3 positive statements in the questionnaire about the relevance of security (Multimedia Appendix 1). These are favorable results that did not have a significant impact on the model results because they did not seem to be of concern to most participants in relation to eHealth utilization.

    Relevance to Previous Literature

    The recent history of eHealth implementation and experimentation in many LRCs has resulted in research and publication of many relevant results, including those that touched on sustainable implementation of eHealth in these countries [3,7,17,18,30,31]. In addition, Mauco et al [32] developed and validated an eHealth readiness assessment framework for developing countries. It includes a comprehensive set of readiness factors, including organizational, technological or infrastructural, government, societal, health care provider, engagement, core, and public- or patient-related. Another study by Ahmed et al [13], adapted from the study by Broens et al [12], synthesized 5 mobile health and telehealth (generalizable to eHealth) success factors, which we referenced in the introduction to this paper. Our research focused on 2 of these success factors (technology and its support infrastructure and user acceptance) to portray their effects on user perceptions.

    The implementation of eHealth systems in LRCs differs from past activities with the more mature systems in developed nations. For example, many LRCs have been implementing pilot eHealth systems, some of which have been successful whereas others have not. Some of these implementations have ignored long-term effects, such as nonstandard systems that do not interoperate with other existing or planned systems [4]. Mistakes of this nature were also made when eHealth systems were initially being used in developed nations, and it is important to avoid making the same expensive mistakes in LRCs.

    Other eHealth Implementation Measures

    Whether hospitals implement experimental or full eHealth applications, their operations are a source of data and user opinions that could be harvested to deduce predictions and possible causes of success or failure in future installations. The problem is that almost all related research on LRCs has been based on single installations or systematic reviews [3,7,22,23,32], and it is difficult to generalize from these to validate a theoretical framework. We note that there are organizations that publish hospital rankings in different regions and countries, including most, if not all, LRCs (eg, Ranking Web of Hospitals [33]). These rankings do not specifically include eHealth considerations. More specific to hospital implementations of eHealth is the Electronic Medical Record Adoption Model (EMRAM; HIMSS Analytics) [34]. EMRAM is an 8-level maturity model, beginning at level 0 (no eHealth facilities) and improving to level 7 (virtually complete implementation of eHealth, including electronic medical records, external digital links, privacy, security, disaster recovery, data analytics, and data governance). Forward-looking hospital managers aspire to move upward on this scale. Although few hospitals outside the United States have reached levels 6 or 7, many hospitals in the United States have reached levels 6 or 7. Although the EMRAM approach has been used primarily in developed nations, its advantage is that it encourages hospitals to modernize their facilities through eHealth implementation in a carefully managed manner. It also helps the hospital management to look for advice from other hospitals to help them move more effectively to a higher level of eHealth implementation. The need for this more organized approach is similar in LRCs, except that LRC hospitals are often starting at an introductory, low level of eHealth use, where initial eHealth system adoption requires extensive changes in hospital operations and employee training.

    Implications for Future Design

    Our research takes the first step in the use of local eHealth experience by combining the opinions of actual eHealth end users from several LRCs. Regardless of the digital design of eHealth systems with which participants worked, the results of this study can be generalized to other proposed installations. Our findings could ultimately influence the design of eHealth systems, apps, and interfaces.

    We also believe that we have demonstrated in a small way how to improve the general theory of eHealth implementations in LRCs by assessing a simultaneous combination of opinions of end users about 2 of the main eHealth success factors [13]. By redesigning the model; extending the survey scope; and expanding the participant audience to users, planners, developers, decision makers, and politicians, an evaluation of all 5 success factors [13] in multiple eHealth installations could be undertaken. This would help LRC planners, aided by a modified maturity model approach, to develop an appreciation of the impact of the various factors [35] that may differ among LRCs or among different installations in the same LRC, through the expressed opinions of participants. This would also help to direct strategic investments in eHealth more effectively.

    Strengths and Limitations

    Our eHealth research in LRCs was the first time this approach was used to gather users’ perceptions of how eHealth utilization differed among the countries involved in the study. Acknowledging some limitations in our survey design, it is clear that studies of this nature with revised survey formats could be undertaken on an expanded scale, involving participants with a wider range of eHealth backgrounds. The ultimate gain would be a wider development and understanding of an approach similar to maturity modeling to help the hospital management move ahead with eHealth implementation in an organized and optimal manner.

    We received many interesting and useful comments from participants that we were unable to analyze and include here because of space limitations.

    Limitations included our use of convenience sampling to identify participants, which was not fully representative of each country’s health care workforce. The participant response rate was also considerably less than statistically desirable, and we could have achieved a higher response rate if we had been able to pay participants a nominal fee.

    Acknowledgments

    This research was supported by a joint grant from the McMaster University Faculty of Health Sciences, the DeGroote School of Business, and the McMaster University Research Excellence Fund. The authors wish to thank them for their generous support. We also relied on our support network members to oversee the gathering of data from eHealth users in the four countries in our study. The support network included Adekunle Ajiboye, MEEI, MSc, CEO AAJIMATICS, Hamilton, Canada; Victor Gbenro, BMBCh, PgD, MSc, Dep. Registrar, Medical & Dental Council Nigeria, Abuja, Nigeria; and the four country research coordinators Chythra Rao, MD, Kasturba Medical College, Manipal, India; Ahmed Shaarawy, MD, MSc, MBBcH, Clinical Informatics Director, Welcare Hospital, Giza, Egypt; Wycliffe Ayieko, PgD, University of Nairobi, Kenya; and Nkem Momah, MBBS, MPH, Deputy Director, eGovernment Development & Regulatory Dept, National Information & Technology Development Agency, Abuja, Nigeria. Finally, we want to thank the participants in the 4 countries who completed the questionnaires for this research.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Survey questionnaire.

    DOCX File , 42 KB
    ‎
    Multimedia Appendix 2

    Information, consent, and invitation to participate in the online survey.

    DOCX File , 30 KB
    ‎
    Multimedia Appendix 3

    Details of the survey process.

    DOCX File , 22 KB
    1. Lewis T, Synowiec C, Lagomarsino G, Schweitzer J. E-health in low- and middle-income countries: findings from the center for health market innovations. Bull World Health Org 2012 May 1;90(5):332-340. [CrossRef]
    2. Granja C, Janssen W, Johansen MA. Factors Determining the Success and Failure of eHealth Interventions: Systematic Review of the Literature. J Med Internet Res 2018 May 01;20(5):e10235 [FREE Full text] [CrossRef] [Medline]
    3. Global Diffusion of EHealth: Making Universal Health Coverage Achievable: Report of the Third Global Survey on EHealth. Geneva, Switzerland: World Health Organization; 2016.
    4. Kiberu VM, Mars M, Scott RE. Barriers and opportunities to implementation of sustainable e-Health programmes in Uganda: A literature review. Afr J Prim Health Care Fam Med 2017 May 29;9(1):e1-e10 [FREE Full text] [CrossRef] [Medline]
    5. Mauco KL, Scott RE, Mars M. Critical analysis of e-health readiness assessment frameworks: suitability for application in developing countries. J Telemed Telecare 2016 Dec 23;24(2):110-117. [Medline]
    6. Omotosho A, Ayegba P, Emuoyibofarhe J, Meinel C. Current state of ICT in healthcare delivery in developing countries. Int J Onl Eng 2019 May 14;15(08):91. [CrossRef]
    7. Zayyad M, Toycan M. Factors affecting sustainable adoption of e-health technology in developing countries: an exploratory survey of Nigerian hospitals from the perspective of healthcare professionals. PeerJ 2018;6:e4436 [FREE Full text] [CrossRef] [Medline]
    8. Venkatesh V, Rai A, Sykes TA, Aljafari R. Combating infant mortality in rural India: evidence from a field study of ehealth kiosk implementations. MISQ 2016 Feb 2;40(2):353-380. [CrossRef]
    9. Obasola OI, Mabawonku IM. Mothers' perception of maternal and child health information disseminated via different modes of ICT in Nigeria. Health Info Libr J 2018 Dec 24;35(4):309-318 [FREE Full text] [CrossRef] [Medline]
    10. Muinga N, Magare S, Monda J, Kamau O, Houston S, Fraser H, et al. Implementing an open source electronic health record system in Kenyan health care facilities: case study. JMIR Med Inform 2018 Apr 18;6(2):e22 [FREE Full text] [CrossRef] [Medline]
    11. Badran MF. eHealth in Egypt: the demand-side perspective of implementing electronic health records. Telecommunications Policy 2019 Jul;43(6):576-594. [CrossRef]
    12. Broens TH, Huis in't Veld RM, Vollenbroek-Hutten MM, Hermens HJ, van Halteren AT, Nieuwenhuis LJ. Determinants of successful telemedicine implementations: a literature study. J Telemed Telecare 2007;13(6):303-309. [CrossRef] [Medline]
    13. Ag Ahmed MA, Gagnon M, Hamelin-Brabant L, Mbemba GI, Alami H. A mixed methods systematic review of success factors of mhealth and telehealth for maternal health in Sub-Saharan Africa. Mhealth 2017;3:22 [FREE Full text] [CrossRef] [Medline]
    14. Petter S, DeLone W, McLean ER. Information systems success: the quest for the independent variables. J Manag Inf Syst 2014 Dec 8;29(4):7-62. [CrossRef]
    15. Goodhue D, Thompson R. Task tchnology fit and individual performance. MIS Q 1995;27(3):213-236.
    16. Mettler T. Anticipating mismatches of HIT investments: developing a viability-fit model for e-health services. Int J Med Inform 2016 Jan;85(1):104-115. [CrossRef] [Medline]
    17. Fanta GB, Pretorius L. A conceptual framework for sustainable ehealth implementation in resource-constrained settings. South African J Indust Eng 2018 Nov;29(3):132-147. [CrossRef]
    18. Hoque MR, Bao Y, Sorwar G. Investigating factors influencing the adoption of e-Health in developing countries: a patient's perspective. Inform Health Soc Care 2017 Jan 11;42(1):1-17. [CrossRef] [Medline]
    19. Adu EK, Todorova N, Mills A. Do Individuals in Developing Countries Care about Personal Health Information Privacy? An Empirical Investigation. AIS E-Library. 2020.   URL: https://aisel.aisnet.org/confirm2019/16 [accessed 2021-06-02]
    20. Wilkowska W, Ziefle M. Privacy and data security in E-health: requirements from the user's perspective. Health Informatics J 2012 Sep;18(3):191-201 [FREE Full text] [CrossRef] [Medline]
    21. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989 Sep;13(3):319. [CrossRef]
    22. Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health 2014 Feb 21;14:188 [FREE Full text] [CrossRef] [Medline]
    23. Rezai-Rad M, Vaezi R, Nattagh F. E-health readiness assessment framework in iran. Iran J Public Health 2012;41(10):43-51 [FREE Full text] [Medline]
    24. Meet the Software That Powers More Than 1 Billion Surveys Every Year. Qualtrics. 2020.   URL: https://www.qualtrics.com/uk/core-xm/survey-software/ [accessed 2021-05-29]
    25. SmartPLS. SmartPLS is here. Smart PLS. 2020.   URL: https://www.smartpls.com/ [accessed 2021-05-21]
    26. Hair JF, Sarstedt M, Ringle CM, Mena JA. An assessment of the use of partial least squares structural equation modeling in marketing research. J Acad Mark Sci 2011 Jun 7;40(3):414-433. [CrossRef]
    27. Henseler J, Ringle CM, Sarstedt M. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 2014 Aug 22;43(1):115-135. [CrossRef]
    28. Wong K. Estimating complex models using higher order construct modeling (HCM). In: Mastering Partial Least Squares Structural Equation Modeling (PLS-SEM) With Smartpls in 38 Hours. Bloomington, Indiana: Universe; 2019:87-106.
    29. Pankomera R, van Greunen D. Privacy and Security Issues for a Patient-centric Approach in Public Healthcare in a Resource Constrained Setting. In: Africa Week Conference. 2016 Presented at: AWC'16; May 6-9, 2016; Durban, South Africa. [CrossRef]
    30. Isabalija SR, Mbarika V, Kituyi GM. A framework for sustainable implementation of e-medicine in transitioning countries. Int J Telemed Appl 2013;2013:615617 [FREE Full text] [CrossRef] [Medline]
    31. Siribaddana P, Hewapathirana R, Jayatilleke AU, Sahay S, Dissanayake VH. Strengthening health systems through informatics capacity development among doctors in low-resource contexts: the Sri Lankan experience. South East Asia J Public Health 2019 Sep;8(2):87-94 [FREE Full text] [CrossRef] [Medline]
    32. Mauco KL, Scott RE, Mars M. Validation of an e-health readiness assessment framework for developing countries. BMC Health Serv Res 2020 Jun 23;20(1):575 [FREE Full text] [CrossRef] [Medline]
    33. Ranking Web of World Hospitals. Webometrics. 2020.   URL: https://hospitals.webometrics.info/en/Methodology [accessed 2021-05-21]
    34. Electronic Medical Record Adoption Model (EMRAM). HIMSS Analytics. 2020.   URL: https://www.himssanalytics.org/emram [accessed 2021-05-21]
    35. Luna D, Almerares A, Mayan JC, González Bernaldo de Quirós F, Otero C. Health Informatics in Developing Countries: Going beyond Pilot Practices to Sustainable Implementations: A Review of the Current Challenges. Healthc Inform Res 2014 Jan;20(1):3-10 [FREE Full text] [CrossRef] [Medline]

    ‎
    AVE: average variance extracted
    EMRAM: Electronic Medical Record Adoption Model
    LRC: low-resource country

    Edited by R Kukafka, G Eysenbach; submitted 20.08.20; peer-reviewed by H Hochheiser, K Yin; comments to author 26.09.20; revised version received 01.11.20; accepted 15.05.21; published 18.06.21

    Copyright

    ©Norman Archer, Cynthia Lokker, Maryam Ghasemaghaei, Deborah DiLiberto. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.06.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.