Development and Evaluation of eHealth Services Regarding Accessibility: Scoping Literature Review

Introduction

Accessibility is acknowledged as a key to inclusion in the Convention of Rights for People with Disabilities [1]. Accessibility is defined as follows: “Extent to which products, systems, services, environments, and facilities can be used by people from a population with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use” [2].

The World Wide Web Consortium (W3C) published the first version of the Web Content Accessibility Guidelines (WCAG) in 1999 as a recommendation to provide accessible websites [3]. The World Health Organization recognizes that digitalization of health care can facilitate universal health coverage and points out that eHealth should be developed with accessibility in mind and designed to propel inclusiveness [4].

In 2016, the European Union Web Accessibility Directive mandated all European Union member states to introduce legislation on web accessibility in national legislation. The legislation regulates member states to comply with the European Standards (EN) 301549 standard [5], which builds on the technical standard WCAG 2.1 AA [6]. Similar legislation can be found in other parts of the world, for example, United States [7], New Zeeland [8], and Australia [9]. There is a large body of knowledge showing that web pages and web services most often do not comply with WCAG [10-14]. People with impairments report difficulties in using general functions on the internet, such as navigation, passwords, and services, compared with the general population [15]. The term impairment relates to problems in body function or structure, such as a significant deviation or loss [16]. Studies on accessibility in eHealth services have shown that the services have accessibility errors [17-19]. Inaccessible design can create exclusion from eHealth and cause disability among people with impairments [15].

The importance of accessible eHealth services has become particularly evident during the COVID-19 pandemic, and it has been suggested that eHealth contributes to reducing the spread of COVID-19 [20]. eHealth is defined as the use of information and communication technologies to exchange information digitally between a health care provider and a patient in order to achieve and maintain health [21]. The technical solutions that provide communication between a patient and a health care provider are often complex systems affected by legislation and regulations on issues, such as privacy, security, identification, internet-related infrastructure, interoperability, information sharing, accuracy, and accessibility. Despite standards and guidelines to improve digital accessibility and legislation to regulate the use of these guidelines, eHealth services are still perceived as inaccessible by users [15,22]. Therefore, it is important to investigate how research on eHealth services has included digital accessibility in the development and evaluation of eHealth services.

A scoping literature review is suitable to summarize findings in cases where the body of knowledge is heterogeneous in methods or disciplines [23]. A scoping review is also valuable to determine the extent to which and the way in which research has been conducted [24,25]. A framework for scoping reviews, developed by Arksey and O’Malley [26], suggests that scoping reviews can include relevant literature regardless of study design. Thus, a scoping literature review was considered suitable for this study since research on eHealth is conducted in several disciplines using different methods. Moreover, a scoping review was considered suitable to map how research is performed around the key concept of accessibility regardless of study design.

The aim of this scoping literature review was to investigate how eHealth services have been developed and evaluated regarding accessibility for people with impairments.

Methods

Search Strategy and Inclusion Criteria

The design of this scoping review was in line with Arksey and O’Malley’s framework for scoping studies following the 5-stage process: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting data; and (5) collating, summarizing, and reporting results [26]. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist and explanations [23] were used in the process (Multimedia Appendix 1). An a priori review protocol for the research group with inclusion and exclusion criteria was generated in line with the framework.

The research questions were as follows:

• How is digital accessibility addressed when eHealth services are developed and evaluated?
• Are the WCAG, other accessibility guidelines, or other means used to address digital accessibility in the development and evaluation of eHealth services?

Inclusion Criteria

Eligible publications were scientific peer-reviewed journal articles and conference papers published at any time and written in English or Swedish. The inclusion criteria for the search strategy were structured according to the PICO model [27]. PICO is an acronym for the 4 elements of Population/Problem, Intervention, Comparison, and Outcome that need to be described to formulate the research question and set the inclusion criteria. The criteria were as follows:

• Population/Problem: Digital accessibility for users with impairment
• Intervention: Health care delivered by any digital solution
• Comparison: Not applicable
• Outcome: Use of and adherence to (1) WCAG (any version), (2) other accessibility guidelines, and (3) other means, for designing or evaluating accessibility in eHealth services

Exclusion Criteria

Digital solutions that only monitored health without providing for health care and only monitored self-care were excluded, since the aim was to investigate complex systems with interaction or information exchange between patients and health care. For the same reason, websites that provided health information to the public were excluded. Literature written in languages other than English or Swedish was excluded. Literature reviews were excluded to avoid multiple data reporting.

Information Sources, Literature Screening, and Selection

Since eHealth includes both medical science and information technology, databases from both areas were chosen. The research strategy was planned by the first (MJ), second (SJ), and last (CG) authors. Librarians were consulted in the choice of databases. There is a lack of consensus regarding the terms used for digital accessibility [28]. Several different terms and spellings are used regarding eHealth, and the search strategy included different spellings. Universal design and accessibility are broad terms also used in the context of the physical environment. A Boolean search was constructed with the decided MeSH (Medical Subject Headings) terms and other terms related to eHealth and terms related to accessibility. When the number of hits exceeded 5000, the hits were combined with other terms to narrow the result. Table 1 describes the search terms and search process exemplified with the search in PubMed. Search queries for Scopus, IEEE, and Web of Science are presented in Multimedia Appendix 2.

The PubMed, Scopus, IEEE, and Web of Science databases were used. The database search was conducted by the first author in October 2020. The search result was exported to the Endnote reference program and duplicates were removed. Duplicates not detected by the “find duplicates function” were manually removed. Included references were exported to Rayyan, a collaborative online system for literature reviews [29]. Rayyan allows blinded individual decision-making, which can be unblinded for consensus discussions in the research group. All authors participated in reviewing the search results. Altogether, the research group represents experience from the areas of medical science, human-computer interaction, accessibility, and computer science. The review process started with establishing common ground for individual decision-making through an iterative process with a small number of publications using blinded reviews, and then discussing differences in assessment within the whole group. This step was iterated 3 times. Then, every publication was assessed for the eligibility criteria based on the titles and abstracts by 2 authors (MJ, SJ, DH, JG, and CG) independently. Each pair of authors resolved differences in assessment by consensus. Moreover, full-text reading and assessment of the eligibility criteria were performed by 2 authors (MJ, SJ, DH, JG, and CG) independently and then discussed for consensus. Publications with any disagreement in the pairs were assessed by the whole research group. An update of the search was conducted by the first author in June 2022, repeating the search process for publications published from year 2021 to 2022 (IEEE and Scopus) or between October 31, 2020, and June 20, 2022 (Web of Science and PubMed). After duplicates were removed from the second search, the publications were assessed for eligibility by the authors using the same procedure as the first search.

A data charting form was developed by the first (MJ) and last (CG) authors and discussed with the second (SJ) and fourth (JG) authors. The first author was responsible for extracting data from the included publications, and the first and last authors discussed and updated the data chart in an iterative process. The extracted data consisted of article characteristics (author, year, country, characteristics of the study population, and description of the eHealth service) and summarized data regarding the 2 research questions. The first author interpreted the meaning of the term “accessibility” by looking for a definition or explanation of accessibility. When accessibility was mentioned without definition or explanation, the first author interpreted the meaning of the term from the context. The extracted data were verified by all authors.

Results

After duplicates were removed, a total of 6911 publications were identified in the first search. The second search resulted in 569 publications. One additional publication was found through a search in one publication’s reference list and was included. A total of 131 publications remained for full-text assessment. Of those, 116 publications were excluded since they did not meet the eligibility criteria for the following reasons: (1) wrong population (ie, patient perspective was missing; n=16); (2) wrong intervention (ie, eHealth service did not include exchange of information or interaction between a patient and a health care provider, included self-care only, or a health care provider was missing; n=31); (3) wrong outcome (ie, accessibility was not addressed and no accessibility guidelines or other means were used; n=59); (4) wrong publication type (ie, not a scientific publication; n=2); (5) wrong language (ie, languages other than English or Swedish; n=4); (6) wrong study design (n=3); and (7) duplicate (n=1). Fifteen publications involving studies of 12 eHealth services were included in the study. Figure 1 provides a flow diagram of the screening process.

Table 2 provides information on the year of publication, place of origin, target population, target eHealth service, study design, methods, and main findings related to the research questions of the included publications. The publications were published from the year 2013 to 2022. All studies were conducted in Global North countries, except for 1 in Ecuador [30-32] and 1 in Indonesia [33]. The eHealth services in the included studies targeted different types of patients or diagnoses: hip arthroplasty surgery [30-32], chronic kidney disease [34], depression [35,36], intellectual disability [37,38], dexterity impairments [38,39], older adults with functional limitations [33,40], acquired brain injury [41], multiple sclerosis [42], children with long-term illness [43], and heart failure and chronic obstructive pulmonary disease [44]. The eHealth interventions in the studies consisted of rehabilitation after surgery [30-32], self-monitoring at home [34,40], mental health programs [35,36], supporting alternative communication [37], symptom reporting [43], facilitating appointments [33,38,41,44], self-management regimens [38,39,41,44], and a precision medicine tool [42].

In 3 publications, authors defined accessibility as how people can access and use systems regardless of abilities [30,31,35]. In 1 study, accessibility was defined as when a system is able to adapt itself to the preferences and characteristics of the user [34]. Two of the studies addressed accessibility as the usability of a product, service, environment, or facility by people with the widest range of capabilities as defined by the International Organization for Standardization (ISO) [32], and as universal design and user friendliness by referring to WCAG 2.0 [36]. One of the publications explained accessibility as when a goal can be easily achieved or accessed according to the person’s needs and the ease of using the service in a safe, comfortable, and independent way [33]. Eight studies did not provide a definition or explanation of accessibility. In those studies, the term accessibility was used in relation to the following: ease of use [38,42]; consistent design and instructive guidance explaining that cognitive, motivational, physical ability, and perception barriers influence usability [44]; language as an accessibility barrier [37]; simplicity of app design [41]; and mention as an aspect of familiarity of the terminology in technology [40]. The term accessibility was also used to indicate gaining access to something (ie, health care, login, and devices) [34,37,41,42].

Studies of three eHealth services, presented in 5 publications, used the WCAG to evaluate accessibility when developing eHealth services [30-33,36]. One publication used Web Accessibility Initiative (WAI) [30] and another used Web Accessibility Initiative: Ageing Education and Harmonization (WAI-AGE) [32] to guide the development and evaluation. One of the publications [34] used the Spanish Association for Standardization (UNE) 139803 standard, which is inspired by WCAG 2.0 together with the ISO 29138, ISO 9241, and ISO/International Electrotechnical Commission (IEC) 30071-1 standards.

Eight studies used other means than formal accessibility guidelines and standards to inform about or evaluate accessibility when developing eHealth services. These other means were as follows: text-level grading [35], literature review to inform about accessible text [40], user testing with a target group [33-35,38-40,44], user-centered design workshops with a target group of patients [37] or health professionals [43], focus groups with health professionals to gather meaningful requirements [43], focus groups with a target group of patients and their relatives to gather opinions on app content and design [41], interviews with a target group of patients regarding opinions and preferences on how and what to communicate within the eHealth service [43], a comparative analysis of existing technical solutions to inform about useful requirements [42], and the MOLD-US framework to assess usability issues [44].

‎

Discussion

Principal Results

In the studies included in this scoping literature review, digital accessibility in eHealth services was addressed in several different ways, mostly starting with the user’s abilities, and also considering the system’s potential to adapt to the user. Only 2 of the 15 publications used a formal definition of accessibility with references to the ISO standard or WCAG, and 6 of the 15 publications targeting 4 eHealth services used a web accessibility guideline or standard in the development or evaluation of eHealth services.

In many studies, the term usability was used as an equivalent to accessibility. For example, a publication [36] explained the WCAG accessibility guideline as a standard for evaluating the usability of software systems, which takes into account new technologies, different user agents, and universal design to a sufficient degree [36]. Awareness of the presence of specific and precise definitions of accessibility seems to be low in the field of eHealth development. It seems that the typical development process of eHealth starts from a usability perspective and then subsequently provides some extra attention to a very specific target group (ie, people with a specific diagnosis). To create a holistic view on accessibility, the W3C provides guidance on how to combine formal WCAG evaluation in collaboration with users having impairments [45]. When designing for a specific target population, it is important to recognize that people in the target population might also have a range of other issues and impairments that affect interaction with technology and impede accessibility. Therefore, a holistic view is vital, and we argue that it is important to always consider and comply with the WCAG during the development process.

Not taking a holistic approach is erroneous since even if the eHealth service as such targets people with a specific diagnosis, those people might also have a range of other issues and impairments when interacting with technology, which can impede accessibility.

A problem with the lack of consensus on the definition of accessibility is the risk of less accessibility for the target audience [28]. There are several ISO standards targeting accessibility issues, and the concept of accessibility is strongly related to the concept of usability, recognizing that accessibility contributes to achieving usability [46]. However, usability does not automatically cover accessibility. This suggests that it is important to use the formal predefined accessibility terms with reference to the WCAG or ISO standard to avoid confusion and to be clear which kind of accessibility is targeted.

Six publications targeting 4 eHealth services used a web accessibility guideline or standard in the development or evaluation of the eHealth services. It is reasonable to believe that researchers who use the WCAG will also use the term accessibility or WCAG as a keyword. Therefore, it is plausible that our findings reflect how infrequently accessibility guidelines have been used when developing or evaluating eHealth services. The low number indicates that eHealth services may exclude people with impairments. Previous research confirms that accessibility is insufficient in eHealth and argues that the WCAG standards are important but not sufficient to develop accessible eHealth services [47]. However, not using accessibility guidelines or standards and only relying on information from users can increase the risk of missing several aspects of accessibility. The information that users provide during user testing is based on their own experience. Therefore, it is most likely not based on any knowledge about accessibility requirements related to the quality of the code or the technical construction of the eHealth service that is tested. Previous research has shown that developers and user experience (UX) professionals have limited knowledge about accessibility [48,49], implying that relying on the knowledge of the developers and UX designers might increase the risk of missing accessibility aspects. Thus, to accurately address accessibility, it is favorable to combine information from accessibility guidelines, the experiences of users in the target group, and the knowledge of developers and UX designers.

Although the COVID-19 pandemic has drawn attention to the need of offering eHealth services to all people, we only found 3 studies [33,39,43] published after 2020 that addressed accessibility. In the framework for implementation research by Peters et al, it is recommended that the target population be described in sufficient detail [50]. When developing eHealth services, it is important to recognize that patients with a specific diagnosis can also have a range of other impairments. Thus, when describing the target population, it is important to also describe the heterogeneity within the population, addressing all kinds of impairments even though they may not be related to the diagnosis. Designing for the widest range of capabilities should be the goal. Complying with accessibility guidelines increases the inclusion of a wide range of user characteristics and reduces the risk of digital exclusion.

Several Global North countries [6-9] have regulated the level of compliance to the WCAG that public websites and applications must achieve, indicating that research addressing digital accessibility for both existing and new eHealth services will increase in this part of the world. It is notable that we only found 2 studies outside the Global North, although several other countries, for example, India and Taiwan, are listed as having mandatory policies referring to the WCAG [51]. The reasons the WCAG are not used more often in the development of eHealth services, despite being available for more than two decades [11], could be that the guidelines are not well known and that the guidelines and standards are difficult to use.

eHealth is developed within multiple disciplines and some of the disciplines may be less aware of digital accessibility. Medical researchers and developers of eHealth may rely on guidelines and information that do not cover digital accessibility. One example is the widely used Medical Research Council (MRC) guidelines for developing and evaluating complex interventions [52], which do not contain any information about accessibility in eHealth. An update of the MRC guidelines targeting eHealth, with special reference to relevant accessibility guidelines and standards, could strengthen future research and development of accessibility in eHealth interventions.

In what we categorized as “other means,” we found a wide range of methods not using accessibility standards or guidelines for evaluating accessibility in eHealth services. This indicates that either accessibility evaluation is not known or there is a lack of knowledge regarding the differences in usability and accessibility. In some of the studies, usability was used as an equivalent to accessibility, although usability does not automatically cover accessibility. For example, in a publication [35], it was explained that accessibility is the ease with which patients can use the health care service in proportion to their needs, as well as the usability of the actual technology.

Using other means to develop or evaluate eHealth may be a way of including the assessment of cognitive accessibility, which is not well covered in the WCAG. WCAG 2.1 and older versions have been criticized for not covering accessibility issues when users have cognitive disability [53,54]. Hence, to cover all relevant accessibility issues, it is suggested to combine the WCAG with other means. Those other means should be a combination of guidelines for cognitive accessibility, understandable text or content, and user testing with a diverse set of users with impairments.

In summary, the means used for dealing with accessibility in the included publications can be referred to principles (eg, the principles for universal design), guidelines (eg, some kind of heuristics), and standards (eg, WCAG 2.1 AA or HTML5) (Figure 2). Principles provide overall orientation and connect the development of the eHealth service to concepts of fairness, human rights, inclusion, and participation. Guidelines and recommendations are important to create awareness and point out a direction for accessibility in the service. Standards and specifications provide the level of precision needed to ensure actual accessibility in eHealth [55,56]. Most of the publications included in our study did not refer to standards or specifications. However, for many people with impairments, accessibility is in the details, meaning that a high level of precision and compliance to detailed specifications is important to ensure accessibility. For example, the technical construction of an eHealth service needs to be very precise to be interoperable with the assistive technology used by blind people or people who use navigation techniques other than moving a mouse pointer and clicking on objects. Most guidelines and principles do not provide sufficient precision.

If the development of an eHealth service solely relates to principles and guidelines, as for most of the publications included in this scoping review, the approach is still far better than not considering accessibility at all. However, those services will probably not conform to the legally mandated level of technical accessibility stated in the EN 310 549/WCAG 2.1 AA standard or in the ISO 21 801-1 standard for cognitive accessibility. Our findings show that although the publications included in this review often advocated for usability, accessibility was not addressed with the level of precision needed to ensure accessible eHealth. Even studies that claimed to have followed guidelines still only investigated usability measures, leaving accessibility as a neglected issue in studies developing eHealth services.

‎

Strengths and Limitations

A strength of this study is the construction of a thorough search strategy with a review protocol before conducting the literature search, which increased the reproducibility and reliability of the study [57]. The process of setting common ground for individual decision-making, the consensus discussions, and the approach of all researchers reading the included publications in full text strengthen the systematic and reproducible study selection [23]. Another strength of this study is that the research group represented experience from several important disciplines and competences with regard to the development of eHealth services: medical science, human-computer interaction, accessibility, and computer science. This increased the validity of the review process, as the publications assessed in this study came from several scientific disciplines and required in-depth knowledge in the aspects of accessibility in digital technology and health care delivery. These together increase the trustworthiness of the results. All but 2 of the studies in this literature review were conducted in the context of a Global North country. Thus, the findings may have limited generalizability to countries outside the Global North. It is possible that research has been published in languages other than English or Swedish, and thus, there may be studies in other languages that were not included in this study.

Conclusions

Although a clear definition of accessibility can enhance operationalization and thus measurability when evaluating accessibility in eHealth services, the results of this literature review show that accessibility was insufficiently defined in most of the included studies. Further, the results show that accessibility guidelines and standards were used to a very limited extent in the development and evaluation of eHealth services. Guidelines for developing complex interventions that include guidance for accessibility are motivated to ensure that accessibility will be considered systematically in eHealth services.