Background: Accessibility is acknowledged as a key to inclusion in the Convention of Rights for People with Disabilities. An inaccessible design can result in exclusion from eHealth and cause disability among people who have impairments.
Objective: This scoping literature review aimed to investigate how eHealth services have been developed and evaluated regarding accessibility for people with impairments.
Methods: In line with Arksey and O’Malley’s framework for scoping studies and using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR), we conducted a search in 4 databases (PubMed, Scopus, IEEE, and Web of Science) in October 2020 and an update of the search in June 2022. The search strategy was structured according to the PICO model as follows: Population/Problem, digital accessibility for users with impairment; Intervention, health care delivered by any digital solution; Comparison, not applicable; Outcome, use of and adherence to (1) Web Content Accessibility Guidelines (WCAG), (2) other accessibility guidelines, and (3) other means, for designing or evaluating accessibility in eHealth services. A Boolean search was conducted by combining terms related to accessibility and eHealth. All authors participated in screening abstracts according to the eligibility criteria. Each publication, containing a potentially relevant abstract, was read (full text) and assessed for eligibility by 2 authors independently and pairwise. Publications deemed eligible were read by all authors and discussed for consensus.
Results: A total of 8643 publications were identified. After abstract screening, 131 publications remained for full-text reading. Of those, 116 publications were excluded as they did not meet the eligibility criteria. Fifteen publications involving studies of 12 eHealth services were included in the study. Of the 15 publications, 2 provided a definition of accessibility, 5 provided an explanation of accessibility, and 8 did not provide any explanation. Five publications used the WCAG to evaluate accessibility when developing eHealth services. One publication used International Organization for Standardization (ISO) 29138, ISO 2941, and ISO/International Electrotechnical Commission (IEC) 30071-1 standards together with the Spanish Association for Standardization (UNE) 139803 standard. Eleven publications used other means to address accessibility, including text-level grading; literature review about accessibility; user tests, focus groups, interviews, and design workshops with target groups of patients, relatives, and health care professionals; and comparative analysis of existing technical solutions to provide information about useful requirements.
Conclusions: Although a clear definition of accessibility can enhance operationalization and thus measurability when evaluating accessibility in eHealth services, accessibility was insufficiently defined in most of the included studies. Further, accessibility guidelines and standards were used to a very limited extent in the development and evaluation of eHealth services. Guidelines for developing complex interventions that include guidance for accessibility are motivated to ensure that accessibility will be considered systematically in eHealth services.
Accessibility is acknowledged as a key to inclusion in the Convention of Rights for People with Disabilities . Accessibility is defined as follows: “Extent to which products, systems, services, environments, and facilities can be used by people from a population with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use” [ ].
The World Wide Web Consortium (W3C) published the first version of the Web Content Accessibility Guidelines (WCAG) in 1999 as a recommendation to provide accessible websites . The World Health Organization recognizes that digitalization of health care can facilitate universal health coverage and points out that eHealth should be developed with accessibility in mind and designed to propel inclusiveness [ ].
In 2016, the European Union Web Accessibility Directive mandated all European Union member states to introduce legislation on web accessibility in national legislation. The legislation regulates member states to comply with the European Standards (EN) 301549 standard , which builds on the technical standard WCAG 2.1 AA [ ]. Similar legislation can be found in other parts of the world, for example, United States [ ], New Zeeland [ ], and Australia [ ]. There is a large body of knowledge showing that web pages and web services most often do not comply with WCAG [ - ]. People with impairments report difficulties in using general functions on the internet, such as navigation, passwords, and services, compared with the general population [ ]. The term impairment relates to problems in body function or structure, such as a significant deviation or loss [ ]. Studies on accessibility in eHealth services have shown that the services have accessibility errors [ - ]. Inaccessible design can create exclusion from eHealth and cause disability among people with impairments [ ].
The importance of accessible eHealth services has become particularly evident during the COVID-19 pandemic, and it has been suggested that eHealth contributes to reducing the spread of COVID-19 . eHealth is defined as the use of information and communication technologies to exchange information digitally between a health care provider and a patient in order to achieve and maintain health [ ]. The technical solutions that provide communication between a patient and a health care provider are often complex systems affected by legislation and regulations on issues, such as privacy, security, identification, internet-related infrastructure, interoperability, information sharing, accuracy, and accessibility. Despite standards and guidelines to improve digital accessibility and legislation to regulate the use of these guidelines, eHealth services are still perceived as inaccessible by users [ , ]. Therefore, it is important to investigate how research on eHealth services has included digital accessibility in the development and evaluation of eHealth services.
A scoping literature review is suitable to summarize findings in cases where the body of knowledge is heterogeneous in methods or disciplines . A scoping review is also valuable to determine the extent to which and the way in which research has been conducted [ , ]. A framework for scoping reviews, developed by Arksey and O’Malley [ ], suggests that scoping reviews can include relevant literature regardless of study design. Thus, a scoping literature review was considered suitable for this study since research on eHealth is conducted in several disciplines using different methods. Moreover, a scoping review was considered suitable to map how research is performed around the key concept of accessibility regardless of study design.
The aim of this scoping literature review was to investigate how eHealth services have been developed and evaluated regarding accessibility for people with impairments.
Search Strategy and Inclusion Criteria
The design of this scoping review was in line with Arksey and O’Malley’s framework for scoping studies following the 5-stage process: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting data; and (5) collating, summarizing, and reporting results . The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist and explanations [ ] were used in the process ( ). An a priori review protocol for the research group with inclusion and exclusion criteria was generated in line with the framework.
The research questions were as follows:
- How is digital accessibility addressed when eHealth services are developed and evaluated?
- Are the WCAG, other accessibility guidelines, or other means used to address digital accessibility in the development and evaluation of eHealth services?
Eligible publications were scientific peer-reviewed journal articles and conference papers published at any time and written in English or Swedish. The inclusion criteria for the search strategy were structured according to the PICO model . PICO is an acronym for the 4 elements of Population/Problem, Intervention, Comparison, and Outcome that need to be described to formulate the research question and set the inclusion criteria. The criteria were as follows:
- Population/Problem: Digital accessibility for users with impairment
- Intervention: Health care delivered by any digital solution
- Comparison: Not applicable
- Outcome: Use of and adherence to (1) WCAG (any version), (2) other accessibility guidelines, and (3) other means, for designing or evaluating accessibility in eHealth services
Digital solutions that only monitored health without providing for health care and only monitored self-care were excluded, since the aim was to investigate complex systems with interaction or information exchange between patients and health care. For the same reason, websites that provided health information to the public were excluded. Literature written in languages other than English or Swedish was excluded. Literature reviews were excluded to avoid multiple data reporting.
Information Sources, Literature Screening, and Selection
Since eHealth includes both medical science and information technology, databases from both areas were chosen. The research strategy was planned by the first (MJ), second (SJ), and last (CG) authors. Librarians were consulted in the choice of databases. There is a lack of consensus regarding the terms used for digital accessibility . Several different terms and spellings are used regarding eHealth, and the search strategy included different spellings. Universal design and accessibility are broad terms also used in the context of the physical environment. A Boolean search was constructed with the decided MeSH (Medical Subject Headings) terms and other terms related to eHealth and terms related to accessibility. When the number of hits exceeded 5000, the hits were combined with other terms to narrow the result. describes the search terms and search process exemplified with the search in PubMed. Search queries for Scopus, IEEE, and Web of Science are presented in .
|Search number||Search query||Number of hits||Specification of not includeda or includedb|
|1||((((((ehealth) OR (e-health)) OR (e health)) OR (mhealth)) OR (mobile health)) OR (telemedicine)) OR (telerehabilitation)||74,326||Not included|
|2||(((((web accessibility) OR (digital accessibility)) OR (universal design)) OR (wcag)) OR (WCAG)) OR (accessibility guidelines)||54,890||Not included|
|3||(((((((ehealth) OR (e-health)) OR (e health)) OR (mhealth)) OR (mobile health)) OR (telemedicine)) OR (telerehabilitation)) AND (accessibility guidelines)||776||Included|
|4||(((((((ehealth) OR (e-health)) OR (e health)) OR (mhealth)) OR (mobile health)) OR (telemedicine)) OR (telerehabilitation)) AND (digital accessibility)||1666||Included|
|5||(((((((ehealth) OR (e-health)) OR (e health)) OR (mhealth)) OR (mobile health)) OR (telemedicine)) OR (telerehabilitation)) AND (web accessibility)||1692||Included|
|6||(((((((ehealth) OR (e-health)) OR (e health)) OR (mhealth)) OR (mobile health)) OR (telemedicine)) OR (telerehabilitation)) AND (wcag)||21||Included|
|7||(((((((ehealth) OR (e-health)) OR (e health)) OR (mhealth)) OR (mobile health)) OR (telemedicine)) OR (telerehabilitation)) AND (universal design)||113||Included|
aSearches with >5000 hits were considered not included.
bA total of 4268 hits were included (searches 3-7); a total of 3285 hits were finally included (searches 3-7) after removing 983 duplicates.
The PubMed, Scopus, IEEE, and Web of Science databases were used. The database search was conducted by the first author in October 2020. The search result was exported to the Endnote reference program and duplicates were removed. Duplicates not detected by the “find duplicates function” were manually removed. Included references were exported to Rayyan, a collaborative online system for literature reviews . Rayyan allows blinded individual decision-making, which can be unblinded for consensus discussions in the research group. All authors participated in reviewing the search results. Altogether, the research group represents experience from the areas of medical science, human-computer interaction, accessibility, and computer science. The review process started with establishing common ground for individual decision-making through an iterative process with a small number of publications using blinded reviews, and then discussing differences in assessment within the whole group. This step was iterated 3 times. Then, every publication was assessed for the eligibility criteria based on the titles and abstracts by 2 authors (MJ, SJ, DH, JG, and CG) independently. Each pair of authors resolved differences in assessment by consensus. Moreover, full-text reading and assessment of the eligibility criteria were performed by 2 authors (MJ, SJ, DH, JG, and CG) independently and then discussed for consensus. Publications with any disagreement in the pairs were assessed by the whole research group. An update of the search was conducted by the first author in June 2022, repeating the search process for publications published from year 2021 to 2022 (IEEE and Scopus) or between October 31, 2020, and June 20, 2022 (Web of Science and PubMed). After duplicates were removed from the second search, the publications were assessed for eligibility by the authors using the same procedure as the first search.
A data charting form was developed by the first (MJ) and last (CG) authors and discussed with the second (SJ) and fourth (JG) authors. The first author was responsible for extracting data from the included publications, and the first and last authors discussed and updated the data chart in an iterative process. The extracted data consisted of article characteristics (author, year, country, characteristics of the study population, and description of the eHealth service) and summarized data regarding the 2 research questions. The first author interpreted the meaning of the term “accessibility” by looking for a definition or explanation of accessibility. When accessibility was mentioned without definition or explanation, the first author interpreted the meaning of the term from the context. The extracted data were verified by all authors.
After duplicates were removed, a total of 6911 publications were identified in the first search. The second search resulted in 569 publications. One additional publication was found through a search in one publication’s reference list and was included. A total of 131 publications remained for full-text assessment. Of those, 116 publications were excluded since they did not meet the eligibility criteria for the following reasons: (1) wrong population (ie, patient perspective was missing; n=16); (2) wrong intervention (ie, eHealth service did not include exchange of information or interaction between a patient and a health care provider, included self-care only, or a health care provider was missing; n=31); (3) wrong outcome (ie, accessibility was not addressed and no accessibility guidelines or other means were used; n=59); (4) wrong publication type (ie, not a scientific publication; n=2); (5) wrong language (ie, languages other than English or Swedish; n=4); (6) wrong study design (n=3); and (7) duplicate (n=1). Fifteen publications involving studies of 12 eHealth services were included in the study.provides a flow diagram of the screening process.
provides information on the year of publication, place of origin, target population, target eHealth service, study design, methods, and main findings related to the research questions of the included publications. The publications were published from the year 2013 to 2022. All studies were conducted in Global North countries, except for 1 in Ecuador [ - ] and 1 in Indonesia [ ]. The eHealth services in the included studies targeted different types of patients or diagnoses: hip arthroplasty surgery [ - ], chronic kidney disease [ ], depression [ , ], intellectual disability [ , ], dexterity impairments [ , ], older adults with functional limitations [ , ], acquired brain injury [ ], multiple sclerosis [ ], children with long-term illness [ ], and heart failure and chronic obstructive pulmonary disease [ ]. The eHealth interventions in the studies consisted of rehabilitation after surgery [ - ], self-monitoring at home [ , ], mental health programs [ , ], supporting alternative communication [ ], symptom reporting [ ], facilitating appointments [ , , , ], self-management regimens [ , , , ], and a precision medicine tool [ ].
In 3 publications, authors defined accessibility as how people can access and use systems regardless of abilities [, , ]. In 1 study, accessibility was defined as when a system is able to adapt itself to the preferences and characteristics of the user [ ]. Two of the studies addressed accessibility as the usability of a product, service, environment, or facility by people with the widest range of capabilities as defined by the International Organization for Standardization (ISO) [ ], and as universal design and user friendliness by referring to WCAG 2.0 [ ]. One of the publications explained accessibility as when a goal can be easily achieved or accessed according to the person’s needs and the ease of using the service in a safe, comfortable, and independent way [ ]. Eight studies did not provide a definition or explanation of accessibility. In those studies, the term accessibility was used in relation to the following: ease of use [ , ]; consistent design and instructive guidance explaining that cognitive, motivational, physical ability, and perception barriers influence usability [ ]; language as an accessibility barrier [ ]; simplicity of app design [ ]; and mention as an aspect of familiarity of the terminology in technology [ ]. The term accessibility was also used to indicate gaining access to something (ie, health care, login, and devices) [ , , , ].
Studies of three eHealth services, presented in 5 publications, used the WCAG to evaluate accessibility when developing eHealth services [- , ]. One publication used Web Accessibility Initiative (WAI) [ ] and another used Web Accessibility Initiative: Ageing Education and Harmonization (WAI-AGE) [ ] to guide the development and evaluation. One of the publications [ ] used the Spanish Association for Standardization (UNE) 139803 standard, which is inspired by WCAG 2.0 together with the ISO 29138, ISO 9241, and ISO/International Electrotechnical Commission (IEC) 30071-1 standards.
Eight studies used other means than formal accessibility guidelines and standards to inform about or evaluate accessibility when developing eHealth services. These other means were as follows: text-level grading , literature review to inform about accessible text [ ], user testing with a target group [ - , - , ], user-centered design workshops with a target group of patients [ ] or health professionals [ ], focus groups with health professionals to gather meaningful requirements [ ], focus groups with a target group of patients and their relatives to gather opinions on app content and design [ ], interviews with a target group of patients regarding opinions and preferences on how and what to communicate within the eHealth service [ ], a comparative analysis of existing technical solutions to inform about useful requirements [ ], and the MOLD-US framework to assess usability issues [ ].
|Author, year, country||Development/evaluation||Study population||eHealth service||How was digital accessibility addressed?||What accessibility guidelines or other means were used?|
|Acosta-Vargas et al, 2018, Ecuador ||Evaluation||Elderly hip arthroplasty surgery patients||Telerehabilitation prototype platform||Guidelines and standards:|
|Acosta-Vargas et al, 2019, Ecuador ||Evaluation||Elderly hip arthroplasty surgery patients||Telerehabilitation prototype platform||Guidelines and standards:|
|Calle-Jimenez et al, 2019, Ecuador ||Development and evaluation||Elderly hip arthroplasty surgery patients||Telerehabilitation platform for postsurgical rehabilitation||Guidelines and standards:|
|Calvillo-Arbizu, 2019, Spain ||Development and evaluation||Patients with chronic kidney disease||eHealth app (AppNephro) for self-monitoring at home||Guidelines and standards:|
|Clunne et al, 2018, Australia ||Evaluation||Post stroke aphasia patients with depression||Eight mental eHealth programs delivered through a website||Other means|
|Gibson et al, 2020, United Kingdom ||Development and evaluation||People with mild intellectual disability (ID)||Augmented and alternative communication tools used in contact with primary health care||Other means|
|Kascak et al, 2013, United States ||Development and evaluation||Older adults with functional limitations relating to age||Remote patient monitoring (RPM) devices, and case study on the BL Healthcare Access Tablet||Other means|
|Osborne et al, 2020, United States ||Development||Persons with acquired brain injury (ABI)||Mobile health app||Other means|
|Schleimer et al, 2020, United States ||Development||Adult patients with multiple sclerosis (MS)||Open MS BioScreen||Other means|
|Wildenbos et al, 2019, Netherlands ||Evaluation||Patients with heart failure or chronic obstructive pulmonary disease (COPD)||App 1 facilitating hospital appointment attendance and App 2 for self-monitoring||Other means:|
|Yogarajah et al, 2020, Norway and Sweden ||Evaluation||Adults with major depression and stable medication||Five applications providing internet-delivered cognitive behavioral therapy (CBT): eMeistring, Assistert Selvhjelp, MoodGym (2.0), Psyktools, Internetpsykiatri||WCAG 2.0|
|Yu et al, 2019, United States ||Development and evaluation||Users with intellectual disabilities or dexterity impairment||iMHere, support for self-management regimens||Other means|
|Syahrina et al, 2021, Indonesia ||Evaluation||Older adults with functional limitations relating to age||Halodoc, eHealth app with several features. Evaluation on consultation with a doctor through chat, booking of appointments to the hospital, and purchasing of medicine. The target group includes all Indonesian people.||Guidelines and standards:|
|Chowdhary et al, 2022, United States ||Development and evaluation||People with disabilities and dexterity impairments||iMHere 1.0 eHealth app, modules MyMeds and SkinCare||Other means|
|Wiljén et al, 2022, Sweden ||Development and evaluation||Children with long-term illness||eHealth app Pictorial Support in Person-Centered Care for Children called PicPecc. App for self-report and managing symptoms at a hospital or at home.||Other means|
In the studies included in this scoping literature review, digital accessibility in eHealth services was addressed in several different ways, mostly starting with the user’s abilities, and also considering the system’s potential to adapt to the user. Only 2 of the 15 publications used a formal definition of accessibility with references to the ISO standard or WCAG, and 6 of the 15 publications targeting 4 eHealth services used a web accessibility guideline or standard in the development or evaluation of eHealth services.
In many studies, the term usability was used as an equivalent to accessibility. For example, a publication  explained the WCAG accessibility guideline as a standard for evaluating the usability of software systems, which takes into account new technologies, different user agents, and universal design to a sufficient degree [ ]. Awareness of the presence of specific and precise definitions of accessibility seems to be low in the field of eHealth development. It seems that the typical development process of eHealth starts from a usability perspective and then subsequently provides some extra attention to a very specific target group (ie, people with a specific diagnosis). To create a holistic view on accessibility, the W3C provides guidance on how to combine formal WCAG evaluation in collaboration with users having impairments [ ]. When designing for a specific target population, it is important to recognize that people in the target population might also have a range of other issues and impairments that affect interaction with technology and impede accessibility. Therefore, a holistic view is vital, and we argue that it is important to always consider and comply with the WCAG during the development process.
Not taking a holistic approach is erroneous since even if the eHealth service as such targets people with a specific diagnosis, those people might also have a range of other issues and impairments when interacting with technology, which can impede accessibility.
A problem with the lack of consensus on the definition of accessibility is the risk of less accessibility for the target audience . There are several ISO standards targeting accessibility issues, and the concept of accessibility is strongly related to the concept of usability, recognizing that accessibility contributes to achieving usability [ ]. However, usability does not automatically cover accessibility. This suggests that it is important to use the formal predefined accessibility terms with reference to the WCAG or ISO standard to avoid confusion and to be clear which kind of accessibility is targeted.
Six publications targeting 4 eHealth services used a web accessibility guideline or standard in the development or evaluation of the eHealth services. It is reasonable to believe that researchers who use the WCAG will also use the term accessibility or WCAG as a keyword. Therefore, it is plausible that our findings reflect how infrequently accessibility guidelines have been used when developing or evaluating eHealth services. The low number indicates that eHealth services may exclude people with impairments. Previous research confirms that accessibility is insufficient in eHealth and argues that the WCAG standards are important but not sufficient to develop accessible eHealth services . However, not using accessibility guidelines or standards and only relying on information from users can increase the risk of missing several aspects of accessibility. The information that users provide during user testing is based on their own experience. Therefore, it is most likely not based on any knowledge about accessibility requirements related to the quality of the code or the technical construction of the eHealth service that is tested. Previous research has shown that developers and user experience (UX) professionals have limited knowledge about accessibility [ , ], implying that relying on the knowledge of the developers and UX designers might increase the risk of missing accessibility aspects. Thus, to accurately address accessibility, it is favorable to combine information from accessibility guidelines, the experiences of users in the target group, and the knowledge of developers and UX designers.
Although the COVID-19 pandemic has drawn attention to the need of offering eHealth services to all people, we only found 3 studies [, , ] published after 2020 that addressed accessibility. In the framework for implementation research by Peters et al, it is recommended that the target population be described in sufficient detail [ ]. When developing eHealth services, it is important to recognize that patients with a specific diagnosis can also have a range of other impairments. Thus, when describing the target population, it is important to also describe the heterogeneity within the population, addressing all kinds of impairments even though they may not be related to the diagnosis. Designing for the widest range of capabilities should be the goal. Complying with accessibility guidelines increases the inclusion of a wide range of user characteristics and reduces the risk of digital exclusion.
Several Global North countries [- ] have regulated the level of compliance to the WCAG that public websites and applications must achieve, indicating that research addressing digital accessibility for both existing and new eHealth services will increase in this part of the world. It is notable that we only found 2 studies outside the Global North, although several other countries, for example, India and Taiwan, are listed as having mandatory policies referring to the WCAG [ ]. The reasons the WCAG are not used more often in the development of eHealth services, despite being available for more than two decades [ ], could be that the guidelines are not well known and that the guidelines and standards are difficult to use.
eHealth is developed within multiple disciplines and some of the disciplines may be less aware of digital accessibility. Medical researchers and developers of eHealth may rely on guidelines and information that do not cover digital accessibility. One example is the widely used Medical Research Council (MRC) guidelines for developing and evaluating complex interventions , which do not contain any information about accessibility in eHealth. An update of the MRC guidelines targeting eHealth, with special reference to relevant accessibility guidelines and standards, could strengthen future research and development of accessibility in eHealth interventions.
In what we categorized as “other means,” we found a wide range of methods not using accessibility standards or guidelines for evaluating accessibility in eHealth services. This indicates that either accessibility evaluation is not known or there is a lack of knowledge regarding the differences in usability and accessibility. In some of the studies, usability was used as an equivalent to accessibility, although usability does not automatically cover accessibility. For example, in a publication , it was explained that accessibility is the ease with which patients can use the health care service in proportion to their needs, as well as the usability of the actual technology.
Using other means to develop or evaluate eHealth may be a way of including the assessment of cognitive accessibility, which is not well covered in the WCAG. WCAG 2.1 and older versions have been criticized for not covering accessibility issues when users have cognitive disability [, ]. Hence, to cover all relevant accessibility issues, it is suggested to combine the WCAG with other means. Those other means should be a combination of guidelines for cognitive accessibility, understandable text or content, and user testing with a diverse set of users with impairments.
In summary, the means used for dealing with accessibility in the included publications can be referred to principles (eg, the principles for universal design), guidelines (eg, some kind of heuristics), and standards (eg, WCAG 2.1 AA or HTML5) (). Principles provide overall orientation and connect the development of the eHealth service to concepts of fairness, human rights, inclusion, and participation. Guidelines and recommendations are important to create awareness and point out a direction for accessibility in the service. Standards and specifications provide the level of precision needed to ensure actual accessibility in eHealth [ , ]. Most of the publications included in our study did not refer to standards or specifications. However, for many people with impairments, accessibility is in the details, meaning that a high level of precision and compliance to detailed specifications is important to ensure accessibility. For example, the technical construction of an eHealth service needs to be very precise to be interoperable with the assistive technology used by blind people or people who use navigation techniques other than moving a mouse pointer and clicking on objects. Most guidelines and principles do not provide sufficient precision.
If the development of an eHealth service solely relates to principles and guidelines, as for most of the publications included in this scoping review, the approach is still far better than not considering accessibility at all. However, those services will probably not conform to the legally mandated level of technical accessibility stated in the EN 310 549/WCAG 2.1 AA standard or in the ISO 21 801-1 standard for cognitive accessibility. Our findings show that although the publications included in this review often advocated for usability, accessibility was not addressed with the level of precision needed to ensure accessible eHealth. Even studies that claimed to have followed guidelines still only investigated usability measures, leaving accessibility as a neglected issue in studies developing eHealth services.
Strengths and Limitations
A strength of this study is the construction of a thorough search strategy with a review protocol before conducting the literature search, which increased the reproducibility and reliability of the study . The process of setting common ground for individual decision-making, the consensus discussions, and the approach of all researchers reading the included publications in full text strengthen the systematic and reproducible study selection [ ]. Another strength of this study is that the research group represented experience from several important disciplines and competences with regard to the development of eHealth services: medical science, human-computer interaction, accessibility, and computer science. This increased the validity of the review process, as the publications assessed in this study came from several scientific disciplines and required in-depth knowledge in the aspects of accessibility in digital technology and health care delivery. These together increase the trustworthiness of the results. All but 2 of the studies in this literature review were conducted in the context of a Global North country. Thus, the findings may have limited generalizability to countries outside the Global North. It is possible that research has been published in languages other than English or Swedish, and thus, there may be studies in other languages that were not included in this study.
Although a clear definition of accessibility can enhance operationalization and thus measurability when evaluating accessibility in eHealth services, the results of this literature review show that accessibility was insufficiently defined in most of the included studies. Further, the results show that accessibility guidelines and standards were used to a very limited extent in the development and evaluation of eHealth services. Guidelines for developing complex interventions that include guidance for accessibility are motivated to ensure that accessibility will be considered systematically in eHealth services.
The study was funded by grants from FORTE - The Swedish Research Council for Health, Working Life and Welfare (grant number 2018-01806) and Habilitation & Health, the Region Västra Götaland. The funding bodies had no role in the undertaking of the study.
All authors contributed to the study. CG, SJ, and MJ participated in the conception and design of the study. MJ conducted the initial literature search. All authors assessed studies for eligibility by reading the abstracts, and all authors evaluated the full text of publications. MJ and CG undertook data extraction. All authors participated in the analysis and interpretation of data by discussing and revising the data extraction. MJ and CG led the drafting of the manuscript, and all authors contributed to critical revisions of the manuscript. All authors approved the final version of the manuscript.
Conflicts of Interest
PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.PDF File (Adobe PDF File), 548 KB
Final search strategy for PubMed, Scopus, IEEE, and Web of Science.PDF File (Adobe PDF File), 121 KB
- Convention on the Rights of Persons with Disabilities (CRPD). United Nations. URL: https://social.desa.un.org/issues/disability/crpd/convention-on-the-rights-of-persons-with-disabilities-crpd [accessed 2023-07-18]
- ISO 26800:2011 Ergonomics — General approach, principles and concepts. International Organization for Standardization. URL: https://www.iso.org/standard/42885.html [accessed 2023-07-18]
- Web Content Accessibility Guidelines (WCAG) 2.1. World Wide Web Consortium. URL: https://www.w3.org/TR/2018/REC-WCAG21-20180605/ [accessed 2023-07-18]
- Global strategy on digital health 2020-2025. World Health Organization. URL: https://apps.who.int/iris/handle/10665/344249 [accessed 2023-07-18]
- EN 301 549 V2.1.2 (2018-08) Accessibility requirements for ICT products and services. European Telecommunications Standards Institute: ETSI. 2018. URL: https://www.etsi.org/deliver/etsi_en/301500_301599/301549/02.01.02_60/en_301549v020102p.pdf [accessed 2023-07-18]
- Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. Official Journal of the European Union. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2016.327.01.0001.01.ENG&toc=OJ:L:2016:327:TOC [accessed 2023-07-18]
- The US General Services Administration. URL: https://www.section508.gov/manage/laws-and-policies/ [accessed 2023-07-18]
- Accessibility. New Zealand Government. URL: https://www.digital.govt.nz/standards-and-guidance/design-and-ux/accessibility/ [accessed 2023-07-18]
- Make it accessible. Australian Government Digital Transformation Agency. URL: https://www.dta.gov.au/help-and-advice/digital-service-standard/digital-service-standard-criteria/9-make-it-accessible [accessed 2023-07-18]
- Campoverde-Molina M, Lujan-Mora S, Garcia LV. Empirical Studies on Web Accessibility of Educational Websites: A Systematic Literature Review. IEEE Access 2020;8:91676-91700 [CrossRef]
- Acosta-Vargas P, Luján-Mora S, Salvador-Ullauri L. Quality evaluation of government websites. 2017 Presented at: Fourth International Conference on eDemocracy & eGovernment (ICEDEG); April 19-21, 2017; Quito, Ecuador [CrossRef]
- Ismail A, Kuppusamy KS. Accessibility analysis of North Eastern India Region websites for persons with disabilities. 2016 Presented at: International Conference on Accessibility to Digital World (ICADW); December 16-18, 2016; Guwahati, India [CrossRef]
- Akram M, Sulaiman R. An Empirical Study to Evaluate the Accessibility of Arabic Websites by Low Vision Users. 2020 Presented at: 8th International Conference on Information Technology and Multimedia (ICIMU); August 24-26, 2020; Selangor, Malaysia [CrossRef]
- Branco F, Pereira L, Gonçalves R, Martins J. Web Accessibility on Online Platforms for the Tourism Sector in Portugal. 2021 Presented at: 16th Iberian Conference on Information Systems and Technologies (CISTI); June 23-26, 2021; Chaves, Portugal [CrossRef]
- Johansson S, Gulliksen J, Gustavsson C. Disability digital divide: the use of the internet, smartphones, computers and tablets among people with disabilities in Sweden. Univ Access Inf Soc 2020 Mar 07;20(1):105-120 [CrossRef]
- World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001.
- Martins J, Gonçalves R, Branco F. A full scope web accessibility evaluation procedure proposal based on Iberian eHealth accessibility compliance. Computers in Human Behavior 2017 Aug;73:676-684 [CrossRef]
- Mason AM, Compton J, Bhati S. Disabilities and the Digital Divide: Assessing Web Accessibility, Readability, and Mobility of Popular Health Websites. J Health Commun 2021 Oct 03;26(10):667-674 [CrossRef] [Medline]
- Yu S. A review of the accessibility of ACT COVID-19 information portals. Technol Soc 2021 Feb;64:101467 [https://europepmc.org/abstract/MED/33324025] [CrossRef] [Medline]
- Bokolo AJ. Application of telemedicine and eHealth technology for clinical services in response to COVID‑19 pandemic. Health Technol (Berl) 2021;11(2):359-366 [https://europepmc.org/abstract/MED/33469474] [CrossRef] [Medline]
- E-hälsa. National Board of Health and Welfare. URL: https://www.socialstyrelsen.se/kunskapsstod-och-regler/omraden/e-halsa/ [accessed 2023-07-18]
- Schreiweis B, Pobiruchin M, Strotbaum V, Suleder J, Wiesner M, Bergh B. Barriers and Facilitators to the Implementation of eHealth Services: Systematic Literature Analysis. J Med Internet Res 2019 Nov 22;21(11):e14197 [https://www.jmir.org/2019/11/e14197/] [CrossRef] [Medline]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med 2018 Oct 02;169(7):467-473 [https://www.acpjournals.org/doi/abs/10.7326/M18-0850?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub 0pubmed] [CrossRef] [Medline]
- Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018 Nov 19;18(1):143 [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-018-0611-x] [CrossRef] [Medline]
- Peters MDJ, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc 2015 Sep;13(3):141-146 [CrossRef] [Medline]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 2005 Feb;8(1):19-32 [CrossRef]
- Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc 2018 Oct;106(4):420-431 [https://europepmc.org/abstract/MED/30271283] [CrossRef] [Medline]
- Persson H, Åhman H, Yngling AA, Gulliksen J. Universal design, inclusive design, accessible design, design for all: different concepts—one goal? On the concept of accessibility—historical, methodological and philosophical aspects. Univ Access Inf Soc 2014 May 7;14(4):505-526 [CrossRef]
- Rayyan. Rayyan Systems Inc. URL: https://www.rayyan.ai/ [accessed 2023-07-18]
- Acosta-Vargas P, Esparza W, Rybarczyk Y, González M, Villarreal S, Jadán J. Educational Resources Accessible on the Tele-rehabilitation Platform. In: Nunes I, editor. Advances in Human Factors and Systems Interaction. AHFE 2018. Advances in Intelligent Systems and Computing, vol 781. Cham: Springer; 2019:210-220 [CrossRef]
- Acosta-Vargas P, Rybarczyk Y, Perez J, Gonzalez M, Jimenes K, Leconte L. Towards Web Accessibility in Telerehabilitation Platforms. 2018 Presented at: IEEE Third Ecuador Technical Chapters Meeting (ETCM); October 15-19, 2018; Cuenca, Ecuador [CrossRef]
- Calle-Jimenez T, Sanchez-Gordon S, Rybarczyk Y, Jadán J, Villarreal S, Esparza W. Analysis and Improvement of the Web Accessibility of a Tele-rehabilitation Platform for Hip Arthroplasty Patients. In: Nunes I, editor. Advances in Human Factors and Systems Interaction. AHFE 2018. Advances in Intelligent Systems and Computing, vol 781. Cham: Springer; 2019:233-245 [CrossRef]
- Syahrina A, Fratista S, Pelangi A, Fauzi R. Accessibility Evaluation of Telemedicine System in Older Adults User: A Case Study. 2021 Presented at: International Conference on Advanced Computer Science and Information Systems (ICACSIS); October 23-25, 2021; Depok, Indonesia [CrossRef]
- Calvillo-Arbizu J, Roa-Romero LM, Estudillo-Valderrama MA, Salgueira-Lazo M, Aresté-Fosalba N, Del-Castillo-Rodríguez NL, et al. User-centred design for developing e-Health system for renal patients at home (AppNephro). Int J Med Inform 2019 May;125:47-54 [CrossRef] [Medline]
- Clunne S, Ryan B, Hill A, Brandenburg C, Kneebone I. Accessibility and Applicability of Currently Available e-Mental Health Programs for Depression for People With Poststroke Aphasia: Scoping Review. J Med Internet Res 2018 Dec 04;20(12):e291 [https://www.jmir.org/2018/12/e291/] [CrossRef] [Medline]
- Yogarajah A, Kenter R, Lamo Y, Kaldo V, Nordgreen T. Internet-delivered mental health treatment systems in Scandinavia - A usability evaluation. Internet Interv 2020 Apr;20:100314 [https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(19)30107-1] [CrossRef] [Medline]
- Gibson R, Dunlop M, Bouamrane M, Nayar R. Designing Clinical AAC Tablet Applications with Adults who have Mild Intellectual Disabilities. In: CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020 Presented at: CHI Conference on Human Factors in Computing Systems; April 25-30, 2020; New York, NY, USA [CrossRef]
- Yu D, Parmanto B, Dicianno B. An mHealth App for Users with Dexterity Impairments: Accessibility Study. JMIR Mhealth Uhealth 2019 Jan 08;7(1):e202 [https://mhealth.jmir.org/2019/1/e202/] [CrossRef] [Medline]
- Chowdhary K, Yu DX, Pramana G, Mesoros M, Fairman A, Dicianno BE, et al. User-Centered Design to Enhance mHealth Systems for Individuals With Dexterity Impairments: Accessibility and Usability Study. JMIR Hum Factors 2022 Feb 24;9(1):e23794 [https://humanfactors.jmir.org/2022/1/e23794/] [CrossRef] [Medline]
- Kascak L, Rébola C, Braunstein R, Sanford J. Mobile Application Concept Development for Remote Patient Monitoring. 2013 Presented at: IEEE International Conference on Healthcare Informatics; September 09-11, 2013; Philadelphia, PA, USA [CrossRef]
- Osborne CL, Juengst SB, Smith EE. Identifying user-centered content, design, and features for mobile health apps to support long-term assessment, behavioral intervention, and transitions of care in neurological rehabilitation: An exploratory study. British Journal of Occupational Therapy 2020 Oct 07;84(2):101-110 [CrossRef]
- Schleimer E, Pearce J, Barnecut A, Rowles W, Lizee A, Klein A, et al. A Precision Medicine Tool for Patients With Multiple Sclerosis (the Open MS BioScreen): Human-Centered Design and Development. J Med Internet Res 2020 Jul 06;22(7):e15605 [https://www.jmir.org/2020/7/e15605/] [CrossRef] [Medline]
- Wiljén A, Chaplin JE, Crine V, Jobe W, Johnson E, Karlsson K, et al. The Development of an mHealth Tool for Children With Long-term Illness to Enable Person-Centered Communication: User-Centered Design Approach. JMIR Pediatr Parent 2022 Mar 08;5(1):e30364 [https://pediatrics.jmir.org/2022/1/e30364/] [CrossRef] [Medline]
- Wildenbos GA, Jaspers MWM, Schijven MP, Dusseljee-Peute LW. Mobile health for older adult patients: Using an aging barriers framework to classify usability problems. Int J Med Inform 2019 Apr;124:68-77 [CrossRef] [Medline]
- Involving Users in Evaluating Web Accessibility. W3C Web Accessibility Initiative. URL: https://www.w3.org/WAI/test-evaluate/involving-users/ [accessed 2023-07-18]
- Gulliksen J, Harker S. The software accessibility of human-computer interfaces—ISO Technical Specification 16071. Universal Access in the Information Society 2004 Mar 1;3(1):6-16 [CrossRef]
- Henni SH, Maurud S, Fuglerud KS, Moen A. The experiences, needs and barriers of people with impairments related to usability and accessibility of digital health solutions, levels of involvement in the design process and strategies for participatory and universal design: a scoping review. BMC Public Health 2022 Jan 06;22(1):35 [https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-021-12393-1] [CrossRef] [Medline]
- Antonelli H, Rodrigues S, Watanabe W, de Mattos Fortes R. A survey on accessibility awareness of Brazilian web developers. 2018 Presented at: 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion; June 20-22, 2018; Thessaloniki, Greece [CrossRef]
- Inal Y, Guribye F, Rajanen D, Rajanen M, Rost M. Perspectives and Practices of Digital Accessibility: A Survey of User Experience Professionals in Nordic Countries. 2020 Presented at: 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society; October 25-29, 2020; Tallinn, Estonia [CrossRef]
- Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. BMJ 2013 Nov 20;347:f6753 [CrossRef] [Medline]
- W3C Web Accessibility Initiative. URL: https://www.w3.org/WAI/policies/ [accessed 2023-07-18]
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008 Sep 29;337:a1655 [https://europepmc.org/abstract/MED/18824488] [CrossRef] [Medline]
- James A, Draffan E, Wald M. Designing Web-Apps for All: How Do We Include Those with Cognitive Disabilities? Stud Health Technol Inform 2017;242:665-668 [Medline]
- Johansson S, Gulliksen J, Lantz A. Cognitive Accessibility for Mentally Disabled Persons. In: Abascal J, Barbosa S, Fetter M, Gross T, Palanque P, Winckler M, editors. Human-Computer Interaction – INTERACT 2015. INTERACT 2015. Lecture Notes in Computer Science, vol 9296. Cham: Springer; 2015:418-435 [CrossRef]
- ISO/IEC 40500:2012 Information technology — W3C Web Content Accessibility Guidelines (WCAG) 2.0. International Organization for Standardization. URL: https://www.iso.org/standard/58625.html [accessed 2023-07-18]
- Benefits of standards. International Organization for Standardization. URL: https://www.iso.org/benefits-of-standards.html [accessed 2023-07-18]
- Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015 Jan 02;350:g7647 [http://www.bmj.com/lookup/pmidlookup?view=long&pmid=25555855] [CrossRef] [Medline]
|EN: European Standards|
|IEC: International Electrotechnical Commission|
|ISO: International Organization for Standardization|
|MRC: Medical Research Council|
|PICO: Population, Intervention, Comparison, and Outcome|
|PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews|
|UNE: Spanish Association for Standardization|
|UX: user experience|
|W3C: World Wide Web Consortium|
|WCAG: Web Content Accessibility Guidelines|
Edited by T Leung, T de Azevedo Cardoso; submitted 16.12.22; peer-reviewed by E Schleimer, R Sapkota; comments to author 30.03.23; revised version received 09.06.23; accepted 12.07.23; published 17.08.23Copyright
©Marika Jonsson, Stefan Johansson, Dena Hussain, Jan Gulliksen, Catharina Gustavsson. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.08.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.