Review
Abstract
Background: During the COVID-19 pandemic, telehealth was expanded without the opportunity to extensively evaluate the adopted technology’s usability.
Objective: We aimed to synthesize evidence on health professionals’ perceptions regarding the usability of telehealth systems in the primary care of individuals with noncommunicable diseases (NCDs; hypertension and diabetes) from the COVID-19 pandemic onward.
Methods: A systematic review was performed of clinical trials, prospective cohort studies, retrospective observational studies, and studies that used qualitative data collection and analysis methods published in English, Spanish, and Portuguese from March 2020 onward. The databases queried were MEDLINE, Embase, BIREME, IEEE Xplore, BVS, Google Scholar, and grey literature. Studies involving health professionals who used telehealth systems in primary care and managed patients with NCDs from the COVID-19 pandemic onward were considered eligible. Titles, abstracts, and full texts were reviewed. Data were extracted to provide a narrative qualitative evidence synthesis of the included articles. The risk of bias and methodological quality of the included studies were analyzed. The primary outcome was the usability of telehealth systems, while the secondary outcomes were satisfaction and the contexts in which the telehealth system was used.
Results: We included 11 of 417 retrieved studies, which had data from 248 health care professionals. These health care professionals were mostly doctors and nurses with prior experience in telehealth in high- and middle-income countries. Overall, 9 studies (82%) were qualitative studies and 2 (18%) were quasiexperimental or multisite trial studies. Moreover, 7 studies (64%) addressed diabetes, 1 (9%) addressed diabetes and hypertension, and 3 (27%) addressed chronic diseases. Most studies used a survey to assess usability. With a moderate confidence level, we concluded that health professionals considered the usability of telehealth systems to be good and felt comfortable and satisfied. Patients felt satisfied using telehealth. The most important predictor for using digital health technologies was ease of use. The main barriers were technological challenges, connectivity issues, low computer literacy, inability to perform complete physical examination, and lack of training. Although the usability of telehealth systems was considered good, there is a need for research that investigates factors that may influence the perceptions of telehealth usability, such as differences between private and public services; differences in the level of experience of professionals, including professional experience and experience with digital tools; and differences in gender, age groups, occupations, and settings.
Conclusions: The COVID-19 pandemic has generated incredible demand for virtual care. Professionals’ favorable perceptions of the usability of telehealth indicate that it can facilitate access to quality care. Although there are still challenges to telehealth, more than infrastructure challenges, the most reported challenges were related to empowering people for digital health.
Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42021296887; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=296887
International Registered Report Identifier (IRRID): RR2-10.21801/ppcrj.2022.82.6
doi:10.2196/44209
Keywords
Introduction
Noncommunicable diseases (NCDs) are responsible for at least 71% of global deaths, with hypertension and diabetes being the main diseases [
- ]. Most of these deaths, however, can be prevented with proper treatment [ - ]. These diseases are chronic and progressive, and often require complex and continuous management of physiological, environmental, and behavioral factors [ - ]. Treatment expenses are high for both patients and health services [ - ]. As a result of the COVID-19 pandemic, there was a loss in the follow-up of patients with NCDs due to the increased risk of illness and death from COVID-19 posed by in-person consultation [ - ]. Patients with chronic diseases, such as hypertension and diabetes, are strongly affected by COVID-19. This can be attested by the high prevalence of patients with NCDs and COVID-19 [ ]. More consequential, patients with hypertension and diabetes are much more likely to develop a more serious illness or die once infected by the virus when compared to patients who do not have a chronic disease [ ].NCDs are a major challenge for sustainable development. The United Nations (UN) 2030 Agenda for Sustainable Development has committed to developing responses to reduce premature mortality from NCDs by one-third. Innovative and cost-effective solutions that improve the care of people with NCDs have become increasingly relevant [
, ], with telehealth emerging as one of the great alternatives to maintain and enhance health care. Solutions have been implemented even in diverse geographical areas, thus enabling clinical and nonclinical, and synchronous and asynchronous services of different modalities and different functions [ ]. Telehealth is a revolutionary patient management approach combining various forms of information communication technologies [ , , ]. It allows access to health assessment, diagnosis, intervention, consultation, rehabilitation, supervision, education, and information at a distance [ , ]. Primary care practices, especially for NCDs, quickly resorted to telehealth to deal with the COVID-19 pandemic [ ]. However, it is unclear whether telehealth will continue to be widely used after the pandemic.Although numerous studies have shown that patients are satisfied with telehealth and value its convenience [
- ], data on the assessment of telehealth usability by health professionals who use telehealth systems are scarce [ ]. Usability is one of the essential quality attributes of any system and is related to its acceptability by its end users [ - ]. Measuring usability allows for the assessment of user and provider satisfaction and establishes strengths and weaknesses that can help improve the effectiveness of the technology and services provided [ - ]. This information is essential for understanding barriers and strengths that can consolidate telehealth.Based on this, the main interest of this review was to understand how health care professionals assess a telehealth usability system to provide and support health care services for patients with hypertension and diabetes starting from the COVID-19 pandemic. This study was based on a comprehensive review of scientific publications on the topic.
Methods
Study Design
We conducted a systematic review of qualitative research following the methodological recommendations of the Cochrane Collaboration Handbook [
] and reported the findings according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) statement [ ]. The study protocol was registered with PROSPERO (CRD42021296887) in December 2021 [ ] and has been published [ ].Identification and Selection of Studies
Three independent reviewers performed the literature search using MeSH (Medical Subject Headings) and appropriate keywords for each database. The strategy was initially designed for MEDLINE (via PubMed) and then adapted for use in the following databases: Embase, BIREME, IEEE Xplore, BVS, Google Scholar, and gray literature. The search started in December 2021 and ended at the end of March 2022. Clinical trials, prospective cohort studies, retrospective observational studies, and studies that used qualitative data collection and analysis methods in English, Portuguese, and Spanish were searched. The following filters were used: (1) publication date: 2020, 2021, or within 1 year; (2) language: English, Portuguese, or Spanish; and (3) study type: human studies only. Detailed search strategies are presented in
.Eligibility criteria based on the PEOTS (patient/population-exposure-outcome-time-study design) framework and inclusion criteria are shown in
. Studies that did not answer our research questions included studies that did not report on telehealth usability by health professionals and incomplete articles. Abstracts, review articles, editorials, blogs, books, academic works, dissertations, theses, duplicate articles, and scientific events were excluded.Study inclusion criteria.
Inclusion criteria
- Participants: All categories of health professionals who work in the care of patients with hypertension and diabetes and used telehealth.
- Exposure: Telehealth.
- Primary outcome: Usability or ease of use regarding patient care.
- Secondary outcome: Satisfaction (acceptance), impression of the patient’s satisfaction, and contexts in which digital tools are used.
- Time: From March 2020 (from the COVID-19 pandemic onwards).
- Study design: Clinical trials, prospective cohort studies, retrospective observational studies, and cross-sectional observational studies.
Article search and selection were carried out blindly and independently by 3 evaluators (2 had a first degree in health sciences and 1 had a first degree in applied linguistics) following a 3-phase process that included identification, screening, and inclusion. In the identification phase, a search was carried out in the databases through descriptors and filters. Titles and abstracts of identified studies were searched. Articles deemed potentially relevant from their titles and abstracts were retrieved as full-text articles and evaluated in terms of meeting eligibility requirements. In the screening phase, studies were selected based on their titles, abstracts, and full text according to the inclusion and exclusion criteria. Disagreements among evaluators were discussed until a consensus was reached. In the inclusion phase, the final selection of included studies was performed for qualitative analysis.
To manage the data, including deduplication, the software Rayyan from Qatar Computing Research Institute (QCRI) was used [
]. After the final selection, data were extracted and information about the included studies was organized into a predefined model, which included title, scientific journal, year of publication, study type, sample population, country, study outcomes, and additional findings of interest for the review. Excluded articles were reported, together with the reasons for their exclusion.Data Analysis
A narrative qualitative evidence synthesis of the findings from the included articles was performed. Screening, selection, data extraction, and bias assessment of the studies were performed by 2 evaluators (a health professional and a linguist) independently, and the results were compared.
Characteristics of the Included Studies
A predefined template was used to collect the main characteristics and outcomes of the studies included. A summary of the qualitative findings table of the review has been displayed for an evidence profile [
].Risk of Bias and Methodological Quality Analysis
The quality of the selected studies and their results were analyzed. The data for the critical quality analysis were consolidated in the Joanna Briggs Institute (JBI) [
] form for observational studies to assess the methodological quality of the studies and determine the extent of possibilities for bias, conduct, and analysis. The JBI tool comprises the following 8 questions:- Were the criteria for inclusion in the sample clearly defined?
- Were the study subjects and the setting described in detail?
- Was the exposure measured in a valid and reliable way?
- Were objective standard criteria used for measurement of the condition?
- Were the confounding factors identified?
- Were strategies to deal with confounding factors stated?
- Were the outcomes measured in a valid and reliable way?
- Was appropriate statistical analysis used?
The studies were categorized according to the percentage of positive responses to the questions present in the assessment instrument. The risk of bias was considered high when the study had below 49% of responses classified as “yes,” moderate when the study had 50% to 69% of responses classified as “yes,” and low when the study had more than 70% of responses classified as “yes.”
Studies were not excluded based on the assessment of the risk of bias and methodological quality, but we used this information to assess confidence in the synthesis findings as part of the GRADE-CERQual approach (Grading of Recommendations Assessment, Development and Evaluation - Confidence in Evidence from Qualitative Research Reviews) [
].Report and Recommendations
The Qualitative Research Checklist of the Critical Appraisal Skills Program (CASP), part of the Oxford Center for Triple Value Healthcare Ltd [
], was used to evaluate the studies qualitatively and systematically by 2 reviewers (a health professional and a linguist) independently ( ). This tool analyzes whether the results of the review are valid, what the results are, and whether the results will be useful locally.The CASP tool comprises the following 10 questions (answer options: yes, can’t tell, and no):
- Was there a clear statement of the aims of the research?
- Is a qualitative methodology appropriate?
- Was the research design appropriate to address the aims of the research?
- Was the recruitment strategy appropriate to the aims of the research?
- Was the data collected in a way that addressed the research issue?
- Has the relationship between researchers and participants been adequately considered?
- Have ethical issues been taken into consideration?
- Was the data analysis sufficiently rigorous?
- Is there a clear statement of the findings?
- How valuable is the research?
Two evaluators (a health professional and a linguist) independently analyzed the quality of evidence and strength of recommendation by the GRADE-CERQual approach [
]. This tool is based on a systematic system and transparent framework to assess confidence in individual review results based on consideration of the following 4 components: (1) methodological limitations, (2) coherence, (3) adequacy of data, and (4) relevance.Methodological limitations refer to whether there is concern with the design or conduct of primary studies that contribute evidence to an individual review finding [
]. Coherence is defined as assessing how clear and cogent the fit is between the data from the primary studies and a review finding that synthesizes that data. By “cogent,” we mean well supported or compelling [ ]. Adequacy of data means an overall determination of the degree of richness and quantity of data supporting a review finding [ ]. Relevance refers to the extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context (perspective or population, phenomenon of interest, and setting) specified in the review question [ ].The methodological analysis of the included studies was carried out in steps [
]. Step 1 aimed to collect and consider the information necessary to report methodological limitations. Step 2 aimed to assess the body of data that contributed to each review finding and decide whether there was a consensus on methodological limitations. Step 3 aimed to pass judgment on the seriousness of the concerns and justify that.The result of the recommendation is considered as follows [
]:- High confidence: if it is highly likely that the outcome of the review is a reasonable representation of the phenomenon of interest. No or very minor concerns regarding methodological limitations/coherence/adequacy/relevance that are unlikely to reduce confidence in the review finding.
- Moderate confidence: if it is likely that the review finding is a reasonable representation of the phenomenon of interest. Minor concerns regarding methodological limitations/coherence/adequacy/relevance that may reduce confidence in the review finding.
- Low confidence: if it is possible that the review finding is a reasonable representation of the phenomenon of interest. Moderate concerns regarding methodological limitations/coherence/adequacy/relevance that will probably reduce confidence in the review finding.
- Very low confidence: if it is not clear whether the review finding is a reasonable representation of the phenomenon of interest. Serious concerns regarding methodological limitations/coherence/adequacy/relevance that are very likely to reduce confidence in the review finding.
Results
Included Studies
A total of 417 abstracts were selected, and 11 full-text articles were evaluated. Eleven studies met our inclusion criteria.
shows our search strategy.Excluded Studies
We excluded 399 studies (
). The reasons for exclusion were different populations than intended, different outcome variables, nonattention to the PEOTS questions ( ), different study periods, different types of publications, and different languages than stipulated.Characteristics of the Included Studies
summarizes the results of the individual studies.
Article title | Population of interest for the review | Outcomes and main findings of interest for the review | Barriers, limitations, and gaps |
Views of patients and providers on the use of telemedicine for chronic disease specialty care in the Alaska native population [ | ]10 physicians from specialist clinics at Alaska Native Medical Center who used telemedicine to provide care (40% had a lot of experience in telehealth, 30% some, and 30% little). Half of the sample was female and half male. Moreover, 80% were aged 40-59 years and 20% were over 60 years. |
|
|
Criteria for assessing and recommending digital diabetes tools: A Delphi study [ | ]15 health care professionals (3 physicians, 10 nurses, 1 nutritionist, and 1 occupational therapist) with diabetes care experience, an average of 19 years of experience in diabetes care working in the 4 health regions of Norway, and an average of 6.8 years of experience with digital tools. |
| The authors cite security and privacy as possible barriers to using digital tools. |
MoTHer, an mHealth system to support women with gestational diabetes mellitus: Feasibility and acceptability study [ | ]Physicians (number not reported) |
| Health professionals point out technological and connection problems as barriers. The authors also argue that funding models have traditionally not considered the use of telehealth interventions, with the lack of codes and manipulation of reimbursement as barriers to the acceptance of telehealth in public and private health settings. They add that interoperability with existing prevailing health care practices and information technology systems (including data privacy regulations) should be prioritized, emphasizing that technology advances much faster than regulatory reform. |
Implementation of digital monitoring services during the COVID-19 pandemic for patients with chronic diseases: Design science approach [ | ]53 health care professionals (37 physicians and 16 nurses) |
| The authors argue that there is a lack of guidelines on remote patient management using digital health solutions, adequate technological literacy, and internet access. They cite the possibility of misdiagnosis, equipment malfunction, and privacy violations as potential risks. They address the need for adequate training of health professionals in skills to deal with telehealth aiming at safe and effective care. |
“In my age, we didn\'t have the computers”: Using a complexity lens to understand uptake of diabetes eHealth innovations into primary care-A qualitative study [ | ]10 health care professionals (5 physicians, 3 nurses, and 2 dietitians) |
| Health care professionals reported limited consultation hours and complex workflows as barriers to telehealth, identifying that senior administrative leadership (eg, medical directors and chronic disease managers), clinical reservation staff, and the information technologist were key players in reducing technological barriers and optimizing workflow-related issues. |
Home telemonitoring for chronic disease management: Perceptions of users and factors influencing adoption [ | ]9 clinicians |
| The authors argue that the main barrier to the implementation of telehealth was acceptance by health professionals. They also argue that the lack of support from information technology technicians would be a barrier. The main challenge is to incorporate telehealth work into their existing work practice, and this includes adapting to the local organizational context, a dedicated telehealth professional, and a dedicated telehealth coordinator. |
Development and validation of a mHealth technology for the promotion of self-care for adolescents with diabetes [ | ]5 health professionals (clinicians) with experience in caring for adolescents diagnosed with diabetes mellitus type 1. |
| The authors did not discuss barriers and limitations. |
Development and implementation of a decision support system to improve control of hypertension and diabetes in a resource-constrained area in Brazil: Mixed methods study [ | ]96 health care professionals (25 physicians, 44 nurses, and 27 other health professionals) aged between 27 and 43 years. |
| The authors argue that many clinical decisions support systems (CDSSs) have not been regularly used or extensively supported by primary care physicians. The barrier cited for using CDSSs was the difficulty in running systems that are not integrated with existing health systems and health records. The authors also point to different levels of internet connectivity in primary care centers and low technological literacy among health professionals as barriers. |
Rapid implementation of a diabetes telemedicine clinic during the Coronavirus Disease 2019 outbreak: Our protocol, experience, and satisfaction reports in Saudi Arabia [ | ]14 health care providers who participated in the virtual clinic. |
| Health professionals considered the need for technical skills with digital tools and the higher cost than usual care as barriers to telehealth. The authors cite as a limitation the difficulty in maintaining patient safety standards and protecting them against loss of privacy. The main limitations of telehealth pointed out were the inability to perform a complete physical examination and the requirement of laboratory testing that cannot be delayed. |
Patient and provider perspectives on a novel mobile health intervention for low-income pregnant women with gestational or type 2 diabetes mellitus [ | ]29 providers (14 physicians, 6 nurses, 2 medical assistants, 1 registered dietitian, 1 licensed clinical social worker, 1 lactation counselor, and 4 certified health educators), with the majority being female. |
| The authors argue that health professionals considered that the characteristics of the devices, interface, and ease of use, such as functional features, visual features, and even the quality of the information, can be barriers to the adoption of digital tools. Another point discussed as a barrier was the lack of integration between platforms, with the need to switch between internal platforms and digital tools during consultations, considering that the use of digital tools took additional time for the provision of health care. |
Mobile-based and cloud-based system for self-management of people with type 2 diabetes: Development and usability evaluation [ | ]7 health professionals (3 physicians and 4 nurses) aged between 35 and 42 years. |
| The barriers discussed by the authors include the lack of technical and human resources in organizations to support the implementation of telehealth systems, the lack of financing for mobile technology solutions, the challenges of merging health care solutions with electronic health records, and other issues. The authors argue that these barriers are aggravated by the reluctance and fear of the health professional to use digital tools and the lack of sufficient time due to excess consultations. |
All sampled studies were published in 2021 [
- ]. Eight of the sampled studies were conducted in high-income countries, including Australia (n=2) [ , ], Canada (n=1) [ ], the United States (n=2) [ , ], Norway (n=1) [ ], Portugal (n=1) [ ], and Saudi Arabia (n=1) [ ]. Three of the sampled studies were conducted in middle-income countries, including Brazil (n=2) [ , ] and Iran (n=1) [ ]. All included studies were published in English, with one of them written in both English and Portuguese [ ].Moreover, 9 studies were qualitative studies [
, - , - ], 1 was a quasiexperimental study [ ], and 1 was a multisite trial [ ]. Furthermore, 7 studies addressed diabetes [ , , , , - ], 1 addressed diabetes and hypertension [ ], and 3 addressed chronic diseases [ , , ]. All studies focused on primary care. Additionally, 7 studies addressed telehealth [ , , , , , , ] and 4 addressed telemedicine [ , , , ] ( ).Variable | Frequency | Citation | |||
Level of socioeconomic development in the study country | |||||
Middle-income country | 8 | [ | - , , ]|||
High-income country | 3 | [ | , , ]|||
Study design | |||||
Qualitative study | 9 | [ | , - , - ]|||
Quasiexperimental study | 1 | [ | ]|||
Multisite trial | 1 | [ | ]|||
Disease addressed in the study | |||||
Diabetes | 7 | [ | , , , , - ]|||
Diabetes and hypertension | 1 | [ | ]|||
Chronic diseases | 3 | [ | , , ]|||
Digital tool used for telehealtha | |||||
Video | 2 | [ | , ]|||
App | 3 | [ | , , ]|||
Smartphone | 3 | [ | , , ]|||
Digital platform | 2 | [ | , ]|||
Website | 1 | [ | ]|||
Visual media | 3 | [ | , , ]|||
Clinical decision support system | 1 | [ | ]|||
Cloud-based mobile tool | 2 | [ | , ]|||
Mobile health tool | 1 | [ | ]|||
Internet-based interactive support system | 2 | [ | , ]|||
Email/WhatsApp message | 1 | [ | ]
aMultiple tools have been used in individual studies.
Most of the included studies focused on the evaluation of a specific tool to deliver telehealth and were studies of the development or validation of digital tools. The telehealth tools in the included studies were apps, smartphones, websites, visual media, digital platforms, and mobile and cloud-based applications. The digital interventions described in the included studies were telemedicine [
, , ], teleconsultation [ ], telemonitoring [ , ], decision support [ ], educational videos, and digital interventions for self-management [ - , ]. Eight studies addressed usability, and 10 studies addressed satisfaction [ - , ]. Five studies used interviews to assess usability [ , , - ], and 6 studies used surveys [ , - ].The population of the included studies covered patients and health care providers. The population of interest for our review was health care providers only. One study did not report the sample number of health care providers [
]. A total of 248 health professionals were studied. Regarding the occupation of health professionals, most studies targeted physicians or nurses as follows: 125 physicians (clinicians, medical assistants, and specialists such as cardiologists and endocrinologists), 83 nurses, 4 nutritionists, 1 occupational therapist, and 35 others not specified. All health professionals investigated in the studies had previous experience with telehealth and with the care of patients with hypertension and diabetes.Findings Related to the Outcomes of Interest
The main finding of this review was that health professionals considered the usability of telehealth to be good and felt comfortable and satisfied using telehealth for the care of patients with hypertension and diabetes in primary care. All studies contributed to this outcome. The review set of analyses supports the assessment finding with moderate confidence (
).Patients with hypertension and diabetes were also satisfied with using telehealth in primary care. Eight studies contributed to this finding. The analysis showed moderate confidence in this outcome.
A summary of the results is reported in
. According to consensus, ease of use was the most important predictor for using digital technologies in health [ - ].Some studies reported that the main barriers identified by health professionals were technological challenges [
, , , ], low computer skills [ , , ], and a lack of appropriate equipment for telehealth [ ]. In addition, the inability to perform comprehensive physical examinations [ , , ], a lack of training [ , ], safety flaws [ , , ], barriers to regulatory issues [ , ], scheduling difficulties [ , ], time spent scheduling new patients [ , , ], and telehealth not integrated with other practices or members of the health care team [ , , ] were considered. It was also reported that telehealth was not suitable when patients needed a diagnosis, procedure, or drug that could only be provided in large hospitals [ , ]; when there was a language/cultural barrier [ , , , ] or hearing difficulty [ ]; when health professionals treated new, unstable, or undiagnosed patients [ , ]; or when the patient expressed a preference for face-to-face consultations to discuss sensitive issues [ ]. Other challenges included logistics [ , , , ], clinical and administrative support [ , - ], absence of a dedicated clinical care coordinator [ , , ], connectivity issues [ , ], complexity [ , ], a lack of platform interoperability [ , ], a lack of integration between devices [ , , ], and fear of using digital tools [ , ].Factors that would improve the usability of the telehealth system were pointed out. These included attractive simple visual and functional features [
, ], having a provider in the room with the patient (ie, a dedicated telepresenter) [ , ], having dedicated time for telehealth [ , ], receiving standardized training [ , , , ], having dedicated staff to schedule and coordinate telehealth [ , , , ], improving security [ , , ] and privacy criteria [ , , ], and running systems that are interoperable and integrated into existing health systems and health records [ , , ].Only 1 study [
] pointed out risks with the use of telehealth, which included the possibility of incorrect diagnosis, equipment malfunction, and privacy violations.Summary of review findinga | Studies contributing to the review finding | Methodological limitations | Coherence | Relevance | CERQualb assessment of confidence in the evidence |
Health professionals considered the usability of telehealth in the care of patients with hypertension and diabetes in primary care to be good. They reported benefits, mainly related to access to specialized services, improved communication, cost reduction, and improved follow-up continuity by allowing remote follow-up. | All studies contributed to this outcome. | Moderate concerns regarding methodological limitations that will probably reduce confidence in the review finding. | No or very minor concerns regarding coherence that are unlikely to reduce confidence in the review finding. | Moderate concerns regarding relevance. Only 8 countries were represented in the studies (6 high-income countries, 2 middle-income countries, and 0 low-income countries). | Moderate confidence: it is likely that the review finding is a reasonable representation of the phenomenon of interest. |
Health professionals felt comfortable and satisfied using telehealth for the primary care of patients with hypertension and diabetes. | Eight articles addressed satisfaction with telehealth. | Moderate concerns regarding methodological limitations that will probably reduce confidence in the review finding. | No or very minor concerns regarding coherence that are unlikely to reduce confidence in the review finding. | Moderate concerns regarding relevance. Most studies evaluated a type of digital tool in a limited setting. Therefore, it is not clear whether the findings of the phenomenon are applicable, transferable, generalizable, or valid in all contexts. | Moderate confidence: it is likely that the review finding is a reasonable representation of the phenomenon of interest. |
Patients felt satisfied with telehealth for hypertension and diabetes. | Eight articles addressed satisfaction with telehealth. | Moderate concerns regarding methodological limitations that will probably reduce confidence in the review finding. | No or very minor concerns regarding coherence that are unlikely to reduce confidence in the review finding. | Moderate concerns regarding relevance. Most studies evaluated a type of digital tool in a limited setting. Therefore, it is not clear whether the findings of the phenomenon are applicable, transferable, generalizable, or valid in all contexts. | Moderate confidence: it is likely that the review finding is a reasonable representation of the phenomenon of interest. |
aThe objective was to synthesize evidence on health professionals’ perceptions of the usability of telehealth in the primary care of individuals with noncommunicable diseases (hypertension and diabetes) from the COVID-19 pandemic onward. The study aimed to enhance our understanding of whether health professionals can use telehealth to achieve specific goals in the care of patients with hypertension and diabetes in primary care with effectiveness, efficiency, and satisfaction in a particular context of use. The types of digital tools addressed in the studies were as follows: web-based video conferencing, apps, websites, social media, smartphones, internet-based interactive support, digital monitoring services, email, WhatsApp, tablets, clinical decision support systems, mobile apps, and mobile-based and cloud-based systems.
bCERQual: Confidence in Evidence from Qualitative Research Reviews.
Methodological Quality, Risk of Bias, and Limitations of the Included Studies
A considerable number of issues compromised methodological quality and increased the risk of bias in the included studies. None of the studies presented a sample size calculation, and all of them were nonprobabilistic convenience samples, with small samples, which reduced the credibility of the data representation. Most health professionals were recruited through expert sampling, which may have impaired the external validity of the studies. Besides this, all studies provided critical information necessary for detailing of the sample, with explicit inclusion and exclusion criteria for recruiting study participants. Most studies did not describe subjects and settings in detail, thus compromising external validity.
Most studies did not use a validated and reliable instrument to assess usability. In some studies, usability results from the health care providers’ perspectives were not presented in depth, which reduced the credibility of the data and impacted the accuracy of the findings.
Another recurring problem was that none of the studies addressed confounders or described strategies to analyze the influence of possible confounders on the results, which increased the risk of bias. Aspects, such as participant age, previous experience with digital tools, and whether telehealth was offered in a private/public service or a rural/urban area, can influence or confuse the results found. Developing an in-depth understanding of where telehealth is offered, by whom, and in what situation, is essential to deepen the discussion of its usability.
Regarding data adequacy, most of the included studies supported the results found (
). Despite the small number of participants in each study, the overall number of participants in our review did not impact our claims about the usability of telehealth from the perspective of health care professionals in the investigated contexts, even though it precluded the possibility of making broad claims about telehealth usability in any context. shows the classification of the risk of bias in the analyzed studies. Six studies (55%) had a moderate risk of bias and were considered to have moderate methodological quality, and 5 studies (45%) had a high risk of bias and were considered to have low methodological quality.Quality of the Evidence and Strength of the Recommendations
The quality of the evidence regarding the usability of telehealth systems for the care of patients with hypertension and diabetes in primary care from the view of health professionals was considered moderate. These findings are a reasonable representation of the phenomenon of interest. Moderate concerns were found regarding methodological limitations, coherence, adequacy, and relevance, which reduced confidence in the findings. Thus, it is possible that more research in different contexts of use will influence the confidence of these findings.
Discussion
Principal Findings
This synthesis of qualitative evidence examined the usability of telehealth systems for patients with diabetes and hypertension in primary care from the perspective of health professionals who use digital tools. Our findings suggest that health care professionals consider the usability of telehealth systems to be good. The quality of evidence for this finding was considered moderate. With moderate confidence, we found that health care professionals felt comfortable and satisfied using telehealth to care for their patients with diabetes and hypertension. Likewise, patients with diabetes and hypertension felt satisfied being cared for using telehealth systems.
Telemedicine and Telehealth
According to the Centers for Disease Control and Prevention (CDC), although often used interchangeably, the terms “telehealth” and “telemedicine” are not synonymous [
]. For the World Health Organization, however, there is no clear distinction between these terms [ ]. Telemedicine is defined as “the use of electronic information and communications technologies to provide and support health care when distance separates the participants” [ ]. A broader definition of telemedicine encompasses both clinical and nonclinical applications [ ]. Clinical applications of telemedicine involve patient care, including diagnosis, treatment, and other medical decisions or services for specific patients. Nonclinical uses of telemedicine include continuing medical education and management meetings [ ].Telehealth is a large and heterogeneous collection of clinical practices, technologies, and arrangements shared with information technologies involving different strategies and tools. These may include telemedicine and various nonmedical services, such as telenutrition, telenursing, telepharmacy, teledentistry, teleaudiology, teleneuropsychology, telerehabilitation, teletrauma, tele-electrocardiography, teleradiology, telepathology, and even teleabortion in countries where this practice is allowed [
]. Telehealth can be conceptualized as the use of electronic information and telecommunication technologies to support long-distance clinical health care, health-related education of patients and professionals, public health, and health administration [ , , , , ]. Technologies include video conferencing, the internet, image storage and forwarding, media streaming, and terrestrial and wireless communications [ ]. The wide and heterogeneous collection of clinical practices and technologies used to offer telehealth expands the range of possibilities, but, on the other hand, it becomes a challenge to synthesize and compare them.In our study, we differentiated between telehealth and telemedicine based on the type of service offered and the terminology used by the authors. The studies included in our review evaluated different telehealth modalities with different systems. The modalities included video telemedicine, teleconsultation, patient data monitoring, alerts, therapeutic management, and remote monitoring, and involved apps, websites, visual media, smartphones, internet-based interactive support systems, web-based decision support systems, educational videos, and mobile devices.
Telehealth Before COVID-19
Telehealth has a long history. Telehealth systems have been used for decades in clinical settings, mainly in high- and middle-income countries [
]. In an 1879 article in the Lancet, there was talk of using the telephone to reduce unnecessary office visits. In 1906, Einthoven published an article dealing with the requirement for transtelephonic transmission of electrocardiography information from the physiology laboratory to the clinic at the Academic Hospital about a kilometer away (“télécardiogramme”) [ ]. The radio has been used since the 1920s to provide medical advice to ship clinics. In 1925, a cover of Science and Invention magazine showed a doctor diagnosing a patient by radio, and the article inside it imagined a device that would allow the video examination of a patient from a distance [ ]. In 1948, 24 miles apart in Pennsylvania, 2 health professionals transmitted the first radiographic images over the telephone. In Alaska, for decades, otoscopy and audiometry have been performed by community health workers in small villages, and information is sent to specialists in other cities.As technology evolves, other digital tools are incorporated into telehealth. An example is home monitoring, which developed more fully during the Mercury space program when the National Aeronautics and Space Administration (NASA) began to perform remote physiological monitoring, and for more than 50 years, NASA’s work in telemetry, remote communication, and life sciences has led to unprecedented advances in digital health [
].Government agencies and health care providers have turned to telehealth in response to disasters over the years. In the year 2000, the North Atlantic Treaty Alliance (NATO) (an intergovernmental military alliance between 29 members, including North American and European countries) developed a Multinational Telemedicine System that was deployed with its military forces during several crises [
]. Through solutions, such as implantable portable telemedicine personal kits and satellite linkage, areas in need received health support from medical specialists located in other countries [ ]. During hurricanes Harvey and Irma, 7 private telemedicine companies provided care to victims [ ]. In 2003, after the severe acute respiratory syndrome (SARS) pandemic, China began to explore telehealth and integrated electronic medical systems for use in similar situations in the future. In 2019, telehealth was offered to people affected by wildfires [ ].The usability challenges of telehealth from the health professional’s perspective and other human factors involving information technology (IT) have been less studied than usability from the patient’s point of view. However, the challenges are huge. The literature on human factors and organizational aspects of telehealth shows usability problems of telehealth systems experienced by health professionals. Examples include a potential negative impact on patient safety, poor workflow integration, frustration, and burnout when practitioners spend more time and effort integrating telehealth into their work routines [
]. Additionally, before the pandemic, some health professionals reported hating their computers, and others said they feared technology, believing that with telehealth, it would not be possible to maintain the quality of care [ ].Telehealth From COVID-19 Onward
The widespread adoption of telehealth, which many proponents have advocated for years, is one of the most significant shifts in health care that the COVID-19 pandemic has brought. This triggered regulatory changes that lowered the barriers to telehealth in several countries worldwide, resulting in the large-scale expansion of its use [
]. Patients and health professionals found extrinsic motivations to expand telehealth, such as the possibility of avoiding face-to-face contact, reducing costs, reducing the need for travel, and providing specialized care, even in remote areas. If, before the pandemic, patients and even health professionals were intimidated by technology and questioned whether telehealth could be an efficient way to take care of health, today, most consider telehealth helpful with quality equal to or greater than that of traditional care, mainly for patients with NCDs [ ]. However, most organizations and professions were not ready for telehealth before the pandemic, causing staff resistance and lack of use of this technology in the interprofessional setting when most needed [ ].Telehealth and its effectiveness have been examined for various diseases [
]. Two recent systematic reviews analyzed the effectiveness of the use of telehealth programs for the care of individuals with hypertension and diabetes mellitus, demonstrating that telehealth was an effective tool for the health care of these patients, which improved the care experience (eg, patient awareness and engagement) [ , ]. Patient satisfaction with telehealth has also been researched, showing good results [ , , , ]. The main contribution of this study to the field of knowledge is the addition of the perception of health professionals who adopted this strategy to care for patients with hypertension and diabetes from the COVID-19 pandemic onward.Usability of Telehealth for the Care of Patients With Hypertension and Diabetes
Usability can be defined as the extent to which a product can be used by specific users to achieve specific goals of effectiveness, efficiency, and satisfaction in a specific context of use [
]. The concept of usability, therefore, comprises a combination of user actions and attitudes so that the quality of the system depends on the degree to which the system satisfies the stated and implied needs of its various stakeholders, providing value. In this way, different approaches to what makes a product “usable” are divided into looking at the product with the product’s vision in mind (ie, functional, visual, and ergonomic characteristics) and looking from the end user’s point of view (ie, effectiveness, efficiency, security regarding confidentiality, and satisfaction of use), involving the ability to solve and manage problems and the ease of use of the product in the real world [ ]. Evidence has associated dissatisfaction and high dropout rates in telehealth with the lack of some of these factors [ , ].The usability of systems is one of the essential quality attributes. Good usability of telehealth is crucial for its adoption and acceptance, and the health professional’s involvement with it [
, , ]. It is one of the most significant predictors of intended use [ ]. However, there are more studies addressing usability from the patient’s perspective than the health professional’s perspective. A Cochrane Database of Systematic Reviews 2020 concluded that further studies are needed to address the perceptions and experiences of health professionals using digital tools to provide primary health services in high-, low-, and middle-income countries [ ]. Our review included high-income countries and middle-income countries. However, no low-income country was included.In several high-income countries, telehealth is already a reality. In this way, important issues, such as those related to implementing and managing change in IT in health, have already been overcome [
]. In these locations, the core IT infrastructure is in place, and health care organizations are investing in designing and implementing the next generation of health care IT “byproducts,” such as clinical dashboards, status displays, clinical decision support, and technologies geared toward the patient [ ]. However, in low- and middle-income countries, telehealth challenges still include issues and costs related to implementing the technology. Furthermore, in low- and middle-income countries, rapid globalization has progressively increased the incidence of NCDs. Low- and middle-income countries have higher NCD mortality rates, whereby 77% of all NCD deaths are in low- and middle-income countries [ ]. Most of these countries face problems with infrastructure and the need for patients to travel long distances to hospitals [ ]. In these countries, telehealth can be particularly beneficial to expand access to health care. In these countries, the barriers and challenges of telehealth may differ from those faced by high- and middle-income countries. In this way, understanding the usability of telehealth from the perspective of the health professional who uses the tool in low-income countries can bring enormous contributions to global health and is a gap in the literature.Usability has been evaluated with various methods and measurements over the years [
, ]. Most studies included in this review, however, evaluated usability as a quality characteristic. Few studies specified other attributes, such as security and efficiency. Additionally, we found that most studies included interviews and surveys, making reproduction and comparison between studies difficult. Only 2 studies used standardized instruments for assessment, and it was more common for studies to use author-created questionnaires for usability. Our results are somewhat in line with findings from a previous review that found that usability was primarily assessed through polls and questionnaires [ ]. It is possible that the wide variety of the types of digital tools has created different assessment needs that are not captured in existing standardized usability questionnaires. Furthermore, usability may more often be a concern of computer science researchers, while the cognitive impact is a concern of researchers and health care professionals. In addition, our review revealed that telehealth usability for the care of patients with hypertension and diabetes was mainly studied by physicians and nurses. Understanding usability from the perspectives of other health professionals is relevant and important. Gender and age issues, which are known to influence telehealth usability, were also not addressed by most studies.It is important to point out that in the usability evaluation, we investigated usability criteria exclusively. We did not evaluate the quality of the content and functions, or the effectiveness of telehealth. Furthermore, it should be mentioned that a usability assessment cannot claim to cover all possible and critical usage situations that may occur.
Main Capabilities of Telehealth
The main positive points from the perspectives of health care professionals were that telehealth enabled the maintenance of care without the need for face-to-face contact, as well as enabled follow-up, co-management between primary care providers and specialists even in distant locations, communication between patients and health professionals, and health education. They also believed that telehealth was successful in influencing the attitudes and behaviors of patients and promoting self-management, with improvements in health conditions, cost savings, decision support, and flexibility regarding the health care system [
- ].Barriers to Telehealth
The biggest challenge for research in the field of telehealth is likely to be looking at obstacles and enablers for both health care professionals and patients. Low level of computer skills, lack of motivation, and little knowledge about the clinical utility of telehealth represent major cultural barriers to the routine use of telehealth, from the point of view of both the health professional and the patient.
Some providers pointed out that telehealth would not be suitable for exacerbations mainly because of the inability to perform a physical examination [
, , ]. Limited specific IT infrastructure, including regular power supply [ , ], internet connection [ , ], gaps in interoperability [ , , , , ], computer availability [ , , , ], and availability of specific staff for support and problem solving [ , , , ], were also barriers cited in the studies. However, gaps were generally larger for staff than for physical infrastructure, suggesting that, in addition to IT infrastructure, the shortage of essential personnel imposes significant constraints on the adoption of telehealth interventions. It has already been demonstrated that low technological competence [ , , ] and an inability to perform physical examination are the main barriers cited by health professionals in adopting telehealth [ , , , ]. As proposed by Nouri et al [ ], proactively exploring potential disparities in access to telehealth, developing solutions to mitigate barriers to digital literacy, removing barriers created by the health system for telehealth, and advocating for policies and infrastructure that facilitate equitable access to telehealth can ensure that the current implementation of telehealth does not exacerbate health disparities. Developing a strong evidence base for the use of telehealth will help to better understand how to overcome its challenges [ ].Strengths
This systematic literature review consulted 7 research databases to control for sample bias. We accepted only peer-reviewed publications to control validity, including manual searching in Google Scholar and gray literature, which helped to reduce publication bias and to improve the internal and external validity of the study. To control design bias, the systematic review protocol was registered in PROSPERO and previously published.
Limitations
The lack of standardization to describe usability was a key study limitation. Different authors used different terminologies to describe usability, which may have limited the inclusion of some articles in the review. Most of the studies evaluated usability through qualitative instruments, such as surveys, making it difficult to obtain objective information. These types of studies have important limitations due to the way they are constructed and the way they are interpreted by the respondents, which can cause difficulties in generalizing the data, therefore affecting its validity and reliability. Another limitation of the study was the impossibility of generalizing the results beyond the context of primary care in high- and middle-income countries, since only studies from high- and middle-income countries and from primary care were included. Future research should assess telehealth usability in low- and middle-income countries to build a broader picture of the challenges faced using telehealth across the world.
Implications of the Results for Practice, Policy, and Future Research
This systematic review showed that health care professionals who used telehealth for the care of patients with hypertension and diabetes from the COVID-19 pandemic onward considered the usability to be good, and most studies showed that health care professionals felt comfortable and satisfied with its use. Based on this, there are some policy implications and recommendations that can be derived from this research.
Improving health care for people with NCDs is a huge challenge. This challenge includes health promotion even in situations where face-to-face contact is not possible, as was the case with the COVID-19 pandemic. People with NCDs require ongoing health care and a team of health professionals with different modalities of care, such as follow-up, monitoring, intervention, rehabilitation, education, self-management learning, and consultation with specialists. Telehealth has the potential to alleviate these challenges, even after the pandemic. Understanding the usability from the perspectives of health professionals who frequently use telehealth systems (with weaknesses and potential) can facilitate the consolidation of telehealth, including expanding to those places where access to health services for people with chronic diseases is limited, such as low- and middle-income countries.
In general, concerns about the adequacy of data regarding the usability of telehealth by health professionals did not affect our confidence in interpreting the results. However, there is a need for research that investigates other factors that may influence perceptions of telehealth usability, such as differences between private and public services; differences in the level of experience of professionals, including professional experience and experience with digital tools; and differences in gender, age groups, occupations, and community settings (eg, rural and urban areas). Questions about internet connectivity quality, interface quality for the provision of telehealth, and previous training in telehealth practice should also be investigated as they may influence the perception of usability among health professionals who use telehealth systems.
Conclusion
Telehealth has been used for years to increase patient access to care and provide effective remote health care services. However, it was after the COVID-19 pandemic that telehealth prospered and emerged around the world as an indispensable resource for improving access to care, empowering patients, influencing their attitudes and behaviors, and ultimately enhancing their medical conditions.
Increasingly, the focus of chronic disease management is shifting to home and community settings, with telehealth quickly becoming the solution of choice. The favorable perception of professionals about telehealth systems in the treatment of NCDs points to the advancement of digital health experiences. The most significant barriers pointed out by health professionals who use telehealth to care for people with NCDs are related to empowering people for digital health. They pointed out that these barriers are greater than infrastructure barriers. The findings of this study can contribute to overcoming barriers and strengthening telehealth in the care of patients with NCDs beyond the period of the COVID-19 pandemic.
Acknowledgments
This research was funded by the Worldwide Universities Network (WUN RESEARCH PROJECTS), as part of the project “Leveraging digital healthcare experiences for post-pandemic non-communicable disease research - a multidisciplinary network engaging Brazil, Ghana, and the UK.” ALPR is supported in part by O Conselho Nacional de Desenvolvimento Científico e Tecnológico (465518/2014-1, 310790/2021-2, and 409604/2022-4), Universidade Federal de Minas Gerais, and Fundação de Amparo a Pesquisa do Estado de Minas Gerais (PPM-00428-17, RED-00081-16, and PPE-00030-21).
Data Availability
All data generated or analyzed during this study are included in this published article and its supplementary information files.
Authors' Contributions
RG, ASP, ZSNR, KB, SKA, MH, JB, and ALPR contributed to conceptualization, methodology, formal analysis, investigation, data curation, and writing, reviewing, and editing of the original draft. TCPL, SAC, and JMN contributed to conceptualization, investigation, and editing. All authors have read and agreed to the published version of the manuscript.
Conflicts of Interest
None declared.
Detailed search strategy.
DOCX File , 17 KB
Critical Appraisal Skills Program checklist.
DOCX File , 336 KB
Excluded studies and reasons for exclusion.
DOCX File , 92 KBReferences
- GBD 2015 Risk Factors Collaborators. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 2016 Oct 08;388(10053):1659-1724 [FREE Full text] [CrossRef] [Medline]
- Noncommunicable diseases. World Health Organization. URL: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases [accessed 2023-02-20]
- Nikoloski Z, Alqunaibet AM, Alfawaz RA, Almudarra SS, Herbst CH, El-Saharty S, et al. Covid-19 and non-communicable diseases: evidence from a systematic literature review. BMC Public Health 2021 Jun 05;21(1):1068 [FREE Full text] [CrossRef] [Medline]
- Kichloo A, Albosta M, Dettloff K, Wani F, El-Amir Z, Singh J, et al. Telemedicine, the current COVID-19 pandemic and the future: a narrative review and perspectives moving forward in the USA. Fam Med Community Health 2020 Aug;8(3):e000530 [FREE Full text] [CrossRef] [Medline]
- Bitar H, Alismail S. The role of eHealth, telehealth, and telemedicine for chronic disease patients during COVID-19 pandemic: A rapid systematic review. Digit Health 2021;7:20552076211009396 [FREE Full text] [CrossRef] [Medline]
- Zhu L, She Z, Cheng X, Qin J, Zhang X, Cai J, et al. Association of Blood Glucose Control and Outcomes in Patients with COVID-19 and Pre-existing Type 2 Diabetes. Cell Metab 2020 Jun 02;31(6):1068-1077.e3 [FREE Full text] [CrossRef] [Medline]
- Pan X, Yang J, Wen Y, Li N, Chen S, Pan A. Non-Communicable Diseases During the COVID-19 Pandemic and Beyond. Engineering (Beijing) 2021 Jul;7(7):899-902 [FREE Full text] [CrossRef] [Medline]
- Omboni S, McManus RJ, Bosworth HB, Chappell LC, Green BB, Kario K, et al. Evidence and Recommendations on the Use of Telemedicine for the Management of Arterial Hypertension: An International Expert Position Paper. Hypertension 2020 Nov;76(5):1368-1383. [CrossRef] [Medline]
- Smith AC, Thomas E, Snoswell CL, Haydon H, Mehrotra A, Clemensen J, et al. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020 Jun;26(5):309-313 [FREE Full text] [CrossRef] [Medline]
- Nouri S, Khoong EC, Lyles CR, Karliner L. Addressing Equity in Telemedicine for Chronic Disease Management During the Covid-19 Pandemic. NEJM Catalyst Innovations in Care Delivery 2020:123. [CrossRef]
- What is telehealth? How is telehealth different from telemedicine? Office of the National Coordinator for Health Information Technology. URL: https://www.healthit.gov/faq/what-telehealth-how-telehealth-different-telemedicine [accessed 2023-02-20]
- Predmore ZS, Roth E, Breslau J, Fischer SH, Uscher-Pines L. Assessment of Patient Preferences for Telehealth in Post-COVID-19 Pandemic Health Care. JAMA Netw Open 2021 Dec 01;4(12):e2136405 [FREE Full text] [CrossRef] [Medline]
- Kruse CS, Krowski N, Rodriguez B, Tran L, Vela J, Brooks M. Telehealth and patient satisfaction: a systematic review and narrative analysis. BMJ Open 2017 Aug 03;7(8):e016242 [FREE Full text] [CrossRef] [Medline]
- Drerup B, Espenschied J, Wiedemer J, Hamilton L. Reduced No-Show Rates and Sustained Patient Satisfaction of Telehealth During the COVID-19 Pandemic. Telemed J E Health 2021 Dec;27(12):1409-1415. [CrossRef] [Medline]
- Doraiswamy S, Abraham A, Mamtani R, Cheema S. Use of Telehealth During the COVID-19 Pandemic: Scoping Review. J Med Internet Res 2020 Dec 01;22(12):e24087 [FREE Full text] [CrossRef] [Medline]
- Powell RE, Henstenburg JM, Cooper G, Hollander JE, Rising KL. Patient Perceptions of Telehealth Primary Care Video Visits. Ann Fam Med 2017 May;15(3):225-229 [FREE Full text] [CrossRef] [Medline]
- Polinski JM, Barker T, Gagliano N, Sussman A, Brennan TA, Shrank WH. Patients' Satisfaction with and Preference for Telehealth Visits. J Gen Intern Med 2016 Mar;31(3):269-275 [FREE Full text] [CrossRef] [Medline]
- Larbi D, Randine P, Årsand E, Antypas K, Bradway M, Gabarron E. Methods and Evaluation Criteria for Apps and Digital Interventions for Diabetes Self-Management: Systematic Review. J Med Internet Res 2020 Jul 06;22(7):e18480 [FREE Full text] [CrossRef] [Medline]
- Alshamari M. Usability Factors Assessment in Health Information System. Intelligent Information Management 2016;8(6):170-180. [CrossRef]
- Ames HM, Glenton C, Lewin S, Tamrat T, Akama E, Leon N. Clients' perceptions and experiences of targeted digital communication accessible via mobile devices for reproductive, maternal, newborn, child, and adolescent health: a qualitative evidence synthesis. Cochrane Database Syst Rev 2019 Oct 14;10(10):CD013447 [FREE Full text] [CrossRef] [Medline]
- Odendaal WA, Anstey Watkins J, Leon N, Goudge J, Griffiths F, Tomlinson M, et al. Health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis. Cochrane Database Syst Rev 2020 Mar 26;3(3):CD011942 [FREE Full text] [CrossRef] [Medline]
- Klaassen B, van Beijnum BJF, Hermens HJ. Usability in telemedicine systems-A literature survey. Int J Med Inform 2016 Sep;93:57-69. [CrossRef] [Medline]
- ISO 9241-11:1998 Ergonomic requirements for office work with visual display terminals (VDTs) — Part 11: Guidance on usability. International Organization for Standardization. URL: https://www.iso.org/standard/16883.html [accessed 2023-02-20]
- Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane. URL: https://training.cochrane.org/handbook [accessed 2023-02-20]
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021 Mar 29;372:n71 [FREE Full text] [CrossRef] [Medline]
- Gonçalves RL, Pagano AS, Reis Z, Afagbedzi SK, Head M, Brackstone K, et al. Telehealth usability evaluation by healthcare professionals in post-pandemic non-communicable disease (hypertension and diabetes) treatment: a systematic review. PROSPERO. URL: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021296887 [accessed 2023-02-23]
- Gonçalves R, Pagano A, Reis Z, Kwaku Afagbedzi S, Head M, Brackstone K, et al. Telehealth usability evaluation by healthcare professionals in post-pandemic treatment of non-communicable diseases (hypertension and diabetes): Systematic Review Protocol. ppcrj 2022 Nov 28;8(2):43-51 [FREE Full text] [CrossRef]
- Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
- Lewin S, Bohren M, Rashidian A, Munthe-Kaas H, Glenton C, Colvin CJ, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table. Implement Sci 2018 Jan 25;13(Suppl 1):10 [FREE Full text] [CrossRef] [Medline]
- The Joanna Briggs Institute. Joanna Briggs Institute Reviewers' Manual 2014 Edition. Adelaide, Australia: The Joanna Briggs Institute; 2014.
- Critical Appraisal Skills Programme. URL: https://casp-uk.net/ [accessed 2023-02-20]
- Colvin CJ, Garside R, Wainwright M, Munthe-Kaas H, Glenton C, Bohren MA, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 4: how to assess coherence. Implement Sci 2018 Jan 25;13(Suppl 1):13 [FREE Full text] [CrossRef] [Medline]
- Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol 2019 Jun 04;19(1):113 [FREE Full text] [CrossRef] [Medline]
- Glenton C, Carlsen B, Lewin S, Munthe-Kaas H, Colvin C, Tunçalp Ö, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 5: how to assess adequacy of data. Implement Sci 2018 Jan 25;13(Suppl 1):14 [FREE Full text] [CrossRef] [Medline]
- Noyes J, Booth A, Lewin S, Carlsen B, Glenton C, Colvin CJ, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 6: how to assess relevance of the data. Implement Sci 2018 Jan 25;13(Suppl 1):4 [FREE Full text] [CrossRef] [Medline]
- Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci 2018 Jan 25;13(Suppl 1):2 [FREE Full text] [CrossRef] [Medline]
- Jordan DN, Jessen CM, Ferucci ED. Views of Patients and Providers on the Use of Telemedicine for Chronic Disease Specialty Care in the Alaska Native Population. Telemed J E Health 2021 Jan;27(1):82-89. [CrossRef] [Medline]
- Larbi D, Randine P, Årsand E, Bradway M, Antypas K, Gabarron E. Criteria for Assessing and Recommending Digital Diabetes Tools: A Delphi Study. Stud Health Technol Inform 2021 May 27;281:850-854. [CrossRef] [Medline]
- Varnfield M, Redd C, Stoney RM, Higgins L, Scolari N, Warwick R, et al. M♡THer, an mHealth System to Support Women with Gestational Diabetes Mellitus: Feasibility and Acceptability Study. Diabetes Technol Ther 2021 May 01;23(5):358-366 [FREE Full text] [CrossRef] [Medline]
- Lapão LV, Peyroteo M, Maia M, Seixas J, Gregório J, Mira da Silva M, et al. Implementation of Digital Monitoring Services During the COVID-19 Pandemic for Patients With Chronic Diseases: Design Science Approach. J Med Internet Res 2021 Aug 26;23(8):e24181 [FREE Full text] [CrossRef] [Medline]
- Yu CH, McCann M, Sale J. "In my age, we didn't have the computers": Using a complexity lens to understand uptake of diabetes eHealth innovations into primary care-A qualitative study. PLoS One 2021;16(7):e0254157 [FREE Full text] [CrossRef] [Medline]
- Li J, Varnfield M, Jayasena R, Celler B. Home telemonitoring for chronic disease management: Perceptions of users and factors influencing adoption. Health Informatics J 2021;27(1):1460458221997893 [FREE Full text] [CrossRef] [Medline]
- Alves L, Maia M, Araújo M, Damasceno M, Freitas R. Development and validation of a MHEALTH technology for the promotion of self-care for adolescents with diabetes. Cien Saude Colet 2021 May;26(5):1691-1700 [FREE Full text] [CrossRef] [Medline]
- Marcolino MS, Oliveira JAQ, Cimini CCR, Maia JX, Pinto VSOA, Sá TQV, et al. Development and Implementation of a Decision Support System to Improve Control of Hypertension and Diabetes in a Resource-Constrained Area in Brazil: Mixed Methods Study. J Med Internet Res 2021 Jan 11;23(1):e18872 [FREE Full text] [CrossRef] [Medline]
- Al-Sofiani ME, Alyusuf EY, Alharthi S, Alguwaihes AM, Al-Khalifah R, Alfadda A. Rapid Implementation of a Diabetes Telemedicine Clinic During the Coronavirus Disease 2019 Outbreak: Our Protocol, Experience, and Satisfaction Reports in Saudi Arabia. J Diabetes Sci Technol 2021 Mar;15(2):329-338 [FREE Full text] [CrossRef] [Medline]
- Yee LM, Leziak K, Jackson J, Strohbach A, Saber R, Niznik CM, et al. Patient and Provider Perspectives on a Novel Mobile Health Intervention for Low-Income Pregnant Women With Gestational or Type 2 Diabetes Mellitus. J Diabetes Sci Technol 2021 Sep;15(5):1121-1133 [FREE Full text] [CrossRef] [Medline]
- Salari R, R Niakan Kalhori S, GhaziSaeedi M, Jeddi M, Nazari M, Fatehi F. Mobile-Based and Cloud-Based System for Self-management of People With Type 2 Diabetes: Development and Usability Evaluation. J Med Internet Res 2021 Jun 02;23(6):e18167 [FREE Full text] [CrossRef] [Medline]
- Public Health Law Anthologies: Telehealth and Telemedicine. CDC. URL: https://www.cdc.gov/phlp/publications/topic/anthologies/anthologies-telehealth.html [accessed 2023-02-20]
- Kruse CS, Molina-Nava A, Kapoor Y, Anerobi C, Maddukuri H. Analyzing the Effect of Telemedicine on Domains of Quality Through Facilitators and Barriers to Adoption: Systematic Review. J Med Internet Res 2023 Jan 05;25:e43601 [FREE Full text] [CrossRef] [Medline]
- Institute of Medicine (US) Committee on Evaluating Clinical Applications of Telemedicine, Field MJ. Telemedicine: A Guide to Assessing Telecommunications in Health Care. National Library of Medicine. Washington, DC: National Academies Press (US); 1996. URL: https://www.ncbi.nlm.nih.gov/books/NBK45440/ [accessed 2023-02-20]
- Butcher L. Telehealth and telemedicine today. Physician Leadersh J 2015;2(3):8-13. [Medline]
- Board on Health Care Services; Institute of Medicine. The Role of Telehealth in an Evolving Health Care Environment: Workshop Summary - The Evolution of Telehealth: Where Have We Been and Where Are We Going? National Library of Medicine. Washington, DC: National Academies Press (US); 2012. URL: https://www.ncbi.nlm.nih.gov/books/NBK207141/ [accessed 2023-02-20]
- Barold SS. Willem Einthoven and the birth of clinical electrocardiography a hundred years ago. Card Electrophysiol Rev 2003 Jan;7(1):99-104. [CrossRef] [Medline]
- Russell J. Telehealth is disrupting traditional care model for doctors, hospitals. Indianapolis Business Journal. URL: https://www.ibj.com/articles/2021-innovation-issue-telehealth-is-disrupting-traditional-care-model-for-doctors-hospitals [accessed 2023-02-20]
- Carpentier WR, Charles JB, Shelhamer M, Hackler AS, Johnson TL, Domingo CMM, et al. Biomedical findings from NASA's Project Mercury: a case series. NPJ Microgravity 2018;4:6 [FREE Full text] [CrossRef] [Medline]
- Doarn CR, Latifi R, Poropatich RK, Sokolovich N, Kosiak D, Hostiuc F, et al. Development and Validation of Telemedicine for Disaster Response: The North Atlantic Treaty Organization Multinational System. Telemed J E Health 2018 Sep;24(9):657-668. [CrossRef] [Medline]
- Latifi R, Doarn CR, Zoicas C, Hostiuc F, Arafat R. cover Telemedicine for Disaster Management: Opportunities, Challenges, and Requirements. In: Doarn CR, Latifi R, Hostiuc F, Arafat R, Zoicas C, editors. Volume 130: A Multinational Telemedicine Systems for Disaster Response: Opportunities and Challenges. Amsterdam, The Netherlands: IOS Press; 2017:112-129.
- Zhao J, Zhang Z, Guo H, Li Y, Xue W, Ren L, et al. E-health in China: challenges, initial directions, and experience. Telemed J E Health 2010 Apr;16(3):344-349. [CrossRef] [Medline]
- Carayon P, Hoonakker P. Human Factors and Usability for Health Information Technology: Old and New Challenges. Yearb Med Inform 2019 Aug;28(1):71-77 [FREE Full text] [CrossRef] [Medline]
- Gomez T, Anaya YB, Shih KJ, Tarn DM. A Qualitative Study of Primary Care Physicians' Experiences With Telemedicine During COVID-19. J Am Board Fam Med 2021 Feb;34(Suppl):S61-S70 [FREE Full text] [CrossRef] [Medline]
- Xu J, Hamadi HY, Hicks-Roof KK, Zeglin RJ, Bailey CE, Zhao M. Healthcare Professionals and Telehealth Usability during COVID-19. Telehealth and Medicine Today 2021;6(3):1-12. [CrossRef]
- Zhang W, Cheng B, Zhu W, Huang X, Shen C. Effect of Telemedicine on Quality of Care in Patients with Coexisting Hypertension and Diabetes: A Systematic Review and Meta-Analysis. Telemed J E Health 2021 Jun 01;27(6):603-614. [CrossRef] [Medline]
- Santos DS, Batistelli CRS, Lara MMDS, Ferreira EDS, Moreira TR, Cotta RMM. The effectiveness of the use of telehealth programs in the care of individuals with hypertension and, or diabetes mellitus: systematic review and meta-analysis. Diabetol Metab Syndr 2022 May 28;14(1):76 [FREE Full text] [CrossRef] [Medline]
- Gomes-de Almeida S, Marabujo T, do Carmo-Gonçalves M. [Telemedicine satisfaction of primary care patients during COVID-19 pandemics]. Semergen 2021;47(4):248-255 [FREE Full text] [CrossRef] [Medline]
- Giani E, Dovc K, Dos Santos TJ, Chobot A, Braune K, Cardona-Hernandez R, ISPAD Jenious Group. Telemedicine and COVID-19 pandemic: The perfect storm to mark a change in diabetes care. Results from a world-wide cross-sectional web-based survey. Pediatr Diabetes 2021 Dec;22(8):1115-1119 [FREE Full text] [CrossRef] [Medline]
- ISO 25010 Certification - Software Quality Model. GMS Quest. URL: https://www.gmsquest.com/certifications/iso-25010-certification/ [accessed 2023-02-20]
- Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health 2020 Aug 01;20(1):1193 [FREE Full text] [CrossRef] [Medline]
- Vera F, Noël R, Taramasco C. Standards, Processes and Instruments for Assessing Usability of Health Mobile Apps: A Systematic Literature Review. Stud Health Technol Inform 2019 Aug 21;264:1797-1798. [CrossRef] [Medline]
- Cruz Zapata B, Hernández Niñirola A, Idri A, Fernández-Alemán JL, Toval A. Mobile PHRs compliance with Android and iOS usability guidelines. J Med Syst 2014 Aug 24;38(8):81. [CrossRef] [Medline]
- Guo XT, Yuan JQ, Cao XF, Chen XD. Understanding the acceptance of mobile health services: A service participants analysis. 2012 Presented at: 19th Annual International Conference on Management Science & Engineering; September 20-22, 2012; Dallas, TX, USA. [CrossRef]
- Mahmoud K, Jaramillo C, Barteit S. Telemedicine in Low- and Middle-Income Countries During the COVID-19 Pandemic: A Scoping Review. Front Public Health 2022 Jun 22;10:914423 [FREE Full text] [CrossRef] [Medline]
Abbreviations
CASP: Critical Appraisal Skills Program |
GRADE-CERQual: Grading of Recommendations Assessment, Development and Evaluation - Confidence in Evidence from Qualitative Research Reviews |
IT: information technology |
JBI: Joanna Briggs Institute |
NASA: National Aeronautics and Space Administration |
NCD: noncommunicable disease |
PEOTS: patient/population-exposure-outcome-time-study design |
Edited by A Mavragani; submitted 10.11.22; peer-reviewed by J Martínez-Miranda, H Alsaqqa; comments to author 03.01.23; revised version received 25.01.23; accepted 06.02.23; published 16.03.23
Copyright©Roberta Lins Gonçalves, Adriana Silvina Pagano, Zilma Silveira Nogueira Reis, Ken Brackstone, Tainá Costa Pereira Lopes, Sarah Almeida Cordeiro, Julia Macedo Nunes, Seth Kwaku Afagbedzi, Michael Head, Wagner Meira Jr, James Batchelor, Antônio Luiz Pinho Ribeiro. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 16.03.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.