Analyzing the Effect of Telemedicine on Domains of Quality Through Facilitators and Barriers to Adoption: Systematic Review

Introduction

Rationale

The World Health Organization defines telemedicine and telehealth as healing at a distance through the use of information communication technologies to improve health outcomes [1]. The World Health Organization does not distinguish between telemedicine and telehealth; therefore, these terms may be used interchangeably in this study. Mobile health and eHealth enable the practice of medicine and public or population health through mobile devices such as phones, tablets, or patient monitoring devices [2]. Mobile devices have blurred the lines between computers and tablets because the processing power of the 2 have become similar. Many apps work in the same manner on these 2 modalities.

There is no exaggeration to correlate advances in technology with advances in telemedicine. Over the last century, technological advances have connected the world in ways never before thought possible. Once technology enabled communication at a distance, it enabled healing at a distance. The telegraph has even been named the “Victorian Internet” by scholars and was used during the American Civil War to send reports about wounded soldiers to medical teams [3]. Radio and telephone were the next technological advances in communication, and these devices continued the practice of healing at a distance, such as consultations with ships at sea [4]. The modern-day internet and cloud storage have made our world smaller, but the adoption of telemedicine is still not universal.

The COVID-19 pandemic continues to teach the medical community many lessons, but one important lesson is that the modality of telemedicine is possible across a spectrum of services [5] and patients will accept it [6]. For those practices that had not already adopted telemedicine, providers adopted this modality owing to the restriction on face-to-face encounters, and the result was positive; patients were satisfied with the services offered, some providers identified improvements in efficiency, and outcome effectiveness was equally, if not better, than traditional care [6]. However, some providers are still reluctant to adopt telemedicine owing to quality concerns.

Health care quality is a broad but measurable concept. In 1999, the Institute of Medicine defined 6 domains of quality: safe, effective, patient-centered, timely, efficient, and equitable [7]. Safe is avoiding harm. Effective is providing evidence-based care and avoiding the underuse and misuse of medical services. Patient-centered is respecting patient autonomy. Timely is the reduction of wait times. Efficient is the avoidance of waste. Equitable is care that does not vary in the face of personal characteristics [7]. These definitions provide measurable data points.

Telemedicine and its quality have been examined from a specialty point of view, but there has not been a comprehensive look across specialties. Telemedicine has been studied for its quality implications in diabetes [8]; liver disease [9]; pediatrics [10]; gastroenterology [11]; ears, nose, and throat [12]; respiratory care [13]; rheumatoid arthritis [14]; and alcohol use disorder [15]. Each study provides a mix of reviews on quality [16-18].

A systematic review was published in 2020 that examined telemedicine use across multiple service lines in the United States [5]. It analyzed 44 studies over a 5-year period. This review highlighted the agility of the health system of United States in rapidly adopting telemedicine in the face of the pandemic, but it did not report on quality outcomes. It highlighted important concepts for consideration such as risk management, compliance, cost, and patient satisfaction.

A systematic review published in 2022 examined the effect of telemedicine on the quality of care in patients with hypertension and diabetes [19]. This review analyzed 5 studies conducted over 3 years. This review focused on the measurement of effectiveness of medical outcome in the areas of hypertension and diabetes and on patient satisfaction. Telemedicine was found to significantly improve the experience of care and care engagement.

Objectives

The purpose of this review was to analyze the effectiveness of telemedicine on quality of care across a spectrum of specialties around the world in studies published over the last year, to date, in academic, peer-reviewed journals, using a randomized controlled trial (RCT) or true experiment as the methodology.

Methods

Eligibility Criteria

The eligibility criteria for this review were as follows: human participants, published in 2022, published in a peer-reviewed journal, and RCTs, but not reviews. Other systematic reviews were excluded because they had already analyzed studies that could also be included in our review. Their exclusion helped to avoid confounding the results. All reports were in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 [20].

Information Sources

The information sources were PubMed (MEDLINE), CINAHL, Web of Science, and ScienceDirect. The databases were searched on September 19, 2022. These databases were chosen because they were readily available to health care researchers and enabled other researchers to duplicate this study. To eliminate duplicates, MEDLINE was excluded from all databases except PubMed.

Search Strategy

A Boolean search string was assembled from the keywords provided by the Medical Subject Headings of the United States. Library of Medicine: (tele* OR mhealth) AND (quality OR safe* OR effective* OR timeliness OR “patient centered” OR equitable). The same search string was used for all databases that allowed wildcards. Where wildcards were not allowed, the following search string was used: (telemedicine OR mhealth) AND (quality OR safe OR effective OR timeliness OR “patient centered” OR equitable). Similar filter strategies were used in all databases, because not all databases offered the same filtering tools.

Selection Process

Following the Kruse protocol, we searched for key terms in all databases, filtered the results, and screened abstracts for applicability [21]. At least two reviewers screened each abstract and analyzed each article. The standard PRISMA diagram was created, as required by the PRISMA standard [20]. Only studies that used the RCT were included in the meta-analysis. Once all filtering and screening were completed, each article was assigned a random number using Microsoft Excel’s random number generator. The first 33 studies were chosen for analysis.

Data Collection Process

A standardized Excel spreadsheet from the Kruse protocol was used as a data extraction tool to collect additional data at each step of the process [21]. We used a series of 3 consensus meetings to identify articles for full analysis, extract data, and identify themes for analysis.

Data Items

We collected the following fields of data for each step: Google Scholar search (date of publication, authors, study title, journal, impact factor from Journal Citations Reports, study design, key terms, experimental intervention, results, and comments from each reviewer); filter articles step (the number of results before and after each filter was applied in all 4 databases); abstract screening step (database source, date of publication, authors, study title, journal, screening decision for each reviewer, notes about rejections, consensus meeting one, determination of screening decision, and a set of rejection criteria); analysis step (database source, date of publication, authors, study title, participants, experimental intervention, results compared with a control group, medical outcomes, study design, sample size, bias effect size, country of origin, statistics used, the strength and quality of evidence patient satisfaction, facilitators to adoption, barriers to adoption, and domains of quality). All but the last 4 data items were standard fields on the standardized Microsoft Excel spreadsheet, whereas the last 4 items were specific to the research objective [21].

Study Risk and Reporting of Bias Assessment

During the data extraction process, reviewers noted individual cases of bias such as sample bias. We combined individual cases of bias with the quality assessment of each study using the Johns Hopkins Nursing Evidence-based Practice (JHNEBP) tool [22]. The strength of evidence was defined by the JHNEBP as level I studies, RCTs or true experiments (with controls and randomization); level II studies, quasi-experimental (control group, but no randomization); level III studies, observational, qualitative, or other nonexperimental methods; and levels IV and V are opinions. Levels IV and V were not considered in this study. We considered instances of bias when interpreting the results because bias can limit external validity [23].

Effect Measures

Our preferred measure of effect was the Cohen d, but other measures were accepted. Measures of effect are summarized in tables for the studies in which they were reported. Measures of effect were reported as Cohen d, odds ratios, and β. For studies that reported an effect size, a weighted average effect size was calculated [24]. A Cohen κ statistic was also calculated to measure agreement between reviewers [25,26].

Synthesis Methods

Reviewers performed a thematic analysis to help make sense of the extracted data [27]. The same or similar observations were consolidated into themes. These themes and individual observations that did not fit into themes were tabulated into affinity matrices for further analysis. The frequency of observations was reported not to imply importance or priority but only to measure the probability of encountering the theme in the group of studies under analysis.

Additional Analyses and Certainty Assessment

We tabulated the effect sizes during data extraction. Certainty assessments were performed by considering both the narrative analysis and effect size. We calculated the frequency of occurrence of each theme and reported these frequencies in affinity matrices. Frequency reporting provided confidence in the analyzed data.

Results

Overview

Figure 1 illustrates the study selection process using the PRISMA flow diagram [20]. The query from the 4 databases returned 882,420 results, of which 195,572 were duplicates. The date range and other filters reduced the group to 342 articles for screening. After the screening, 97 studies were included in the analysis. We assigned random numbers to these 97 and chose the highest 33 for data extraction and analysis. Figure 1 also illustrates the articles filtered out for weak methodology if the studies did not use an RCT study design. A ĸ statistic was calculated to reflect the level of agreement between the reviewers (ĸ=0.90, strong agreement) [25,26].

‎

Study Selection and Characteristics

Following the PRISMA checklist and the Kruse protocol for each study, we extracted the data fields for a Participants, Intervention, Comparison (with the control), Outcome (medical), Study Design table: participants, intervention, comparison (with control or other group), observation, and study design (Table 1). This was performed to summarize the study characteristics in a manner established in the literature. Of the 33 studies analyzed, all were published over a 1-year period [28-60]. Only 6% (2/33) of the studies involved participants younger than 18 years [40,47]. The rest of the studies included participants aged from 18 to 85 years. A total of 5 studies helped participants manage diabetes, 3 helped those living with HIV manage symptoms and relationships, 3 helped participants manage mental health conditions, 2 helped manage hypertension, 2 heart disease, 2 pain management, 2 activity management, and 2 cancer recovery. The rest helped manage tuberculosis, alcohol consumption, smoking cessation, preventive medicine services like vaccinations and tests, stroke recovery, nutrition autism management, Alzheimer disease, and telerehabilitation. More than half (17/33, 52%) of the studies were mobile health, eHealth (10/33, 30%), telephone (3/33, 9%), or general telemedicine (3/33, 9%). All studies demonstrated positive effects, but not all were statistically different from traditional methods of care. The studies originated from 16 countries: China (5/33, 15%), the United States (4/33, 12%), Australia (3/33, 9%), Bangladesh (2/33, 6%), Columbia (2/33, 6%), Germany (2/33, 6%), Japan (2/33, 6%), Spain (2/33, 6%), the United Kingdom (2/33, 6%), Brazil (1/33, 3%), Nepal (1/33, 3%), the Netherlands (1/33, 3%), Norway (1/33, 3%), Sweden (1/33, 3%), Taiwan (1/33, 3%), and Turkey (1/33, 3%).

Risk of Bias in and Across Studies

The JHNEBP quality assessment tool identified 100% (33/33) of the studies as level I and level A because all but RCTs were screened out. The JHNEBP tool assessed the strength of evidence as levels I to V: I is an RCT or experiment; II is quasi-experimental; III is qualitative or observational; and IV and V are opinion articles. The JHNEBP tool assessed the quality of evidence as A-C: A was defined by consistent results with adequate sample and control sizes (based on a power analysis), definitive conclusions, and consistent recommendations based on extensive literature reviews. Level B was defined by reasonably consistent results, adequate sample and control sizes, definitive conclusions, and recommendations. Level C was defined by little evidence with inconsistent results, insufficient sample sizes, and nondefinitive conclusions.

Reviewers also noted instances of bias, such as sample and selection bias, because these affect external and internal validity, respectively. There were 33 instances of selection bias and 32 of sample bias. Selection bias was identified when samples were taken from one locality, city, or country. Selection bias was identified when the sample comprised a majority of one sex or race.

Results of Individual Studies

Table 2 summarizes the results of the individual studies through themes. Several themes are repeated in this table because there were multiple observations in the same study that qualified under these themes. For instance, the theme of improved mental health included improvements in anxiety, mental well-being, stress, loneliness, depression, fear, personal satisfaction, helplessness, rumination, acceptance, resilience, and suicidal ideation. Multimedia Appendices 1 and 2 [28-60] provide an observation-to-theme match for all studies. Multimedia Appendix 3 [28-60] provides other data fields collected during the data extraction phase of the systematic literature review.

Results of Syntheses, Additional Analysis, and Certainty of Evidence

Overview

Thematic analysis was performed to help make sense of the data collected. Themes and individual observations that did not fit the themes were tabulated. The mean sample size was 351.7 (SD 501.1). A total of 11 studies reported the effect sizes [29,31,35-37,41,51,54-57]. Odds ratios were converted to Cohen d and a weighted average effect size was calculated (0.21, small effect) [61,62].

Results of Telemedicine and Quality

Table 3 summarizes the results. The reviewers identified 7 themes and 3 individual observations for 92 occurrences in the literature. As these were the result of a thematic analysis, there are cases of multiple observations of the same theme in the same article; therefore, the frequency may not have aligned with the number of references. Of the 92, there were 31 (34%) instances of an increase in physical health [30,33,39-42, 44,47,49,51-55,57,58]. This theme included maximum oxygen consumption, comparison of bad cholesterol with good cholesterol, pain, diastolic blood pressure, systolic blood pressure, hypoglycemia, lipids, overall blood pressure, average blood sugar over last 3 months, physical function, fasting blood glucose, cholesterol, and BMI. There were 17% (16/92) of instances of increased mental health [31,32,37,39,43,45,47, 54,57]. This theme encompassed anxiety, well-being, stress, loneliness, depression, fear, personal satisfaction, helplessness, rumination, acceptance, suicidal ideation, and resilience. There were 13% (12/92) of instances of improved medical engagement [28,30,32,35,36,40,43,46,48,60]. This theme included medication compliance, program or treatment adherence, follow-up visits, medical knowledge, and decrease in emergency department visits. There were 12% (11/92) of instances of 2 themes: changed behavior, which included sexual behavior, self-care, drinking, smoking, physical activity, weight loss, and salt intake [29,32,34,50,53,55,56,58-60] and increased quality of life (QoL), which included sleep, vitality, interpersonal problems, attitude, or as measured by the EuroQoL visual analog scale [31,36,37,39-41,45,54,56,57,59,60]. There were 7% (6/92) of instances of increased self-efficacy [28,35,36,41,47]. This theme included an increase in self-care, vaccinations, and preventive measures, such as Papanicolaou smears, colorectal exams, and mammograms. There were 2 instances of increased social support, which included a reduction in web-based discrimination [28,32]. A total of 3 observations did not fit any themes and each occurred only once in the literature: fewer immune-related adverse events (for cancer patients), increased quality-adjusted life years, and reduced costs [34,38,60].

Medical Outcomes of Telemedicine and Quality

Table 4 summarizes the observed medical outcomes. The reviewers identified 7 themes and 2 individual observations for 86 occurrences in the literature. The results compared with the control group and the medical outcomes were highly similar, but they focused on themes and observations relevant to the provider. Some results did not directly correlate with medical outcomes; therefore, the numbers were not exactly the same.

Satisfaction Associated With the Adoption of Telemedicine

A total of 24 studies reported on satisfaction. Of the 33 studies, 22 (67%) reported satisfaction or high satisfaction [28-36,39,40,42-44,46-48,50,56-59], 2 (6%) reported dissatisfaction [38,45], and 9 (27%) did not report satisfaction or dissatisfaction [37,41,49,51-55,60].

Facilitators to the Adoption of Telemedicine and Quality Implications

Table 5 summarizes the observed facilitators. The reviewers identified 7 themes and 1 individual observation for 166 occurrences in the literature. All 33 (100%) studies reported that the intervention was an effective as modality of treatment [28-36,38-55,57-60]. A digital preference was observed in 97% (32/33) of studies [28-34,36-60]. Convenience was observed in 94% (31/33) of studies [28-30,32-60]. The authors noted the convenience of telemedicine and its ability to be available anytime, anywhere. Telemedicine patients did not need to take off work, drive to a clinic, find a place to park, wait for appointments, and personally expose themselves to the germs in the medical environment [28-31,33,34,36-60]. These savings in time and mileage were observed in 91% (30/33) of studies. In addition, some patients preferred a technologically oriented mode of care or delivery [28-31,33,34,36-60]. Patients valued technology and saved time and mileage in 91% (30/33) of studies. Many studies involved an educational dimension to the intervention. Patients appreciated the telemedicine modality for medical education because it allowed them to absorb or learn at their own pace. This was observed in 21% (7/33) of studies [28,30,38,39,44,52,53]. In 6% (2/33) of studies, one on alcohol consumption and one on HIV management, avoidance of stigma was mentioned [29,32]. Finally, patients valued the personal guidance of a telephone navigator. This was observed in 3% (1/33) of studies [35].

Barriers to the Adoption of Telemedicine and Quality Implications

Table 6 summarizes the observed barriers. The reviewers identified 4 themes for 93 occurrences in the literature. Of the 33 studies, the need for staff training appeared in 94% (31/33) of the studies [28-34,36-60]. Thus, telemedicine may not be the preferred modality of care. This was observed in 91% (30/33) of the studies [28-30,32-34,36-60]. The cost of acquiring the servers to manage telemedicine, apps on mobile and computer platforms, and phones themselves were significant barriers. This was observed in 88% (29/30) of the studies [29-37,39-42,45-60]. Finally, in countries where reimbursement was an issue, the rate was lower for telemedicine than for traditional modalities of care. This was observed in 9% (3/33) of the studies [32,35,37].

Domains of Quality Incident to the Adoption of Telemedicine

Table 7 summarizes the domains of quality observed in the adoption of telemedicine. Of the 6 domains of quality, 4 observed in all (33/33, 100%) the studies: safe, effective, patient-centered, and timely [28-60]. Efficient was identified in 97% (32/33) of the studies [29-60]. Equitable was only identified in 6% (2/33) of the studies because of the digital divide that often falls on socioeconomic lines [32,38]. The results of these 2 studies were collected and reported on racial and socioeconomic lines.

Discussion

Summary of Evidence

Commensurate with the objective statement, this systematic literature review analyzed 33 RCT studies from 16 countries published in 2022, to date, to analyze the effectiveness (weighted average effect size 0.21, small) of telemedicine through the lens of 6 domains of quality. All these 33 studies reported the positive effectiveness of telemedicine as a modality across all 6 domains of quality. These studies showed positive outcomes in physical [30,33,39-42,44,47,49,51-55,57,58] and mental health [31,32,37,39,43,45,47,54,57], medical engagement [28,30,32,35,36,40,43,46,48,60], changed behavior [29,32,34, 50,53,55,56,58-60], increased QoL [31,36,37,39-41,45,54, 56,57,59,60], increased self-efficacy [28,35,36,41,47], increased social support [28,32], and reduced costs [34].

Patient engagement is important because it plays a central role in patient safety, chronic disease self-management, adverse event reporting, and medical record accuracy [63]. It also affects health literacy and shared decision-making [64]. Changing patients’ behavior is difficult, and advances in this area often require motivational interviewing [65]. Leveraging telemedicine to increase shared decision-making contributed to behavioral changes in about a third of the studies analyzed. An increase in health-related QoL was also an important conclusion. This facet of health care has become especially important during the COVID-19 pandemic [66]. Finally, leveraging telemedicine to reduce the cost burden is commensurate with other literature [67]. Telemedicine reduces miles driven, time taken off work, and childcare expenses, while maintaining high-quality outcomes [67].

Telemedicine was effective for patients. Studies reviewed in this study mentioned that it is effective [28-36,38-55,57-60], and it meets the digital preference of patients [28-34,36-60] because many patients value technology [28-31,33,34,36-60]. The pandemic has taught health care that telemedicine increases patients’ perception of the availability of care and most patients prefer this modality [68]. It is convenient, saves time and mileage [28-31,33,34,36-60], enables education at one’s own pace [28,30,38,39,44,52,53], avoids stigma [29,32], and provides personal navigation through the health care system. These results serve as strong facilitators for the adoption of telemedicine because they show strong quality results in favor of patient commensurate with other published literature [6].

There are several barriers to telemedicine adoption. Staff must be trained in delivering care through telemedicine to ensure that quality does not decline [28-34,36-60]. Patients must be asked if telemedicine is acceptable because it may not be their preferred modality of care [28-30,32-34,36-60]. Hardware and software costs are associated with enabling this modality care [29-37,39-42,45-60]. The cost of telemedicine infrastructure is consistent with published literature [6]. Finally, in countries where reimbursement remains a consideration, there are low rates of reimbursement for this modality of care [32,35,37]. These results serve as barriers to the adoption of telemedicine, which can be addressed through policies and incentives.

Of the 6 domains of quality, 4 (67%) were identified in all of the analyzed studies: safe, effective, patient-centered, and timely. Efficiency was only mentioned in 97% (32/33) of studies and equitable in only 6% (2/33) of studies. This is largely owing to the technology gap that occurs along socioeconomic lines. This disparity has been identified in other literature [69]. Identifying all 6 domains of quality in the literature also serves as a strong indicator of the positive effect incurred through the modality of telemedicine, and it serves as another facilitator to its adoption commensurate with the literature [70]. The treatment results were not always statistically different from treatment as usual; however, in every case, the treatment modality still resulted in a positive effect on symptoms, conditions, or behavior. This was an important finding because even if a treatment modality was not significantly better than treatment as usual, it might meet the digital preference of a patient.

Future research should expand some of these RCTs to help firmly establish telemedicine as an acceptable modality of care. This systematic literature review analyzed only 33 studies, but these studies focused on a wide range of specialties: tuberculosis, hypertension, alcohol consumption, mental health, HIV management, heart disease, smoking cessation, preventive medicine, stroke rehabilitation, nutrition, pain management, autism behavior management, diabetes management, Alzheimer disease, activity management, telerehabilitation for physical activity, and cancer recovery. Further research could expand on these specialties to identify where telemedicine is not an acceptable modality of care. After a family of systematic reviews was published, a review of these reviews summarized the effectiveness of telemedicine across all aspects of care.

This study has both practical and policy implications. Health care administrators should be confident in the investment of technology infrastructure to support the modality of telemedicine. The pandemic introduced transformational telehealth adoption, and restrictive regulations on modality were lifted [71]. Telemedicine is scalable and enables the web-based expansion of clinics without physically expanding the health care plant [71]. Providers should feel confident in the continued provision of telemedicine in their practice because it is rapidly becoming a preference for patients, even older adults, despite the technology gap [72,73]. Policy makers should encourage the modality of telemedicine because it increases access to care and saves patients the cost of travel and time off work [74].

Limitations

This systematic literature review queried 4 research databases to control for sample bias. Additional research databases can also be queried. We only accepted published peer-reviewed literature to control for validity. Accepting gray literature could have better controlled for publication bias, but it may have introduced questionable internal and external validity. Our team has identified several instances of selection and sample bias. Our assessment was that their effect was small. However, it is possible that these instances could have presented significant challenges to both internal and external validity. To control for design bias, this systematic literature review used a previously published protocol. Other protocols could have been used. This review queried only 10 months of 2022 and only 33 articles were analyzed. Additional years and articles could have yielded more robust results.

Conclusions

Telemedicine serves as an effective modality of care for a wide range of medical services, and its effectiveness has been demonstrated across all 6 domains of quality. These interventions have a positive effect on physical and mental health, engagement with the medical community, changed behavior, increased QoL, self-efficacy, and social support. This modality is patient-centered because it puts the patient’s schedule before the providers, saves time and mileage, avoids the stigma of care associated with some clinics, and patients often prefer it. The results of this systematic review should enable providers to adopt telemedicine as a standard option of care for patients. Studies with robust designs have shown telemedicine to be an effective modality of care, and it falls within the preference of many patients. Administrators should be confident in investing in technology to enable this modality of care. Policy makers should focus on removing the barriers to adoption.