Patient Experience in Virtual Visits Hinges on Technology and the Patient-Clinician Relationship: A Large Survey Study With Open-ended Questions

Introduction

Virtual visits have emerged at a time in health care when clinicians’ decisions have become only one part of patients’ experiences. Patients’ and their families’ needs, values, and inputs influence care decisions [1]. Patients are researching treatments on the internet and engaging as advocates and even as partners with physicians both on the internet and in person [1-5]. A crucial aspect of patient experience is the patient-clinician engagement or using a relationship-centered care approach. This approach prioritizes how the communication and emotions shared between patients and clinicians gradually affect patients’ experiences as a whole person [5,6]. Furthermore, whether physicians empathize with their patients strongly shapes the patient-physician relationship and patient satisfaction [7]. Virtual visits must continue the best aspects of in-person care to be an acceptable substitute for in-person care. However, no studies have evaluated whether virtual visits are addressing patients’ experiences comprehensively rather than simply whether or not they are satisfied.

Studies on virtual visits have examined patient satisfaction. For example, patients have reported positive experiences about communication, engagement, and building rapport with clinicians during their virtual visits [8-12]. Furthermore, patients have expressed that virtual visits are convenient in decreasing travel time, wait time, and stress and increasing comfort and convenience [13-15]. Virtual visits increased substantially during the initial phase of the COVID-19 pandemic, and physicians reported that patients respond positively to virtual visits [16], that telemedicine addresses both urgent and nonurgent care [17], and that telehealth has become a valuable way to evaluate patients before they enter a medical facility, thereby limiting potential clinician or patient exposure to COVID-19 [18]. Despite the increasing use of telehealth during the COVID-19 pandemic, these studies have not evaluated whether virtual visits actually serve the patient-clinician relationship [16-18]. In addition, studies that have examined patient satisfaction with virtual visits are mostly specialty- or disease-specific [19-21], have small sample sizes or low survey response rates [9,10,21,22], or include only one item to measure patient satisfaction [13].

To gain a more comprehensive understanding of the patient experience during virtual visits, especially patient-clinician engagement in a range of types of virtual visits, we conducted a survey study with patients who received virtual care from a variety of clinicians representing different practices.

Methods

We conducted a large survey study that included open-ended questions to comprehensively assess patients’ experiences with virtual visits in the Cleveland Clinic Health System. This cross-sectional study was conducted with adults who had a synchronous virtual visit between June 21, 2017, and July 12, 2017. These visits took place on a synchronous video-visit platform developed by the American Well [23]. The study examined scheduled and on-demand virtual visits that used audio and video (telephonic only were not available). With this app, patients can locate and connect with clinicians in an on-demand fashion, and clinicians can also schedule visits in advance so that patients can simply click on a link at the designated time to connect directly with their clinician. Visits were termed an Express Care Online visit because patients could access a clinician quickly, though still comprehensively receive help. The user interface facilitates virtual visits over smartphones, tablets, and computers [23].

Eligible patients were identified by daily census reports from the previous day’s virtual visits. When patients had more than one virtual visit during study accrual, only the first visit was included. In addition, visits used for clinician training, digital health testing, and those with a duration of 0 minutes were excluded from the study. All 648 eligible patients with visits during that period were sent an email invitation to complete a brief web-based survey regarding their experiences with their recent virtual visit. All nonresponders were sent reminder email invitations and were then called by a member of the research team. Patients were given a US $5 Amazon e-gift card upon completion of the survey. As part of their participation, they were informed about the gift card at the time of recruitment. Given the small amount of compensation, participation bias is likely to be minimal.

Survey items were designed to consider patients as whole persons, evaluate the patient-clinician relationship, and broadly assess patients’ experiences with virtual visits; specifically, we sought to assess patient’s overall satisfaction, overall ease, technical ease, convenience, relationship with the clinician, and perception of clinician empathy. The survey also aimed to evaluate how patients compared virtual visit quality and value with in-person visits. Survey questions were developed after an extensive review of the literature [14,15,24] and several iterations of expert stakeholder input from members of the digital health team, the office of patient experience, and patient volunteers. The survey consisted of 21 items evaluated on a 5-point Likert scale (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, and 5=strongly agree) and text boxes for comments on what patients liked best about their virtual visit and what could be improved. The survey items are presented in Multimedia Appendix 1.

Survey data were collected and managed using the REDCap (Research Electronic Data Capture) electronic data capture system [25]. Survey data were merged with data from the hospital system that included virtual visit characteristics, such as visit type (scheduled or on-demand), device type (computer or mobile device), clinician type (physician or advanced practice provider), clinician specialty, insurance type, and patient demographic information.

First, we computed descriptive statistics. Factor analysis of the 21 survey items revealed three factors: overall satisfaction with the virtual visit (standardized Cronbach α=.93), comfort and ease using virtual visit technology (Cronbach α=.89), and patient-clinician engagement (Cronbach α=.92). Items associated with each factor were summed and dichotomized at the mean for analysis. We used multivariable logistic regression with backward variable selection to assess the associations among patient demographics; virtual visit clinician type and prior patient relationship with their virtual visit clinician; and overall satisfaction with their virtual visit, patient-reported experience using the virtual visit technology, and patient-clinician engagement. Quantitative data were analyzed using SAS 9.4 [26].

For the open-ended questions, a framework analysis of the content was used to identify themes in patients’ narrative comments. Themes were used to create a coding checklist consisting of dichotomous (yes or no) variables. Themes that emerged from comments in response to what patients appreciated about their virtual visit included not having to travel outside home to receive care, enhanced access to care, and convenience. Themes related to recommendations for improvements included enhancements to the user interface and instructions to better prepare for virtual visits. A coding checklist was used to code the text data quantitatively and descriptive statistics were computed.

This study was approved by our medical center’s institutional review board.

Results

Overview

Out of the 426 patients who responded to the survey (426/648, 65.7%), nearly two-thirds self-identified as female (273/426, 64.1%). The average age was 46 (range 18-86) years; 70.2% (299/426) of the patients were from Ohio, 6.8% (29/426) were from Florida, 4.2% (18/426) were from Pennsylvania, and 18.7% (80/426) resided in other states. Insurance coverage was undetermined for more than half of the patients (245/426, 57.5%), 23.7% (101/426) had the hospital’s employee health insurance, and 18.7% (80/426) had other private insurance. The demographic characteristics of the respondents were similar to those of the nonrespondents.

A total of 58.4% (249/426) patients had an on-demand virtual visit for an acute concern, and 41.5% (177/426) had a scheduled virtual visit; 28.6% (121/423) of patients knew their virtual visit clinician from a previous in-person appointment with that same clinician, and those patients were older (P=.003) and more often male (P=.03) compared with patients who did not know their virtual visit clinician. Moreover, most patients who knew their virtual visit clinician had a scheduled rather than on-demand virtual visit (119/121, 98.3%). About 41.8% (178/426) of clinicians were family physicians, whereas the rest represented other physician specialties, and 20.9% (89/426) were advanced practice providers. Most patients used mobile devices such as phones or tablets for their visits (348/424, 82.1%), whereas the rest were connected using a computer. Patient and virtual visit characteristics are presented in Table 1.

Overall Satisfaction

The mean overall satisfaction score was 4.4 out of 5 with an SD of 0.78 (see Table 2 for detailed Likert scale survey results). Overall, 81.9% (344/420) of respondents agreed (107/420, 25.5%) or strongly agreed (237/420, 56.4%) that their virtual visit was as good as an in-person visit with a clinician. More than half of the respondents agreed (73/417, 17.5%) or strongly agreed (149/417, 35.7%) that their virtual visit was better than an in-person visit with a clinician. In multivariable logistic regression, employee patient status (vs nonemployee) was associated with higher odds of overall satisfaction with their virtual visit (odds ratio [OR] 1.9, 95% CI 1.14-3.47). The results from the multivariable logistic regression are presented in Table 3.

Most patients (383/421, 90.9%) reported that their virtual visit made it easy to receive the care they needed. A respondent noted, “It was quick and easy. Instead of finding babysitters for my 4-month-old twins, it was convenient to do right from my home.” Open-ended responses (Table 4) support this finding: patients valued receiving care from their home without traveling to their clinician’s office (145/363, 39.9%) and reported that virtual visits provided convenient access to health care (121/363, 33.3%) and saved them time (98/363, 26.9%). Most patients reported that they would use virtual care again (392/422, 92.9%) and would recommend it to other people (380/414, 91.8%).

Comfort and Ease Using Virtual Visit Technology

The mean overall technology experience score was 4.6 out of 5 with an SD of 0.57, and 93.4% (394/422) of patients reported that the virtual visit interface was easy to use. Participants agreed and strongly agreed that they felt comfortable using it (401/423, 94.8%), and most agreed or strongly agreed that the wait times for using it were reasonable (400/422, 94.8%). Respondents could see (403/424, 95.1%), hear (394/423, 93.1%), and talk with (400/423, 94.6%) their clinician easily during their virtual visit. However, 19.9% (84/423) of patients reported that they had technical difficulties during their virtual visit (Table 5), and technical difficulties were associated with lower odds of overall satisfaction among patients (OR 0.46, 95% CI 0.28-0.76) in multivariable logistic regression. In addition, in multivariable logistic regression, having a prior relationship with their virtual visit clinician was associated with less comfort and ease with virtual technology among patients (OR 0.58, 95% CI 0.35-0.98). In open-ended comments, 23.5% (100/426) of patients commented on what could be improved, and 30% (30/100) of these comments recommended improvement of the user interface (see Table 4 for additional patient suggestions for improvement).

Patient-Clinician Engagement

Most patients (393/423, 92.9%) agreed that their virtual visit clinician was interested in them as a person. A respondent commented, “The physician was very kind and really listened to my issue and what I had already tried to resolve it” (for additional patient responses, see Table 4). In addition, the patients felt that they were able to work together with their clinicians reciprocally—94.5% (398/421) of respondents reported that together with their virtual visit clinician, they made a plan of action to resolve their health concern. In multivariable logistic regression, a visit with an advanced practice provider (nurse practitioner or physician assistant) was associated with higher odds of patient-clinician engagement compared with visits with a family medicine physician (OR 2.28, 95% CI 1.25-4.16). In addition, employees of our medical center had higher odds of patient-clinician engagement than nonemployees (OR 1.73, 95% CI 1.01-2.95).

The mean patient-clinician engagement score was 4.6 out of 5, with an SD of 0.53. In open-ended comments, patients described the high quality of their virtual visit clinician (50/363, 13.8%), and 95.7% (399/417) of patients agreed that their virtual visit clinician fully understood their health concern. However, some results suggest a need to improve the patient-clinician relationship. Open-ended comments reflected a need for better orientation to virtual visits: 14% (14/100) of respondents recommended that patients be given more information before their virtual visits to know what to expect and how to prepare for their appointment.

Discussion

Principal Findings

The large and diverse sample size and high response rate (426/648, 65.7%) suggest robust findings about patient experience of virtual visits as compared with prior studies [27].

This large mixed methods study in a major health system found that patients reported high satisfaction with virtual visits, that the technology was easy to use, and that virtual visits were comparable or better than an in-person visit. However, technical difficulties were associated with lower odds of overall satisfaction among patients (OR 0.46, 95% CI 0.28-0.76) in the multivariable logistic regression. Our findings of high patient satisfaction with virtual visits align with those of other published studies [15,20,28,29]. For example, 95% of patients who participated in a MinuteClinic telehealth visit were very satisfied with the quality of the health care they received and rated telehealth as better than or just as good as a traditional in-person visit [15]. However, our findings are unique and go beyond these other studies because we measured satisfaction in terms of several indicators of patient-clinician engagement. We found that most patients in our study reported high engagement with their virtual visit clinician. There are concerns about the effect of telemedicine on trust-based relationships between patients and clinicians [30] and the ability to express empathy in digital settings [31,32]. Our study suggests that it is possible to measure the patient-clinician engagement and begin to evaluate empathy and collaborative relationships with patients during a virtual visit.

However, both clinician and patient identity influenced the likelihood of strong patient-clinician engagement during virtual visits. Having a pre-existing relationship with the clinician was associated with lower satisfaction. In terms of gender and age, older men who had previous relationships with their virtual visit clinicians were some of the patients that were least satisfied with their experiences. Patients who had a scheduled virtual visit were older than those who had an on-demand visit and were perhaps less comfortable with virtual technology than younger patients. In multivariable logistic regression, having a prior relationship with their virtual visit clinician was associated with less comfort and ease with virtual technology among patients (OR 0.58, 95% CI 0.35-0.98). However, patient age was not a significant predictor of overall satisfaction, patient-clinician engagement, or comfort and ease with virtual technology. Patients who received care from their clinician via a traditional in-person appointment may have experienced discomfort with, or simply did not like, this unfamiliar mode of care with a clinician they know. Gender, age, and the necessity of multiple or repeated virtual visits instead of in-person visits may contribute to less satisfaction and should be explored further in future studies.

In terms of employment and insurance status, employee status was associated with overall satisfaction with virtual visits. In multivariable logistic regression, employee patient status (vs nonemployee) was associated with higher odds of overall satisfaction with their virtual visit (OR 1.90, 95% CI 1.14-3.47). In addition, employees of our medical center had higher odds of patient-clinician engagement than nonemployees (OR 1.73, 95% CI 1.01-2.95). This may be because of benefit coverage for the virtual visit; medical center employees received the care they needed with little to no out-of-pocket costs. Employee patients were not charged for their virtual visit, whereas nonemployee patients were charged up to US $49, depending on insurance coverage. Employee patients may also be more familiar with clinicians, telehealth in general, and virtual visits specifically; thus, their expectations may have been better aligned with the virtual health care they received.

Our study has limitations that need to be considered. Although the sample was diverse in age, gender, and visit and clinician type, a large percentage of patients were from Ohio, insurance was unknown for more than half, and a quarter were employees, all of which may limit the generalizability of our findings. In addition, despite our relatively high response rate, we may have a response bias, but this is unlikely given the similarity of the responders to the nonresponders. Furthermore, we should be cautious not to generalize our findings to all virtual visits, given that this was a nonprobability sample in one health system.

In addition, we do not know the reason for patient visits. We collectively analyzed on-demand and scheduled visits together, although on-demand visits are typically low acuity infections (upper respiratory and urinary tract), whereas scheduled visits are usually more complex conditions where patients have more intense needs [13]. As nearly all patients who had a prior relationship with their virtual visit clinician had a scheduled virtual visit rather than an on-demand one, their expectations for the care they would receive were likely different from those of patients who did not know their clinicians. Furthermore, scheduled and on-demand visits may be fundamentally different. This study suggests that virtual visits may be most satisfactory when used for acute problems or when health care access is otherwise limited rather than nonacute or more complex issues when patients may prefer speaking with their provider in person at a medical facility. Although this study provides an analysis of many kinds of visits, future studies should disaggregate the types of visits and analyze them separately.

Future studies should consider both the quality of these visits and medical outcomes because both are likely to influence patient satisfaction.

In addition, future studies should further explore the range of clinician experience in virtual visits. In multivariable logistic regression, a visit with an advanced practice provider (nurse practitioner or physician assistant) was associated with higher odds of patient-clinician engagement than visits with a family medicine physician (OR 2.28, 95% CI 1.25-4.16). Our finding that patients were more likely to report high engagement with advanced practice providers rather than with family physicians aligns with findings on in-person visits [32] and highlights opportunities to leverage advanced practice providers in telehealth. Future studies should explore the basis of these differences.

Future research and efforts should also focus on the user interface, facilitating patient expectations of the technology, and associations with quality.

Conclusions

The impact of the virtual interface on patient-clinician relationships is largely unknown, but our findings are encouraging. Our study found that virtual visits facilitate health care access and relationship-building, contributing to satisfying relationship-centered care, a crucial aspect of contemporary patient experiences. Even during a single virtual visit, we found that patients and clinicians could meaningfully engage in relationship-building practices. Strategies to prepare established patients for virtual visits with their clinicians may ease the transition from in-person care to virtual care, resulting in better experiences for both. Patients should be aware of the capabilities and limitations of patient-clinician engagement in virtual visits [33]. Strategies to prepare clinicians for virtual visits would also support a seamless transition to delivering health care virtually (a tip sheet that outlines 10 best practices for communicating effectively with patients during a virtual visit has been provided in Multimedia Appendix 2) [34].