This paper is in the following e-collection/theme issue:

JMIR Theme Issue: COVID-19 Special Issue (2452) Telehealth and Telemonitoring (1671) Neurology and Neurosciences (185) Outbreak and Pandemic Preparedness and Management (1727) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1517)

Published on in Vol 22, No 12 (2020): December

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/24328, first published .
Rapid Implementation of Video Visits in Neurology During COVID-19: Mixed Methods Evaluation

Rapid Implementation of Video Visits in Neurology During COVID-19: Mixed Methods Evaluation

Rapid Implementation of Video Visits in Neurology During COVID-19: Mixed Methods Evaluation

Authors of this article:

Erika A Saliba-Gustafsson1 Author Orcid Image ;   Rebecca Miller-Kuhlmann2 Author Orcid Image ;   Samantha M R Kling1 Author Orcid Image ;   Donn W Garvert1 Author Orcid Image ;   Cati G Brown-Johnson1 Author Orcid Image ;   Anna Sophia Lestoquoy1 Author Orcid Image ;   Mae-Richelle Verano1 Author Orcid Image ;   Laurice Yang2 Author Orcid Image ;   Jessica Falco-Walter2 Author Orcid Image ;   Jonathan G Shaw1 Author Orcid Image ;   Steven M Asch1, 3 Author Orcid Image ;   Carl A Gold2 Author Orcid Image ;   Marcy Winget1 Author Orcid Image
Article Authors Cited by (31) Tweetations (9) Metrics

    Original Paper

    1Primary Care and Population Health, Stanford University School of Medicine, Stanford University, Palo Alto, CA, United States

    2Department of Neurology & Neurological Sciences, Stanford University School of Medicine, Stanford University, Palo Alto, CA, United States

    3Center for Innovation to Implementation (Ci2i), VA Palo Alto Health Care System, Menlo Park, CA, United States

    *these authors contributed equally

    Corresponding Author:

    Erika A Saliba-Gustafsson, BSc, MMSc, PhD

    Primary Care and Population Health

    Stanford University School of Medicine

    Stanford University

    Stanford MSOB

    1265 Welch Rd, Mail Code 5475

    Palo Alto, CA, 94305

    United States

    Phone: 1 650 723 1146

    Email: esalgus@stanford.edu


    Background: Telemedicine has been used for decades. Despite its many advantages, its uptake and rigorous evaluation of feasibility across neurology’s ambulatory subspecialties has been sparse. However, the COVID-19 pandemic prompted health care systems worldwide to reconsider traditional health care delivery. To safeguard health care workers and patients, many health care systems quickly transitioned to telemedicine, including across neurology subspecialties, providing a new opportunity to evaluate this modality of care.

    Objective: To evaluate the accelerated implementation of video visits in ambulatory neurology during the COVID-19 pandemic, we used mixed methods to assess adoption, acceptability, appropriateness, and perceptions of potential sustainability.

    Methods: Video visits were launched rapidly in ambulatory neurology clinics of a large academic medical center. To assess adoption, we analyzed clinician-level scheduling data collected between March 22 and May 16, 2020. We assessed acceptability, appropriateness, and sustainability via a clinician survey (n=48) and semistructured interviews with providers (n=30) completed between March and May 2020.

    Results: Video visits were adopted rapidly; overall, 65 (98%) clinicians integrated video visits into their workflow within the first 6 implementation weeks and 92% of all visits were conducted via video. Video visits were largely considered acceptable by clinicians, although various technological issues impacted their satisfaction. Video visits were reported to be more convenient for patients, families, and caregivers than in-person visits; however, access to technology, the patient’s technological capacity, and language difficulties were considered barriers. Many clinicians expressed optimism about future utilization of video visits in neurology. They believed that video visits promote continuity of care and can be incorporated into their practice long-term, although several insisted that they can never replace the in-person examination.

    Conclusions: Video visits are an important addition to clinical care in ambulatory neurology and are anticipated to remain a permanent supplement to in-person visits, promoting patient care continuity, and flexibility for patients and clinicians alike.

    J Med Internet Res 2020;22(12):e24328

    doi:10.2196/24328

    Keywords

    teleneurology; telemedicine; telehealth; ambulatory neurology; video visits; COVID-19; implementation; outcomes; video; neurology; mixed methods; acceptability; sustainability 


    Telemedicine has been used for decades, yet its uptake and rigorous evaluation of feasibility across neurology’s ambulatory subspecialties has been sparse [1,2]. Studies have shown several advantages of telemedicine including improved access to care [3,4], less travel burden, and fewer associated out-of-pocket and health care costs [1,5-8]. Nonetheless, integration of virtual visits into ambulatory neurology care, where the physical examination and nuanced communication play a strong role, has been received with some hesitation [7,9-13]. Further, issues related to technology, compensation, payor reimbursement, policy, hardware/software costs, credentialing, liability, and requirements for in-person evaluations prior to virtual care, have also limited its adoption [1,8,11,13-17].

    The declaration of COVID-19 as a pandemic in March 2020 prompted health care systems worldwide to reconsider traditional health care delivery and quickly transition to telemedicine [18-21]. In regions with rapidly increasing COVID-19 cases and early stay-at-home directives, health care systems rapidly built and implemented infrastructure for telemedicine technologies to protect health care workers and patients, and conserve personal protective equipment. To further support this pivot and maintain health care access, the United States loosened previously stringent federal regulations on reimbursements, licensing, and Health Insurance Portability and Accountability Act (HIPAA) compliance [22,23]. In neurology specifically, the urgent need to provide care safely to patients with chronic illnesses while preventing disease transmission during clinic visits, led to rapid implementation of video visits across all subspecialties [24-29].

    In this study, we evaluate the implementation of video visits in Stanford Health Care’s (California, United States) ambulatory neurology clinics using mixed methods to assess adoption and explore clinicians’ perspectives on the acceptability, appropriateness, and sustainability of this broad expansion.


    Setting

    This quality improvement project received a nonresearch determination by Stanford University’s Institutional Review Board (IRB-55644). It was conducted at Stanford University’s Department of Neurology and Neurological Sciences, which includes 11 ambulatory subspecialties staffed by 60 physicians and 8 advanced practice providers (APPs). As previously described [27], at the beginning of the local COVID-19 stay-at-home directive in March 2020, approximately 50 in-clinic computers were video visit–enabled, and 50 additional computer devices were readied for remote use by providers. Clinic staff converted >90% of scheduled in-person visits to video. In-person visits were reserved for procedures such as autonomic testing, implanted device programming, injections, electromyography, and electroencephalograms, as well as patients determined either during previsit screening or initial video visit to require a full examination.

    Primary Outcomes, Data Collection, and Analysis

    Overview

    The primary implementation outcomes assessed were adoption, acceptability, appropriateness, and perceived sustainability of video visits [30], as described in Table 1. Clinician-level scheduling data were used to assess adoption. A combination of a clinician survey and interviews were used to assess acceptability, appropriateness, and perceived sustainability. Clinicians included physicians and APPs.

    Table 1. Implementation outcomes, definitions, and data sources applied in the evaluation of the implementation of video visits in ambulatory neurology.
    Proctor et al’s [30] implementation outcomesDefinitionsData sources
    AdoptionUptake of video visits in ambulatory neurologyScheduling data
    AcceptabilityClinicians’ overall satisfaction with video visitsClinician interviews and clinician survey
    Appropriateness“Fit for purpose”: clinicians’ perceived suitability and practicability of video visits for a successful patient visit to achieve similar patient outcomes to an in-person visitClinician interviews and clinician survey
    SustainabilityClinicians’ views on the future use of video visits in their practiceClinician interviews and clinician survey
    Scheduling Data

    Clinician-level scheduling data were extracted for all clinicians who had the opportunity to conduct video visits. The first week of the stay-at-home order (March 15-21, 2020) was considered a transition week where the physician champion onboarded physicians onto the video visit platform. To assess our primary outcome, adoption, scheduling data from the 8-week implementation period (March 22 through May 16, 2020) was used for analysis; the transition week was excluded. Adoption was assessed in three ways: (1) proportion of clinicians who conducted video visits, (2) proportion of visits completed via video during the COVID-19 pandemic, and (3) proportion of all “expected” visits that were done via video, where “expected” was defined as the estimated number of visits completed had COVID-19 not occurred (ie, visit volume from the same calendar period in the prior year, March 24 through May 18, 2019).

    The number of visits completed during the implementation period (ie, numerator) and comparator period (ie, denominator) included visit types that could be feasibly conducted in-person or via video. Visit types inherently requiring in-person patient interaction, such as procedures and research visits, were excluded. The percentage of “expected” visits is shown for video and in-person visits separately, along with the percentage of lost potential visits. The proportion of all “expected” visits completed is a measure of the proportion of “saved” or “protected” visits attributable to video visits during the COVID-19 stay-at-home period.

    Clinician Survey

    We developed a 20-item survey informed by early interview findings and reflections from the clinical improvement team. The survey was administered via Stanford’s REDCap platform [31] and emailed to all clinicians using video visits in May 2020, followed by two reminders. To increase response rates, clinicians were encouraged to complete the survey during division meetings and reminded personally by clinical leaders. Descriptive analyses were conducted on 5 items (Multimedia Appendix 1): (1) need for and timing of in-person visits to supplement video visits, (2) top three concerns regarding video visits, (3) top three reasons for excitement for video visits, (4) video visits support of overall clinician wellbeing, and (5) video visit usage to shift uncompensated to compensated work. The number and percentage of clinicians indicating each response are reported for questions 1-3 and for those indicating “agree” or “strongly agree” for questions 4 and 5.

    All quantitative data were processed and analyzed using SAS (Version 9.4; SAS Institute Inc), R (Version 4.0.2; R Foundation for Statistical Computing) [32] and associated packages [33-36].

    Clinician Interviews

    We designed the interview guide (Multimedia Appendix 2) to encompass three implementation outcomes of interest: acceptability, appropriateness, and perceived sustainability [30]. A purposive sample of 47 clinicians who had conducted at least one video visit were invited to participate in semistructured phone interviews held between March and May 2020. Stakeholders were intentionally recruited from different neurology subspecialties to obtain a representative departmental sample. In total, 30 clinicians were interviewed and interviews lasted 25 minutes, on average.

    Interview notes and transcripts were used for analysis. To ensure anonymity, identifiable information was removed from transcripts and subspecialties with <5 clinicians were grouped. Data were analyzed using both deductive and inductive approaches. Deductive codes were derived from the implementation outcomes of interest [30]. During analysis, barriers, facilitators, and other emergent themes were identified and coded. A multiphase analysis approach leveraged rapid analytic procedures (eg, template summaries) to extract early themes, consensus coding of transcript summaries to produce interim results, and a matrix analysis [37] of interview excerpts for final comparison of themes across neurology subspecialties. First, individual interview transcripts were summarized independently by two coauthors into a templated summary document. After review and consensus discussion, transcript summaries were consolidated into a matrix to identify themes and allow for comparison across participants using Microsoft Excel (Microsoft Corp). Four qualitative coauthors met weekly for two months to discuss preliminary results and achieve reporting consensus.

    Mixed Methods Analysis

    Quantitative and qualitative data were consolidated during analysis and interpreted in parallel to understand the impact of video visits in ambulatory neurology more comprehensively. This approach allowed us to harness strengths and offset weaknesses of the two methodologies [38,39]. We were also able to identify converging and diverging issues regarding adoption, acceptability, appropriateness, and perceived sustainability of video visits.


    Participants

    Out of the 68 clinicians in the department’s 11 ambulatory subspecialties, 66 clinicians conducted regular video visits during the 8-week implementation period and were included in analyses (two clinicians were on leave and thus excluded). Table 2 summarizes the number of surveyed and interviewed clinicians, by subspecialty. In total, 48 (73%) clinicians responded to the survey and 30 (45%) were interviewed. In total, 21 (32%) clinicians participated in both the survey and interviews. Only 9 (14%) did not participate in either modality.

    Table 2. Number of ambulatory neurology clinicians who completed the video visit survey and were interviewed, by subspecialty.
    SubspecialtyInterview respondents, nSurvey respondents, nTotal clinicians, n
    Autonomic, neuro-oncology, and neuro-ophthalmologya349
    Epilepsy4910
    General neurology255
    Headache435
    Memory346
    Movement disorders389
    Neuroimmunology355
    Neuromuscular458
    Stroke459
    Total30 (45%)b48 (73%)c66 (100%)

    aThe autonomic, neuro-oncology, and neuro-ophthalmology subspecialties had <5 clinicians and were therefore grouped to ensure anonymity.

    bOf the 30 interviewees, 29 were physicians and 1 was an advanced practice provider.

    cOf the 48 survey respondents, 41 were physicians and 7 were advanced practice providers.

    Adoption

    During the transition week, the clinician champion onboarded 30 of the 66 physicians included in this study, and 129 video visits were conducted. Prior to this transition week, clinicians in these ambulatory neurology clinics had not used video visits. During the 8-week implementation period, video visit adoption was high based on both percentage of clinicians using them, and percentage of visits completed via video. Within the first two weeks, 52 (79%) clinicians integrated video into their practice, increasing to 65 (98%) clinicians by week 6. Almost all (92%) visits were conducted via video and adoption of video visits was high for both new and return patient visits (93% and 91% of completed visits, respectively). The total number of completed visits, conservatively estimated, was almost 80% of expected visits based on the 2019 comparison period (Figure 1A). Clinics recuperated more expected return patient (83%) than new patient visit volumes (68%; Figures 1B-C).

    ‎
    Figure 1. Percentage of video visits conducted between March 22 and May 16, 2020, as a proportion of (A) all "expected" visits overall, (B) "expected" new patient visits, and (C) "expected" return patient visits. The "expected" number of visits is defined as the number of completed visits in the comparable 2019 time period. Data only reflect operational scheduling data captured through the EPIC Hyperspace Platform. The data do not reflect no shows, visits conducted on other HIPAA-compliant software or visits that reverted to phone calls. (HIPAA: Health Insurance Portability and Accountability Act).
    View this figure

    Acceptability

    Overall acceptability of video visits was high, but clinicians experienced technological and scheduling issues, and identified additional needs. Two themes emerged related to acceptance: technology and workflow efficiency.

    Technology

    While some clinicians experienced a smooth transition to video visits and seamless connection with patients, others dealt with numerous technical issues, jeopardizing their overall satisfaction. Indeed, technology was one of the top 3 concerns regarding video visits for 30 (63%) survey respondents (Table 3).

    Table 3. Ambulatory neurology clinicians’ (n=48) top 3 areas of concern related to video visits as reported through survey data.
    With regard to video visits, clinicians (n=48) were concerned about... (indicated top 3)Survey respondents, n (%)
    Technological limitations30 (63)
    Being able to engage in training and education of residents and fellows21 (44)
    Missing/losing the in-person connection/relationship with patients18 (38)
    Including interpreters on video calls17 (35)
    Difficulties arranging and completing necessary follow-ups after the video visit14 (29)
    Insurance reimbursements for video visits are not the same as for in-person visits8 (17)
    Patients’ unwillingness to come into clinic for requested in-person visits in the future7 (15)
    Press-Ganey scores5 (10)
    Patient expectations to have video visits as an option3 (6)
    Maintaining access to readily available technology and equipment needed for video visits2 (4)
    Other (providers listed various reasons)9 (19)

    Several video functionalities were suggested for a successful visit, including the following: (1) screen sharing to facilitate patient education and explain imaging results, (2) a waiting room function to replicate “stepping out of the room” when engaging with trainees, (3) a chat box for troubleshooting, (4) file sharing capabilities, (5) screenshot capabilities to support efficient charting, and (6) multiperson teleconferencing to include other members of the multidisciplinary team, interpreters, trainees, and family members in different physical locations. These functionalities were not available during the study period.

    ...when we're discussing end-of-life care issues, to have the physician and respiratory therapist, social worker and palliative care physician, all present in the same ‘virtual room’, that would be really nice.
    [MD19]

    Engaging in education of trainees was one of the top 3 concerns for 21 (44%) surveyed clinicians (Table 3). The initial interface available through the electronic medical record (EMR) only allowed for a two-way call; therefore, parallel software was needed to include trainees.

    Workflow Efficiency

    Perceptions of workflow efficiency were mixed. Major issues included rigid video visit scheduling, note-taking efficiency, and previsit planning. Several clinicians mentioned that, provided there were no technological issues, video visits helped them stay on schedule. However, hard time limits set by the video visit platform caused frustration. Clinicians could not notify the next patient when running late nor initiate the next visit earlier than scheduled as they might do at the clinic.

    …it's much easier for someone to wait in a physical waiting room if you do happen to run over with a patient who’s before them. It’s a lot harder or less acceptable if someone is waiting to see you via video and staring at their computer screen for 10 or 15 minutes. … You can’t open up another video and say, ‘I'm here. I'm just with somebody else, and I'm going to switch back.’
    [MD10]

    Views on documentation during a video call also differed. Note-taking was considered easier by some during video visits, reducing after-hours charting time. Others were unable to document simultaneously, making them less efficient. Dictation was considered a possible solution.

    …unless you get the screen set up correct[ly], it actually can be really hard to document and type while doing the video visit. The typing can be noisy on the patient’s end…
    [MD5]

    However, most surveyed clinicians (n=34; 71%) agreed or strongly agreed that video visits allowed them to shift uncompensated to compensated work (ie, scheduling a video visit to address concerns that would previously have been managed through either EMR messaging or unscheduled and uncompensated phone calls).

    The lack of integration of medical assistants (MAs) in the video visit workflow, resulting in a lack of previsit charting and medication reconciliation, was another concern. Incorporating MAs, or even involving the patient in their own previsit preparation, was deemed necessary for video visits to be sustained.

    I'd like to see it be more efficient… In advance of the visit, it would be nice if they [patients] could do their medication reconciliation. …enter in the last set of vitals that they took for themselves. …enter in their own review of symptoms and chief complaints.
    [MD16]

    Appropriateness: Suitability, Usefulness, and Practicability of Video Visits

    Three themes emerged related to appropriateness of video visits: benefits and barriers, physical examination, and continuity of care, each of which are described below.

    Benefits and Barriers of Video Visits for Patients and Families/Caregivers

    Clinicians highlighted several benefits of video visits for patients and families/caregivers, including convenience, impact on travel and cost, and seeing the patient’s home environment. Barriers noted included access to technology, patient’s technological capacity, and language. Video visits were considered advantageous and convenient, saving patients time and money, particularly for older adults and those who travel long distances for appointments. Survey respondents (n=37; 77%) agreed that saving patients unnecessary travel was one of the top 3 benefits (Table 4). Several clinicians supported referring patients to local laboratories to further avoid unnecessary travel. Clinicians also noted that for patients who required assistance (eg, patients with dementia, epilepsy, or mobility issues), the video visit alleviated the travel burden on families/caregivers.

    …it's very convenient for our patients, especially those who are elderly or have neurologic issues. [It] Spares them unnecessary travel and costs...
    [MD16]
    Table 4. Ambulatory neurology clinicians’ (n=48) top 3 areas of excitement related to video visits as reported through survey data.
    With regard to video visits, clinicians (n=48) were excited about... (indicated top 3)Survey respondents, n (%)
    Saving patients from unnecessary travel37 (77)
    Increased access for vulnerable populations33 (69)
    Ability to see my patients from my home or nonclinic location23 (48)
    Reduced uncompensated work18 (38)
    Flexible scheduling of patient visits17 (35)
    Ability to see patients in their home environment3 (6)
    Ability to connect with patients’ caregivers/family members3 (6)
    Other (providers listed various reasons)4 (8)

    Several clinicians saw value in seeing patients in their home. Not only did it allow more family involvement, but clinicians could also troubleshoot daily functioning issues or modify the patient’s environment by directly assessing fall hazards, medications, and home devices.

    …maybe you see … a lot of stuff strewn over the floor and that is why they're falling so much … you might observe things … that might impact their neurological disease on a day to day basis … or you don't think to ask about … in a clinic visit because you're not seeing the world that they're actually living in.
    [MD25]

    Even though increased access to vulnerable populations was rated as one of the top 3 benefits of video visits (n=33; 69%; Table 4), numerous clinicians mentioned that some patients, particularly older adults and lower-income patient populations (eg, unhoused individuals or rural farm workers), lacked the necessary access to technology and technological capability to support a video call. In the absence of a supportive family member/caregiver, video visits with patients with cognitive, hearing, or visual impairment were also considered nonideal.

    ... not suited are patients with some mild to moderate cognitive impairment. ...they have more trouble interacting with the physician and understanding who they’re talking to.
    [MD25]

    Language barriers were also considered a possible limitation of video visit utilization by both interviewees and survey respondents. Specifically, 17 (35%) surveyed clinicians indicated inclusion of interpreters was one of their top 3 concerns (Table 3).

    Virtual Physical Examinations

    Clinicians reported that video visits were superior to a phone call, allowing them to gather more information than just a medical history. Although exam needs varied by clinician and subspecialty, several clinicians described that despite challenges, they were pleasantly surprised to be able to perform several modified physical examinations over video. Nevertheless, many stated that the inability to perform a hands-on physical examination was a limitation of video visits that was best paired with a timely in-person follow-up. Several clinicians mentioned that virtual exams were more time-consuming and occasionally required assistance from a caregiver to position the phone to properly observe the patient, perform certain physician-directed exams, confirm what the patient says, or catch the patient if s/he is at risk of falling.

    You have to have a caregiver, you can’t check gait without it [a caregiver] or the patient needs some sort of stand or desk or something where they can put the computer down at the end of a hallway and then asking the patient to find a hallway that they can walk back and forth is really the best way to do it. But a caregiver probably works best.
    [MD21]

    Additional limitations were related to the patient’s immediate environment, including adequate space for the patient to move around, and sufficient lighting. Camera positioning was also critical. If positioned too close, the clinician lacked sufficient visual field to observe the patient’s entire body. Clinicians also noted that occasionally patients took their video call at inappropriate times (eg, while working or driving), despite previsit counseling.

    Promoting Continuity of Care

    Clinicians considered video visits beneficial to ensure continuity of care for chronic conditions. There was general agreement that video visits are best suited for established patients, especially those who are stable/uncomplicated, or for a quick checkup without extensive examination or testing. Although several clinicians felt that most patients were appropriate for video visits, the majority agreed that new patient visits, and patients with acute conditions and declining health, were less suited due to relative ease of the complete physical examination when conducted in person. Many preferred to first see the patient in person before determining whether further follow-up care can be provided, at least partly, over video. Clinicians also recognized that some patient populations encounter significant barriers to attending the clinic in person and that video offers an opportunity to continue care.

    We deal with progressive conditions, and once you get into that moderate to advanced stage of illness it just becomes not worth their effort to come anymore. It’s too difficult on the patient, it’s too difficult on the family, and so we lose contact with the vast majority of our patients as they become more advanced. ...telemedicine actually might allow continuity of care into the more advanced stages…
    [MD12]

    Perceived Sustainability

    Most clinicians were positive toward video visits and believed they could incorporate video into their practice long-term, although several insisted that video cannot replace a full in-person examination. Moreover, 40 (83%) surveyed clinicians agreed or strongly agreed that video visits supported their overall well-being. The general view was that for long-term sustainability, patient video visits will need to be selected carefully to optimize care and respect preferences. Patients’ suitability for video visits would need to be determined during scheduling based on several criteria (eg, physical examination needs, patient’s technological capacity and demographics, new versus return). However, previsit screening and triage to determine which patients are best suited for video was considered burdensome, which could compromise their sustainability. Seldom was the video visit itself considered a good tool for triage.

    I would like to be able to offer it for my patients who are a little less mobile and who come from far away. I don’t think it will completely replace in-person visits, but if we can alternate visits and do in-person versus telehealth, I think that’ll be huge.
    [MD21]

    Most survey respondents (n=39; 81%) agreed that video visits should be supplemented with in-person visits. The recommended frequency of supplemental in-person visits varied among the respondents: 6 (15%) recommended quarterly, 11 (28%) biannually, 17 (44%) annually, and 5 (13%) every 2 years. A concern mentioned by a minority of interviewees and 7 (15%) surveyed clinicians was that patients may find video too convenient and opt out of recommended in-person visits. In contrast, 9 (19%) survey respondents with representation from epilepsy, memory, headache, stroke, and neuro-oncology, indicated that an all-video practice would be feasible for their patient population.


    Principal Findings

    Video visits were rolled out rapidly at Stanford University’s Department of Neurology and Neurological Sciences during the COVID-19 pandemic. A high proportion of visits were conducted via video at volumes near prepandemic volumes, indicating robust and rapid adoption by both clinicians and patients. This necessary and widespread adoption allowed for thorough assessment of new opportunities and barriers in video-based care across almost all ambulatory neurology subspecialties.

    Clinicians recognized the patient benefits of video visits, including saving patients unnecessary travel and increasing access to vulnerable populations, as is well-documented [1,3-8]. As patients with chronic progressive neurologic conditions lose function, in-person visits become increasingly challenging and care by video is a welcome new supplemental resource. Video visits could improve the experience for these patients and others, but various challenges remain. As clinicians noted, the most vulnerable high-need patients were often the hardest to care for virtually. Vulnerability is not uniform across patients and could be related to diagnosis (eg, patients with sensory or cognitive deficits), support (eg, caregiver status), socioeconomic status, or access (eg, poor access to technology). New creative solutions are being explored to address these challenges at various levels, with the goal of rendering virtual care an effective catchment for all. For instance, a pilot program delivering tablets to patients cared for by the US Department of Veterans Affairs has been successful in addressing concerns related to diagnosis and technology [40]. Further, municipality-level internet and broadband for all could address access issues [41].

    Clinicians reported that video visits were beneficial for seeing the patient’s home and meeting caregivers and family members. Observing patients at home is a documented benefit of video visits; it can reduce patient and caregiver anxiety, and allow assessment of fall hazards and habitual behaviors that are not always possible to capture during an in-person visit [6,42]. These benefits were rarely identified within the top 3 benefits of video visits in the survey, which is likely related to the many perceived benefits of video visits as well as the diverse needs of neurology subspecialties’ patient populations. Interviewed clinicians emphasized that the presence of a caregiver is often essential to a successful video visit. For example, a caregiver is essential for collateral history when the patient has cognitive impairment or for safely examining gait for a complaint of parkinsonism, but not for a younger patient presenting with headache. Communicating the importance of including caregivers in certain video visits is thus important for optimal care provision.

    Previously, the perceived limitations of virtual physical examinations have been a source of hesitant incorporation of telemedicine into neurology practice [10-12]; however, virtual physical examinations are possible, especially to supplement heavily history-based presentations [1,6,43]. A previous investigation demonstrated that virtual exams were adequate even for many patients with multiple sclerosis and neuroimmunology issues [44]. Presence of a caregiver can further facilitate a detailed and complete medical history and examination [28]. In this study, several clinicians reported that video cannot replace a full in-person examination, but others reported that the virtual physical examination was more feasible than expected. Best practices that were followed across clinics included consenting patients and explaining video limitations prior to the visit, and scheduling patients for in-person follow-up as soon as they are able to do so safely whenever an in-person examination was deemed important during the video visit. As clinics adjust to new routines and workflows, measures are also being taken to further optimize the virtual exam experience. For instance, the department has organized teleneurology professional development webinars on optimal virtual physical examination techniques [23,45], and guidance documents on this topic have also been developed internally. Together with time and experience, these efforts may further increase clinician comfort with the virtual physical examination.

    Clinicians were keen about the ability to work remotely, reduction in uncompensated work, and flexible scheduling facilitated by video visit use. In the current configuration, video visits are not constrained by clinic spaces nor specific staffing needs, thereby allowing clinicians flexibility to vary timing and length of both their overall clinic session and individual patient appointments. This is a clear benefit, as lack of flexibility in work is a key driver of burnout [46], and even more salient given increased family and childcare duties for many due to the pandemic. Furthermore, reducing uncompensated work has potential well-being benefits. Historically, clinicians with full clinic schedules answered patient messages and phone calls uncompensated outside of normal clinical hours; this is another driver of physician burnout [47]. Video visits have relatively fewer constraints than in-person visits that are often booked out for months, and could enable the ability to address a new patient concern in short order “on the clock” and compensated. Further, most respondents also reported that video visits support their well-being, which is key in a specialty where an estimated 60% of clinicians reported at least one symptom of burnout even before the pandemic [48].

    Perceived sustainability of video visits was high; clinicians emphasized a desire for video visits to remain a permanent fixture in their practice. However, the proportion of video visits and in-person visits within their practice was anticipated to change in their postpandemic practice and vary by individual providers and subspecialty. Most clinicians preferred to have a practice that uses both video and in-person visits, but more than half (57%) expressed a preference for less frequent in-person visits (ie, every 1 or 2 years). Perhaps most surprisingly, nearly 20% of clinicians from a variety of neurology subspecialties indicated that an all-virtual practice was possible. This variation in preferences may reflect the wide variation in the nature and examination demands of neurologic subspecialties, with subspecialties known to be especially exam-dependent (eg, movement and neuromuscular disorders) notably absent from those indicating feasibility of an entirely virtual practice.

    The capabilities of mobile technology and remote monitoring have facilitated great advances in telemedicine [2,6,8,14]; however, several challenges remain and need to be addressed to enable long-term sustainability of video visits. Audio and video connectivity issues may be partially ameliorated by software updates and through increased experience with telemedicine, particularly for clinicians. On the other hand, some technological issues experienced by patients could potentially be addressed with targeted efforts, including more previsit planning and education with a staff member. To enhance the video visit experience for both patients and clinicians, pertinent features included virtual waiting rooms and multiuser interfaces to promote incorporation of trainees, interpretation services, and multidisciplinary care. Some of these features were added in subsequent updates and parallel workflows were developed to use other HIPAA-compliant platforms for visits including trainees or interpreters. In our experience, this parallel platform and approach were important for resolving some issues of functionality and service; however, the lack of integration and interoperability with standard EMR video functionality could impact sustainability.

    Further analysis of the workflow around telemedicine in ambulatory neurology is needed as there is growing recognition that workflow is nontrivial and complex [49]. In this study, initial workflows for video visits did not include MAs taking patient vitals and performing medication reconciliation, key aspects of in-person visits for new patients. Clinic processes should reincorporate MAs in video previsits, perhaps through phone or video visits in the days prior to the clinical video visit. As clinical workflows evolve to sustain video visit use and meet the needs of the most vulnerable patient populations, these video previsits could potentially also provide patient education on optimal lighting, how to position devices, space, and presence of caregivers.

    Strengths and Limitations

    Perceptions of clinicians using video visits were captured in real time during early rapid implementation through interviews and/or a survey across many ambulatory neurology subspecialties. However, given its nascent state, operational scheduling data only captured data of video visits conducted via the EPIC Hyperspace Platform and did not consistently reflect visits conducted on other HIPAA-compliant software, such as Zoom. Therefore, the data presented herein likely underestimate the actual total number of completed visits. Further, when video visits were diverted to phone calls or other software during the 8-week implementation period, the scheduling system incorrectly categorized the encounter as cancelled, no show, or patient left without being seen. This limitation has been recently addressed, presenting an opportunity to investigate the impact of video visits on clinic utilization in future investigations. Finally, views of other essential health care staff, such as patient care coordinators and MAs, who performed essential activities (eg, scheduling and confirming patients’ technology capabilities), as well as residents and patients/caregivers, are not presented here.

    Conclusions

    Video visit adoption was rapid at Stanford’s ambulatory neurology clinics. Almost all clinicians conducted video visits by the eighth week of implementation and achieved near-normal patient volumes during the COVID-19 pandemic stay-at-home order. Despite the sudden change in workflow, clinicians largely expressed positive views toward video visits; clinicians supported permanent integration of video visits and noted them to be conducive to physician well-being. However, overall clinician satisfaction was impacted by technological issues, limitations with the physical examination, and challenges accessing vulnerable patient populations. Although our mixed methods evaluation confirmed the success of video visits in all subspecialty neurology clinics across many dimensions, innovations must be developed to address their limitations. Additional solutions are also needed for the most vulnerable patient populations. A crucial next step for optimization is to understand patients’ experiences and preferences.

    Acknowledgments

    We would like to extend our gratitude to all participating clinicians who shared their insights with us. Thank you to Dr Frank Longo, Dr Yuen So, Alison Kerr, Sophia Loo, and Dana Gonzales for their support of this clinical innovation. We also thank Dr David Larson and the leadership of the Improvement Capability Development Program for arranging the partnership between the Department of Neurology & Neurological Sciences and the Evaluation Sciences Unit.

    This project was supported by Stanford Health Care as part of the Improvement Capability Development Program. The Stanford REDCap platform was used for quantitative survey data collection. The Stanford REDCap platform [31] is developed and operated by the Stanford Medicine Research information technology team. The REDCap platform services at Stanford are subsidized by both the Stanford School of Medicine Research Office and the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grant UL1 TR001085.

    Authors' Contributions

    RMK, SMRK, CGBJ, LY, JFW, JGS, SMA, CAG, and MW contributed to conception and design. RMK, LY, JFW, and CAG contributed to implementation. EASG, SMRK, and DWG contributed to data acquisition, analysis, and interpretation. CGBJ, ASL, and MRV contributed to data acquisition and analysis. EASG, RMK, SMRK, MW contributed to manuscript drafts and significant revisions. CGBJ, DWG, ASK, MRV, LY, JFW, JGS, SMA, and CAG contributed to manuscript revisions. All authors gave final approval of the published version.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Selected questions from 20-item neurology video visit clinician survey administered electronically via the REDCap survey tool.

    DOCX File , 22 KB
    ‎
    Multimedia Appendix 2

    Interview guide used to understand ambulatory neurology clinicians’ views on the acceptability, appropriateness, and sustainability of video visits in their practice.

    DOCX File , 15 KB
    1. Hatcher-Martin JM, Adams JL, Anderson ER, Bove R, Burrus TM, Chehrenama M, et al. Telemedicine in neurology. Neurology 2020 Jan 07;94(1):30-38. [CrossRef]
    2. Dorsey ER, Glidden AM, Holloway MR, Birbeck GL, Schwamm LH. Teleneurology and mobile technologies: the future of neurological care. Nat Rev Neurol 2018 May 6;14(5):285-297. [CrossRef] [Medline]
    3. Davis LE, Harnar J, LaChey-Barbee LA, Pirio Richardson S, Fraser A, King MK. Using teleneurology to deliver chronic neurologic care to rural veterans: analysis of the first 1,100 patient visits. Telemed J E Health 2019 Apr;25(4):274-278. [CrossRef] [Medline]
    4. Demaerschalk BM, Berg J, Chong BW, Gross H, Nystrom K, Adeoye O, et al. American telemedicine association: telestroke guidelines. Telemed J E Health 2017 May;23(5):376-389 [FREE Full text] [CrossRef] [Medline]
    5. Appireddy R, Khan S, Leaver C, Martin C, Jin A, Durafourt BA, et al. Home virtual visits for outpatient follow-up stroke care: cross-sectional study. J Med Internet Res 2019 Oct 07;21(10):e13734 [FREE Full text] [CrossRef] [Medline]
    6. Ben-Pazi H, Browne P, Chan P, Cubo E, Guttman M, Hassan A, International Parkinson and Movement Disorder Society Telemedicine Task Force. The promise of telemedicine for movement disorders: an interdisciplinary approach. Curr Neurol Neurosci Rep 2018 Apr 13;18(5):26. [CrossRef] [Medline]
    7. Mammen JR, Elson MJ, Java JJ, Beck CA, Beran DB, Biglan KM, et al. Patient and physician perceptions of virtual visits for Parkinson's disease: a qualitative study. Telemed J E Health 2018 Apr;24(4):255-267. [CrossRef] [Medline]
    8. Schneider RB, Biglan KM. The promise of telemedicine for chronic neurological disorders: the example of Parkinson's disease. Lancet Neurol 2017 Jul;16(7):541-551. [CrossRef] [Medline]
    9. Howard IM, Kaufman MS. Telehealth applications for outpatients with neuromuscular or musculoskeletal disorders. Muscle Nerve 2018 Oct 17;58(4):475-485. [CrossRef] [Medline]
    10. Helleman J, Kruitwagen ET, van den Berg LH, Visser-Meily JMA, Beelen A. The current use of telehealth in ALS care and the barriers to and facilitators of implementation: a systematic review. Amyotroph Lateral Scler Frontotemporal Degener 2020 May 26;21(3-4):167-182. [CrossRef] [Medline]
    11. Hassan A, Dorsey ER, Goetz CG, Bloem BR, Guttman M, Tanner CM, et al. Telemedicine use for movement disorders: a global survey. Telemed J E Health 2018 Dec;24(12):979-992. [CrossRef] [Medline]
    12. Geronimo A, Wright C, Morris A, Walsh S, Snyder B, Simmons Z. Incorporation of telehealth into a multidisciplinary ALS clinic: feasibility and acceptability. Amyotroph Lateral Scler Frontotemporal Degener 2017 Nov 05;18(7-8):555-561. [CrossRef] [Medline]
    13. George BP, Scoglio NJ, Reminick JI, Rajan B, Beck CA, Seidmann A, et al. Telemedicine in leading US neurology departments. Neurohospitalist 2012 Oct 17;2(4):123-128 [FREE Full text] [CrossRef] [Medline]
    14. Achey M, Aldred JL, Aljehani N, Bloem BR, Biglan KM, Chan P, International Parkinson and Movement Disorder Society Telemedicine Task Force. The past, present, and future of telemedicine for Parkinson's disease. Mov Disord 2014 Jun 17;29(7):871-883. [CrossRef] [Medline]
    15. Wechsler LR. Advantages and limitations of teleneurology. JAMA Neurol 2015 Mar 01;72(3):349-354. [CrossRef] [Medline]
    16. Reider-Demer MM, Eliashiv D. Expanding the use of telemedicine in neurology: a pilot study. JMTM 2018 Oct;7(2):46-50. [CrossRef]
    17. Patel UK, Malik P, DeMasi M, Lunagariya A, Jani VB. Multidisciplinary approach and outcomes of tele-neurology: a review. Cureus 2019 Apr 08;11(4):e4410 [FREE Full text] [CrossRef] [Medline]
    18. Bashshur R, Doarn CR, Frenk JM, Kvedar JC, Woolliscroft JO. Telemedicine and the COVID-19 pandemic, lessons for the future. Telemed J E Health 2020 May 08;26(5):571-573. [CrossRef] [Medline]
    19. Hollander JE, Carr BG. Virtually perfect? Telemedicine for Covid-19. N Engl J Med 2020 Apr 30;382(18):1679-1681. [CrossRef] [Medline]
    20. Reeves JJ, Hollandsworth HM, Torriani FJ, Taplitz R, Abeles S, Tai-Seale M, et al. Rapid response to COVID-19: health informatics support for outbreak management in an academic health system. J Am Med Inform Assoc 2020 Jun 01;27(6):853-859 [FREE Full text] [CrossRef] [Medline]
    21. Myers US, Birks A, Grubaugh AL, Axon RN. Flattening the curve by getting ahead of it: how the VA healthcare system is leveraging telehealth to provide continued access to care for rural veterans. J Rural Health 2020 Apr 13:1 [FREE Full text] [CrossRef] [Medline]
    22. Centers for Medicare and Medicaid Services. Medicare telemedicine health care provider fact sheet. 2020.   URL: https://tinyurl.com/y4qz4am4 [accessed 2020-08-17]
    23. American Academy of Neurology. Telemedicine and COVID-19 implementation guide. 2020.   URL: https://tinyurl.com/y37xxkrv [accessed 2020-08-17]
    24. Waldman G, Mayeux R, Claassen J, Agarwal S, Willey J, Anderson E, et al. Preparing a neurology department for SARS-CoV-2 (COVID-19). Neurology 2020 Apr 06;94(20):886-891. [CrossRef]
    25. Guidon AC, Amato AA. COVID-19 and neuromuscular disorders. Neurology 2020 Apr 13;94(22):959-969. [CrossRef]
    26. Klein BC, Busis NA. COVID-19 is catalyzing the adoption of teleneurology. Neurology 2020 Apr 01;94(21):903-904. [CrossRef]
    27. Yang L, Brown-Johnson CG, Miller-Kuhlmann R, Kling SM, Saliba-Gustafsson EA, Shaw JG, et al. Accelerated launch of video visits in ambulatory neurology during COVID-19. Neurology 2020 Jul 01;95(7):305-311. [CrossRef]
    28. Grossman SN, Han SC, Balcer LJ, Kurzweil A, Weinberg H, Galetta SL, et al. Rapid implementation of virtual neurology in response to the COVID-19 pandemic. Neurology 2020 May 01;94(24):1077-1087. [CrossRef]
    29. Strowd RE, Strauss L, Graham R, Dodenhoff K, Schreiber A, Thomson S, et al. Rapid implementation of outpatient teleneurology in rural Appalachia: barriers and disparities. Neurol Clin Pract 2020 Jul 13:10.1212/CPJ.0000000000000906. [CrossRef]
    30. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar 19;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
    31. REDCap. Welcome to REDCap @Stanford. 2020.   URL: https://redcap.stanford.edu/ [accessed 2020-08-17]
    32. R Foundation for Statistical Computing. R: A language and environment for statistical computing. Vienna, Austria; 2020.   URL: https://www.r-project.org/ [accessed 2020-08-17]
    33. Auguie B, Antonov A. gridExtra: miscellaneous functions for Grid graphics. 2017.   URL: https://CRAN.R-project.org/package=gridExtra [accessed 2020-08-17]
    34. Wickham H, Pedersen TL. gtable: arrange Grobs in tables. RStudio. 2019.   URL: https://CRAN.R-project.org/package=gtable [accessed 2020-08-17]
    35. Urbanek S, Horner J. Cairo: R graphics device using Cairo graphics library for creating high-quality bitmap (PNG, JPEG, TIFF), vector (PDF, SVG, PostScript) and display (X11 and Win32) output. RStudio. 2020.   URL: https://CRAN.R-project.org/package=Cairo [accessed 2020-08-17]
    36. Barthelme S, Tschumperle D, Wijffels J, Assemlal HE, Ochi S. Imager: image processing library based on CImg. RStudio. 2019 Jun.   URL: https://CRAN.R-project.org/package=imager [accessed 2020-08-17]
    37. Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health Res 2002 Jul 01;12(6):855-866. [CrossRef]
    38. Tariq S, Woodman J. Using mixed methods in health research. JRSM Short Rep 2013 Jun 31;4(6):2042533313479197 [FREE Full text] [CrossRef] [Medline]
    39. Bekhet AK, Zauszniewski JA. Methodological triangulation: an approach to understanding data. Nurse Res 2012 Nov 22;20(2):40-43. [CrossRef] [Medline]
    40. Slightam C, Gregory AJ, Hu J, Jacobs J, Gurmessa T, Kimerling R, et al. Patient perceptions of video visits using Veterans Affairs telehealth tablets: survey study. J Med Internet Res 2020 Apr 15;22(4):e15682 [FREE Full text] [CrossRef] [Medline]
    41. Benda NC, Veinot TC, Sieck CJ, Ancker JS. Broadband internet access is a social determinant of health!. Am J Public Health 2020 Aug;110(8):1123-1125. [CrossRef]
    42. Dorsey ER, Vlaanderen FP, Engelen LJ, Kieburtz K, Zhu W, Biglan KM, et al. Moving Parkinson care to the home.. Mov Disord 2016 Sep;31(9):1258-1262 [FREE Full text] [CrossRef] [Medline]
    43. Blue R, Yang AI, Zhou C, De Ravin E, Teng CW, Arguelles GR, et al. Telemedicine in the era of coronavirus disease 2019 (COVID-19): a neurosurgical perspective. World Neurosurg 2020 Jul;139:549-557 [FREE Full text] [CrossRef] [Medline]
    44. Bove R, Garcha P, Bevan CJ, Crabtree-Hartman E, Green AJ, Gelfand JM. Clinic to in-home telemedicine reduces barriers to care for patients with MS or other neuroimmunologic conditions. Neurol Neuroimmunol Neuroinflamm 2018 Nov;5(6):e505 [FREE Full text] [CrossRef] [Medline]
    45. Boes CJ, Leep Hunderfund AN, Martinez-Thompson JM, Kumar N, Savica R, Cutsforth-Gregory JK, et al. A primer on the in-home teleneurologic examination. Neurol Clin Pract 2020 May 21:10.1212/CPJ.0000000000000876. [CrossRef]
    46. Shanafelt TD, Noseworthy JH. Executive leadership and physician well-being: nine organizational strategies to promote engagement and reduce burnout. Mayo Clin Proc 2017 Jan;92(1):129-146. [CrossRef] [Medline]
    47. Gardner R, Cooper E, Haskell J, Harris D, Poplau S, Kroth P, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019 Feb 01;26(2):106-114. [CrossRef] [Medline]
    48. Busis NA, Shanafelt TD, Keran CM, Levin KH, Schwarz HB, Molano JR, et al. Burnout, career satisfaction, and well-being among US neurologists in 2016. Neurology 2017 Jan 25;88(8):797-808. [CrossRef]
    49. Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc 2010;17(3):265-273 [FREE Full text] [CrossRef] [Medline]

    ‎
    APP: advanced practice provider
    EMR: electronic medical record
    HIPAA: Health Insurance Portability and Accountability Act
    MA: medical assistant

    Edited by G Eysenbach, R Kukafka; submitted 15.09.20; peer-reviewed by K Nevel, B Clay; comments to author 16.10.20; revised version received 22.10.20; accepted 23.11.20; published 09.12.20

    Copyright

    ©Erika A Saliba-Gustafsson, Rebecca Miller-Kuhlmann, Samantha M R Kling, Donn W Garvert, Cati G Brown-Johnson, Anna Sophia Lestoquoy, Mae-Richelle Verano, Laurice Yang, Jessica Falco-Walter, Jonathan G Shaw, Steven M Asch, Carl A Gold, Marcy Winget. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.12.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.