Original Paper
Abstract
Background: Rapid implementation of telehealth for cancer care during COVID-19 required innovative and adaptive solutions among oncology health care providers and professionals (HPPs).
Objective: The aim of this qualitative study was to explore oncology HPPs’ experiences with telehealth implementation during the COVID-19 pandemic.
Methods: This study was conducted at Moffitt Cancer Center (Moffitt), an NCI (National Cancer Institute)-Designated Comprehensive Cancer Center. Prior to COVID-19, Moffitt piloted telehealth visits on a limited basis. After COVID-19, Moffitt rapidly expanded telehealth visits. Telehealth visits included real-time videoconferencing between HPPs and patients and virtual check-ins (ie, brief communication with an HPP by telephone only). We conducted semistructured interviews with 40 oncology HPPs who implemented telehealth during COVID-19. The interviews were recorded, transcribed verbatim, and analyzed for themes using Dedoose software (version 4.12).
Results: Approximately half of the 40 participants were physicians (n=22, 55%), and one-quarter of the participants were advanced practice providers (n=10, 25%). Other participants included social workers (n=3, 8%), psychologists (n=2, 5%), dieticians (n=2, 5%), and a pharmacist (n=1, 3%). Five key themes were identified: (1) establishing and maintaining patient-HPP relationships, (2) coordinating care with other HPPs and informal caregivers, (3) adapting in-person assessments for telehealth, (4) developing workflows and allocating resources, and (5) future recommendations. Participants described innovative strategies for implementing telehealth, such as coordinating interdisciplinary visits with multiple HPPs and inviting informal caregivers (eg, spouse) to participate in telehealth visits. Health care workers discussed key challenges, such as workflow integration, lack of physical exam and biometric data, and overcoming the digital divide (eg, telehealth accessibility among patients with communication-related disabilities). Participants recommended policy advocacy to support telehealth (eg, medical licensure policies) and monitoring how telehealth affects patient outcomes and health care delivery.
Conclusions: To support telehealth growth, implementation strategies are needed to ensure that HPPs and patients have the tools necessary to effectively engage in telehealth. At the same time, cancer care organizations will need to engage in advocacy to ensure that policies are supportive of oncology telehealth and develop systems to monitor the impact of telehealth on patient outcomes, health care quality, costs, and equity.
doi:10.2196/29635
Keywords
Introduction
The onset of the COVID-19 pandemic in March 2020 accelerated the rapid adoption of telehealth, which is health care delivery at a distance [
- ]. Prior to the COVID-19 pandemic, health care systems faced substantial barriers to telehealth adoption, such as limited reimbursement and start-up hurdles (eg, infrastructure costs) [ - ]. In the United States, telehealth had spread in some health care markets (eg, neurology) due to policy reform, while diffusion into other markets, such as oncology, was minimal [ - ]. Widespread federal and state changes to telehealth regulation, payment, and insurance benefit design as a result of COVID-19 reduced many entry barriers, allowing health care systems to deploy telehealth for oncology [ , - ]. This seismic shift in telehealth policy created a window of opportunity to redesign cancer care for a virtual setting. In response, cancer care systems invested in large-scale organizational change, such as adopting new technologies and changing billing processes, to support telehealth implementation [ - ]. Much of this work was led by cancer care frontline workers who quickly adapted their practice to a virtual environment. The 1-year anniversary of COVID-19 was an opportunity to evaluate oncology care teams’ experiences with telehealth and assess the role that telehealth can play in cancer care in the future.Before the pandemic, the Centers for Medicare & Medicaid Services (CMS) defined telehealth as real-time videoconferencing between health care providers and professionals (HPPs) and patients [
]. During the pandemic, CMS broadened its definition of telehealth to include virtual check-ins (ie, brief communication with a practitioner by telephone or another form of telecommunication) and e-visits (ie, asynchronous communication between patients and HPPs through a secure platform, such as a patient portal). In response, oncology care teams began to implement telehealth in various forms [ - ]. In the span of just a few weeks after the pandemic hit, many cancer care systems implemented telehealth for the first time and rapidly scaled virtual care to more than half of their patient population [ - ]. HPPs were required to adapt elements of the physical exam to a telehealth visit and maintain strong relationships with patients from a distance [ , , , - ]. HPPs were also faced with a long-standing digital divide, that is, disparities in patients’ technology know-how and access [ ]. Early reports of telehealth implementation described challenges, such as limited device access, lack of home broadband, limited digital literacy, and lack of technology education programs for patients [ ]. Further, HPPs were required to make complex decisions around patient care, such as determining which conditions and patients are appropriate for virtual care [ , ].To capture experiences with telehealth implementation during COVID-19, we conducted a qualitative study with HPPs at a cancer center. We discuss insights from oncology HPPs regarding the successes and challenges with telehealth implementation and recommendations for future telehealth delivery.
Methods
Setting, Recruitment, and Eligibility Criteria
This study was conducted at Moffitt Cancer Center (Moffitt), an NCI (National Cancer Institute)-Designated Comprehensive Cancer Center, located in Tampa, Florida. Prior to COVID-19, Moffitt piloted telehealth visits (ie, real-time videoconferencing between HPPs and patients) [
] on a limited basis. After the start of the COVID-19 pandemic, Moffitt rapidly expanded telehealth visits; the volume increased by 5000% from pre–COVID-19 to after the start of the COVID-19 pandemic [ ]. Moffitt also instituted virtual check-ins (ie, brief communication with an HPP by telephone only) [ ] as a part of telehealth delivery. Moffitt did not employ any other telecommunication technologies for virtual check-ins (eg, secure text messages) and did not implement any other forms of telehealth (eg, e-visits). The Moffitt Virtual Health Program—a centralized support system for the institution—provided information technology (IT) support for patients and HPPs, technical assistance for HPPs (eg, help with out-of-state licensure questions), and an interpreter service for patients with limited English proficiency. Telehealth visits were implemented through Zoom software (Zoom Video Communications, Inc), which was not integrated into Moffitt’s electronic health record (EHR) system. The telehealth visits were conducted in various locations depending on what space was available to the HPP (eg, private office, shared office, and conference room). Some HPPs also elected to deliver telehealth visits from their home.The target population included all full-time HPPs in oncology at Moffitt. HPPs were defined as health care providers who are licensed to diagnose and deliver treatment (eg, physicians, advanced practice providers [APPs], and psychologists) and professionals who are licensed to deliver other health services (eg, dieticians, social workers, and pharmacists). Oncology was defined, using the NCI’s definition, as a field of medicine concerned with the diagnosis and treatment of cancer [
]. We excluded trainees (eg, fellows, residents, and interns), given that their experience with adapting to telehealth would likely be distinct from HPPs. Since trainees are new to delivering care, we opted for interviewing HPPs who had experience with delivering care prior to the pandemic and may be better positioned to describe how health care had changed because of the pandemic. For recruitment, we sent recruitment emails to two institution-wide listservs on July 1, 2021; presented the study at a leadership meeting that included all department chairs on July 15, 2021; and asked participating HPPs to suggest additional participants. We recruited until no new themes emerged or when data saturation was achieved [ ].Data Collection
The research team developed a semistructured interview guide to assess oncology HPPs’ experiences with telehealth during COVID-19 (
). The interviews assessed how HPPs were using telehealth, adaptations necessary to shift in-person care to virtual, and perceived successes, challenges, and lessons learned. The interview guide was developed based on the Consolidated Framework for Implementation Research (CFIR) that identifies common, multilevel barriers and facilitators to implementing new programs, such as leadership support, resources, and individuals’ knowledge about the intervention [ ]. The guide was structured around the five CFIR domains and focused on constructs most relevant to telehealth implementation, including the following: (1) intervention characteristics (eg, ease of implementation of telehealth platform), (2) inner setting characteristics (eg, access to information about the intervention), (3) outer setting characteristics (eg, reimbursement and licensure policies related to telehealth), (4) individual characteristics (eg, patient and HPP knowledge about telehealth), and (5) process (eg, planning for telehealth implementation).Participants were also asked about the future of oncology telehealth and whether they were supportive of using telehealth beyond COVID-19. An individual trained in qualitative methods conducted 40 individual interviews from August to December 2020 with HPPs via Zoom videoconference. An audio recording was generated through QuickTime Player (Apple Inc). We limited the interviews to 30 minutes to ensure that HPPs could participate. To stay close to the 30-minute time frame, we alerted all participants when 25 minutes had passed, assessed how many questions were left at the 25-minute mark, and if more than one question was left, we asked participants if they could stay on for a few additional minutes to finish all the interview questions. We were able to ask all participants all the interview questions using this approach. Participants provided informed consent via videoconference. Participants were asked to provide oral consent for interview participation and then asked to provide oral consent to having the interview recorded. The interviews were recorded and transcribed verbatim by a professional transcription service (GMR Transcription Services, Inc). The transcripts were deidentified (ie, all identifying information about participants was removed) and were assigned a participant ID. The Advarra Institutional Review Board reviewed and exempted this study. Participants were not compensated for their participation.
Data Analyses
We applied a hybrid approach of integrating deductive and inductive coding, which is commonly used in implementation science research and allows researchers to use theory-driven and data-driven coding [
, ]. We developed a codebook based on deductive codes [ ] from the interview guide and inductive codes [ ] generated from themes within the data. The interviews were coded by two independent coders using Dedoose software (version 4.12). The coders applied codes to an initial set of transcripts (n=5) independently, compared and refined coding until consensus was achieved, and finished coding the remaining transcripts. A high level of interrater reliability (κ=0.81) was achieved. The κ coefficient was defined as the number of characters coded and not coded by both coders (ie, agreement) divided by the total number of characters [ ]. A summary of key themes was sent to 5 participants to confirm that the researchers’ interpretation of the data was consistent with participants’ experiences (ie, member checking) [ ]. We reported on themes that were discussed by a minimum of 10 participants to ensure that we were summarizing central themes. Within themes, we noted when there was consensus or divergence across HPPs. The findings are reported based on the Consolidated Criteria for Reporting Qualitative Studies checklist [ ].Results
Participant Characteristics
Approximately half of the 40 participants were physicians (n=22, 55%), and one-quarter of the participants were APPs (n=10, 25%;
). Other participants included social workers (n=3, 8%), psychologists (n=2, 5%), dieticians (n=2, 5%), and a pharmacist (n=1, 3%). Most participants were female (n=24, 60%). Participants represented 17 clinical areas. Few participants (n=6, 15%) had used telehealth prior to COVID-19, but all were using telehealth during COVID-19. Participants used telehealth across the cancer care continuum, including screening, diagnosis and follow-up, surveillance, supportive care, procedure preparation and follow-up, and survivorship care.Characteristics | Value (N=40) | |
Job role, n (%)a | ||
Advanced practice providerb | 10 (25) | |
Dietician | 2 (5) | |
Pharmacist | 1 (3) | |
Physicianc | 22 (55) | |
Psychologist | 2 (5) | |
Social worker | 3 (8) | |
Gender, n (%) | ||
Female | 24 (60) | |
Male | 16 (40) | |
Clinical focus, n (%) | ||
Breast oncology | 2 (5) | |
Bone marrow transplant | 2 (5) | |
Cutaneous oncology | 2 (5) | |
Endocrinology | 1 (3) | |
Gastrointestinal oncology | 4 (10) | |
Genitourinary oncology | 3 (8) | |
Gynecologic oncology | 2 (5) | |
Head and neck cancer | 2 (5) | |
Interventional radiology | 2 (5) | |
Neuro-oncology | 1 (3) | |
Radiation oncology | 2 (5) | |
Sarcoma | 3 (8) | |
Senior adult | 1 (3) | |
Social work | 3 (8) | |
Supportive care | 4 (10) | |
Survivorship clinic | 2 (5) | |
Thoracic oncology | 4 (10) | |
Clinical affiliation, n (%) | ||
Single-site practice | 37 (93) | |
Multisite practice | 3 (8) | |
Job tenure (years), mean (SD) | 12.5 (6.9) | |
Virtual visits pre–COVID-19, n (%) | ||
Yes | 6 (15) | |
No | 34 (85) | |
Virtual visits during COVID-19, n (%) | ||
Yes | 40 (100) | |
No | 0 (0) |
aSome of the percentages may not add to 100 due to rounding.
bAdvanced practice providers included nurse practitioners and physician assistants.
cPhysicians included endocrinologists, medical oncologists, palliative care specialists, psychiatrists, radiation oncologists, radiologists, and surgeons.
Qualitative Analysis
Five key themes were identified: (1) establishing and maintaining patient-HPP relationships, (2) coordinating care with other HPPs and informal caregivers, (3) adapting in-person assessments for telehealth, (4) developing workflows and allocating resources, and (5) future recommendations. For each theme, the codes, code definitions, and frequency of use across participants are presented in tables in the following sections, while illustrative quotations are presented in the sections following the tables.
Theme 1: Establishing and Maintaining Patient-HPP Relationships
Overview
Participants described how telehealth changed the patient-HPP relationship, including information exchange, patient engagement, and emotional response (
).Parent code and child codes | Code definition | Code frequency across participants (N=40), n (%) | |
Patient-HPPa communication | |||
Patient receptivity to information | Apply code when participant discusses how patient-HPP communication is easier due to patient’s increased receptivity to receive information in their home environment. | 21 (53) | |
Easier to share screen with patient to display results | Apply code when participant discusses how it is easier to share screen and display results with a patient during telehealth visits. | 12 (30) | |
Patient-initiated discussion and questions | Apply code when participant describes how telehealth visits affect patients’ willingness to initiate discussion about condition or ask questions. | 11 (28) | |
Patient-HPP engagement | |||
Requires more energy from the HPP | Apply code when participant describes having to put on an act, be more dynamic, or put more energy into telehealth visits to engage patients. | 27 (68) | |
Value of video | Apply code when participant describes how the video component of telehealth visits is important for patient-HPP engagement. | 10 (25) | |
Lack of physical connection | Apply code when participant describes how lack of physical connection (ie, ability to touch patient) affects the delivery of medicine through telehealth. | 12 (30) | |
Communicating difficult news | Apply code when participant describes the challenges of delivering difficult news through telehealth (eg, new and serious diagnosis). | 23 (58) |
aHPP: health care provider and professional.
Information Exchange
Participants felt that patient education was easier to deliver virtually because patients were more relaxed at home compared to the clinic and may be more receptive to receiving information. For example, one dietician explained as follows:
Because they’re typically at home, they’re more relaxed. They’re not in this clinic environment. So, it’s almost like they’re a little bit more receptive to what you’re saying because they’re home in their own comfortable environment.
Patient Engagement
In contrast, some participants felt that patients were more reluctant to speak up and ask questions during telehealth visits compared to in-person visits, making information exchange more difficult. One APP described this as follows:
I think the connection, it’s not always there. It makes it a little harder to have that quick back-and-forth dialogue. When you’re trying to get a lot of information across, I feel like patients are listening more than they are having a conversation back with you.
Participants noted that patient engagement was more challenging during telehealth visits. Participants described how telehealth visits required more energy from HPPs. One APP said the following:
When I’m talking to patients over Zoom, those are really shorter visits. There’s not as much discussion over Zoom. I fear that we’re gonna miss some things because of it. I try to slow down a little bit during the Zoom visit and really try to hold their attention a little bit longer. It requires more focus on my part than an in-person visit.
Participants also noted that engagement was harder during virtual check-ins delivered by phone compared to telehealth visits with video. One social worker indicated the following:
Some people they’re not using their picture or they’re calling in, so you don’t know what’s going on. So, something does get lost.
Emotional Response
Some participants also found it more challenging to respond to patients’ emotions during telehealth visits. One physician described the following:
I think sometimes it’s nice when you can hold somebody’s hand or give them a hug. Obviously, now, the COVID world, you know, we’re not able to do that as much. But especially through a screen when you’re not there in person, you lose a bit of that connection.
As a result, some HPPs felt that certain tasks, such as delivering difficult news, are better suited for in-person visits. For example, one APP shared the following:
This is a new diagnosis. They should be in the clinic so that I can emotionally support them, offer other services while they're in here—if they need to talk to the social worker. Otherwise, they may hang up, and it may be too devastating, and we may not be able to connect again.
Theme 2: Coordinating Care With Other HPPs and Informal Caregivers
Overview
Participants indicated that telehealth made it easier to coordinate care with other HPPs and informal caregivers, such as family members or friends (
). This theme was primarily discussed by health care providers (eg, physicians and APPs) as opposed to other health care professionals (eg, dieticians).Parent code and child codes | Code definition | Code frequency across participants (N=40), n (%) | |||
HPPa-HPP coordination | |||||
Coordinating with external HPPs | Apply code when participant describes coordinating telehealth visits with health care HPPs external to Moffittb. | 18 (45) | |||
Coordinating with internal HPPs | Apply code when participant describes coordinating telehealth visits with health care HPPs internal to Moffitt (eg, other specialties). | 22 (55) | |||
HPP-caregiver coordination | |||||
Allowing caregivers to join in-person visits through telehealth | Apply code when participant describes using Zoom or other platform to allow caregiver to participant in a patient’s in-person visit. | 30 (75) | |||
Allowing caregivers to join patients’ telehealth visits | Apply code when participant describes including caregivers in patient’s telehealth visit. | 25 (63) |
aHPP: health care provider and professional.
bMoffitt: Moffitt Cancer Center.
Care Coordination With HPPs
Participants discussed how telehealth made it possible to bring together HPPs within the same institution. One physician recalled the following:
I’ve had one of the surgeons, myself, and the medical oncologist, and the radiation oncologist, and the patient all on Zoom at one time, so true multidisciplinary care provided through the Zoom platform. I’ve done that in one instance where the patient was in the room with me and I was doing the physical exam and we Zoomed-in the medical oncologist as part of a kind of multidisciplinary assessment.
Participants also described how telehealth visits made it much easier to coordinate care with HPPs outside of Moffitt. For example, one physician described the following:
I’ve got a few patients that I share with oncologists in other parts of the country and there’s one particularly memorable patient in my mind that I saw while he was at home as part of his visit with one of the medical oncologists at [name of organization]. So, the three of us had a three-way conversation. So, we set up interdisciplinary, interinstitutional care for that patient because he’s a snowbird and winters down here and summers up there and so we shared his care. That went remarkably well.
Care Coordination With Informal Caregivers
Participants described how telehealth made it easier to coordinate information with multiple informal caregivers (eg, inviting family members to a visit to go over a patient’s prognosis). One physician said the following:
We used Zoom so the whole family could be there. I was able to share my screen, demonstrating the tumor. They thought that was just the greatest. And, so everybody was able to be together for all that information.
Participants also used telehealth to enhance in-person visits, such as calling or videoconferencing caregivers who could not participate in the in-person visit due to on-site guest restrictions during COVID-19. A physician recalled the following:
It allowed them to have their family with them when we went over the results. Because, right now, we’re still not allowing family in the outpatient clinic. And, so it allowed our patients to be with their families so that they can ask questions and hear everything at the same time.
Theme 3: Adapting In-Person Assessments for Telehealth
Overview
Participants identified challenges with patient assessments (eg, obtaining vital signs, patient-reported outcomes [PROs], and images) and conducting physical exams during telehealth visits (
).Parent code and child codes | Code definition | Code frequency across participants (N=40), n (%) | |
Lack of physical exam | |||
Inability to examine lymph nodes | Apply code when participant describes the inability to feel, measure, or inspect a patient’s lymph nodes during a telehealth visit. | 12 (30) | |
Missing a clinical problem | Apply code when participant describes concerns over missing a clinical problem because of the inability to visualize the patient during a telehealth visit. | 26 (65) | |
Getting the patient involved in the exam | Apply code when participants describe strategies for getting the patient to help with the physical exam during the telehealth visit. | 16 (40) | |
Lack of data | |||
Images | Apply code when participants describe challenges with image resolution during telehealth visits to visualize condition (eg, wound monitoring). | 15 (38) | |
Vital signs and other biometrics | Apply code when participant describes not having access to vital sign or other biometric data (eg, blood pressure) that is relevant for clinical decision-making. | 22 (55) | |
Patient-reported outcomes | Apply code when participant describes not having access to patient-reported outcomes as a barrier for telehealth visit delivery. | 11 (28) |
Vital Sign Collection
Some participants lacked data due to scheduling challenges (eg, having a virtual visit scheduled before imaging data were available) or lack of remote monitoring (ie, ability to gather data between in-person visits). A few participants described how some patients would obtain their own vital sign data (eg, from wearable devices) and report it during the visit. As an example, one APP said the following:
It would be helpful to have vitals. I have one patient who does this by themselves, checking their own vitals (heart rate, oxygen saturation) and their blood pressure and they share it during the visit, which is great.
Patient-Reported Outcomes
Some participants described how it was challenging to deliver certain types of care (eg, supportive care) virtually without access to PROs (eg, depression symptoms). One psychologist explained as follows:
We used to collect the ESAS [Edmonton Symptom Assessment System] [] for patients in person, but now, if we see a patient virtually, we don’t have that information. For supportive care, monitoring depressive symptoms is really important and now we don’t have that. We talked about having nurses call the patient to collect it over the phone before the virtual visit, but that is really not the same as getting the symptoms from the patient perspective.
During COVID-19, some clinics suspended in-person collection of PROs and did not have a means for collecting PROs as a part of virtual care. Participants also described how image quality could be challenging, requiring HPPs to follow up and obtain images after the visit. One physician indicated the following:
Some patients might have a limitation in the internet provider bandwidth, so the image resolution is poor quality. In that circumstance, what I do is I tell the patient, “Please take a picture of your surgical site with your phone, and email it to me.” But it would be nice if there was a more systematic way to do this so I could see the image during the visit.
Image Collection
Participants felt that most patients were comfortable sharing images with HPPs digitally and did not cite concerns about privacy. A few HPPs noted that patients over 65 years of age were more reluctant to share images due to privacy concerns compared to younger patients. One APP said the following:
Our clinic sees a lot of older patients [over 65 years] who tend to have more privacy issues with sharing images than our younger patients.
Physical Exams
Participants were also concerned about missing important clinical problems due to the lack of a physical exam (eg, checking lymph nodes). One APP said the following:
I really do think there’s something lost, the personal touch and the things you see with your eyes and a physical exam for a cancer patient is very important. To feel for lymph adenopathy and wherever they say their cancer was, the nearest lymph node drainage. I mean, there’s no way that I could discern that over telemedicine.
Similarly, a physician explained as follows:
I see things on my patients all the time when I see them in person. I’m like, “You need to go get that little thing on your arm there checked.” It might be a new skin lesion that they need to have looked at. I worry about missing things during virtual visits when I can’t really visualize the patient.
Some participants described getting the patient to help with conducting the physical exam (eg, having the patient measure a visible tumor) during the telehealth visit to overcome this limitation. For example, one APP shared the following:
Sometimes, I’ve had patients who’ve had visible tumors on their neck or visible tumors, I can see it on telemedicine, but I can’t really measure it. So I ask the patient to go get a tape measure and measure it.
Theme 4: Developing Workflows and Allocating Resources
Overview
Participants described the importance of developing workflows for telehealth visits that are equivalent to in-person visits (eg, check-in process, EHR integration, and scheduling) and ensure sufficient resources are allocated for HPPs and patients (
).Parent code and child codes | Code definition | Code frequency across participants (N=40), n (%) | |
Workflow | |||
Check-in process | Apply code when participant describes the check-in process used during telehealth visits. | 28 (70) | |
Scheduling | Apply code when participants describe how telehealth visits are scheduled (eg, batching visits). | 33 (83) | |
Electronic health record (EHR) integration | Apply code when participant describes lack of Zoom EHR integration (eg, inability to find visit in EHR). | 16 (40) | |
Resources | |||
Equipment | Apply code when participants describe equipment necessary for telehealth delivery (eg, cameras). | 35 (88) | |
Space | Apply code when participants describe space where telehealth visit is conducted. | 36 (90) | |
Clerical support | Apply code when participants describe the amount of administrative support available for telehealth visit delivery. | 26 (65) | |
Information technology (IT) support for patients with low digital literacy | Apply code when participants discuss IT support for patients who may have low computer or mobile health literacy. | 27 (68) | |
Tools for patients with disabilities | Apply code when participants discuss need for tools for patients with disabilities (eg, closed captioning and speech-to-text tools). | 30 (75) |
Check-in Process
Participants, for example, felt that the patient check-in process, which is usually handled by staff, was left up to the HPP for telehealth visits, creating inefficiency. One physician described the following:
I really am concerned about the way we’ve implemented the virtual visits is we’ve cut the nurse and the PAR [Patient Access Representative] team out of the equation so now I’m having to check in my own patients when they show up on a Zoom visit. That becomes a little more cumbersome because any time you ask me to do an administrative task, the chance that I’m gonna execute it accurately is less than if you have a nurse doing it.
EHR Integration
Participants also found it challenging that telehealth visits were not integrated within the EHR, making it difficult to ensure that all the HPPs involved in preparing a visit (eg, nurse and APP) could access the visit link. One APP mentioned the following:
I have great nurses who work with me and they prep the clinics and they don’t have the Zoom invitation. So, the invitation comes to me, but it doesn’t go to the nurses working with me, which makes it hard for them to prep the visit. But if somehow, the Zoom meetings were discoverable in the EHR then that would really help.
Scheduling
Some participants described refining their schedule to ensure that telehealth visits were grouped together rather than interspersed between in-person visits. Having telehealth and in-person visits mixed together created problems, such as forcing HPPs to run back and forth between their clinic and office where telehealth visits were conducted. One physician mentioned the following:
So, for a month or two I was running around like a chicken without a head. And finally, I said, “Enough. We’re gonna have designated days where we’re doing all the virtual visits.” So, I can sit here and do one patient after the other because it was not working, running back and forth between clinics and my office where I take the virtual visits.
HPP Resources
Some participants described lacking sufficient resources for telehealth visits, including equipment (eg, cameras and headphones), space (eg, finding a private or well-lit space), and administrative support. One social worker shared the following:
How are we supposed to do virtual visits when we don’t have a camera? Here we are, five or six months into this pandemic and most of us have no access to a camera outside of our cellphones.
Participants also felt that the amount of administrative support for telehealth visits was inferior to in-person visits and, as a result, much of the administrative work fell on APPs and nurses. One APP mentioned the following:
Right now, our nurse is looking ahead at who’s scheduled and making sure everything is there for the visit. But, it’s almost like if there could be someone on the back end—like they do for new patient visits that are in person—doing that, it would help the nurses. It ends up being a lot of clerical work for the nurses, taking them away from patient education.
Patient Resources
Most participants felt that there were sufficient resources for patients, such as the interpreter service and IT support. Participants felt that the level of IT support worked for the majority of patients but was not sufficient for patients with low mobile health (mHealth) literacy or those with a lower ability to use mHealth apps with efficiency to accomplish a task [
]. As a result, some HPPs spent a significant amount of time helping their patients with the Zoom app during telehealth visits or converted telehealth visits to virtual check-ins by phone. One pharmacist described the following:The patients that struggled the most were patients who had to access the telehealth visit from their phone. They didn’t know how to download or locate the app. We need to make sure the training that patients receive covers how to use the app so that we don’t have to spend so much time on this during the visit.
Participants also indicated a need for more resources for patients with disabilities that may interfere with technology use. One psychologist described the following:
It was really challenging to meet virtually with patients who had trouble with hearing. I ended up converting those visits [to in person].
Similarly, a physician described the difficulty of delivering telehealth visits to patients with speech impairments, as follows:
We see patients who may not be able to talk after surgery. We would try to get the caregiver on or use the chat feature, but if they don’t have help at home, we’d have to bring them in.
Theme 5: Recommendations for Telehealth Implementation in the Future
Patient-Level Recommendations
Nearly all participants were supportive of continuing telehealth beyond COVID-19 but recommended changes to ensure implementation is sustainable (
). Participants provided recommendations at the patient, HPP, and organizational levels for improving telehealth implementation in the future. At the patient level, participants discussed the importance of overcoming the digital divide and recommended real-time IT support for patients, closed captioning for patients with communication-related disabilities, and educational materials on how to prepare for a telehealth visit (eg, finding a place that is comfortable). For example, one psychologist suggested the following:I would have said a bit more patient training, just some training on not just using Zoom, but telemedicine in general, and sort of optimizing the whole visit because I’m sure there are ways to do it. For example, encouraging them to find a comfortable space, have what they need on hand.
Parent code and child codes | Code definition | Code frequency across participants (N=40), n (%) | |||
Patient-level recommendations | |||||
Greater telehealth accessibility | Apply code when participants recommend strategies for improving the accessibility of telehealth (eg, closed captioning). | 31 (78) | |||
Real-time, information technology support | Apply code when participants recommend strategies to deliver more timely technology support to patients. | 12 (30) | |||
Patient education | Apply code when participants recommend strategies to deliver more patient education on how to use telehealth or prepare for telehealth visits. | 23 (58) | |||
HPPa-level recommendations | |||||
Sharing information about telehealth policy changes | Apply code when participants recommend strategies to promote discussion about ongoing telehealth policy changes (eg, licensure). | 21 (53) | |||
Sharing best practices | Apply code when participants recommend strategies to promote discussion about telehealth best practices (eg, tips for patient engagement and checklists). | 33 (83) | |||
“Webside manner” training | Apply code when participants recommend HPP-level training to engage patients in a telehealth environment (eg, how to maintain eye contact). | 32 (80) | |||
Production support | Apply code when participants recommend support needed to professionalize the telehealth visit (eg, background and lighting). | 26 (65) | |||
Organizational-level recommendations | |||||
Optimizing workflow | Apply code when participants recommend strategies for optimizing workflow (eg, virtual waiting room and batching telehealth visits). | 30 (75) | |||
Policy advocacy | Apply code when participants recommend organizational strategies for policy advocacy (eg, being more engaged with advocacy organizations). | 10 (25) | |||
Long-term planning | Apply code when participants recommend long-term planning strategies, such as how telehealth will be evaluated and how it will fit with other organizational priorities. | 20 (50) |
aHPP: health care provider and professional.
HPP-Level Recommendations
Participants thought that HPPs should have more training on engaging patients in a virtual environment (eg, optimal eye contact and communication strategies to create a dialogue). One APP mentioned the following:
I think more training would be helpful. Not on the technology, but what is the best way to maintain eye contact with patients, how do you keep patients engaged and talking with you.
Participants also wanted more discussion about best practices in telehealth use and updates on policy changes (eg, Medicare reimbursement). One physician said the following:
I wish there was more ongoing communication, not just deliver the news without an ongoing conversation. When Medicare is going to stop covering all the telemedicine visits is a question that we all have heard. It would be helpful to have a chance to discuss this.
Participants also wanted more resources to ensure that telehealth visits were delivered in a professional and consistent way across HPPs. For example, one physician shared the following:
Professionalizing it is the highest priority. Making sure the equipment is in place, that a dedicated room is in place, that the background is place, the lighting is in place. I think we should be trying to go in a direction more like a professional broadcasting company with that level of quality because I think that adds to the performance art that is at the heart of a lot of medicine.
Organizational-Level Recommendations
At the organizational level, participants suggested that Moffitt engage in telehealth policy advocacy (eg, reimbursement and out-of-state licensure). One physician recalled the following:
What we had asked that patient to do was to drive themselves across the border and to have the telemedicine visit from their car. I mean, it’s an absurdity. There’s nothing we can actionably do about that except work on our government relations team and kind of change that into the future.
Participants also had suggestions for optimizing workflow, such as having a virtual waiting room and letting patients know when an HPP is running late for a telehealth visit. For example, one APP shared the following:
When I go to my cardiologist, they have a virtual waiting room that is managed by their MAs [medical assistants]. So, I log in and the MA immediately says to me, “Hi. Glad you’re with us. The doctor is running 10 minutes behind.” In our scenario, if I’m not on time, the patients think there’s something wrong with the technology and they’ll be calling the nurse. They’ll be calling like crazy. And it’s just another thing for the nurse to have to pay attention to.
Participants also recommended developing a long-term strategy for telehealth, such as how telehealth aligns with other organizational priorities and an evaluation plan to see how telehealth affects patient outcomes and health care quality and costs. One physician shared the following:
I’d like to see a long-term vision and plans for evaluating progress. We don’t really know how it will impact patients or the care we deliver or reimbursement. We should see if it is being used to its full potential in other priority areas, like clinical trials.
Discussion
Principal Findings
The goal of this study was to capture oncology HPPs’ experiences with rapid implementation of telehealth during the COVID-19 pandemic. Overall, oncology HPPs saw telehealth as an integral part of health care delivery moving forward—a finding consistent with other studies [
, ]—but recommended addressing key barriers to improve sustainability. At Moffitt, we plan to present the findings of this research to our leadership team to determine how the results can inform future telehealth implementation. More broadly, our research findings also provide implications for future telehealth research and practice. For example, our findings suggest that more work is needed to overcome the digital divide and ensure that HPPs have access to the resources and data necessary to deliver high-quality care in a virtual environment (eg, vital signs). Further, the results suggest that a long-term strategy is needed to determine how telehealth will be integrated across the cancer care continuum and monitored to assess impact on patient outcomes and health care delivery. At the same time, health care systems will need to develop a research and policy agenda to ensure that evidence informs telehealth policy approaches, and that the regulatory and payment landscape accelerates and facilitates optimal telehealth use in cancer care.Addressing the Digital Divide
Health care systems have implemented innovative strategies to address the digital divide (ie, disparities in technology access, skills, and use) during COVID-19, such as assessing patients’ readiness for telehealth [
- ]. A key step to overcoming the digital divide is understanding which patients within a system are impacted by the digital divide and what specific barriers they experience. For example, smartphone-only internet access may bridge the digital divide for some patients, but it could limit health care access for others who have limited data plans or limited experience with mobile apps [ , ]. To address digital health literacy (ie, the ability to use computers and search for and evaluate health information electronically), some health care systems have deployed social work staff to help patients access telehealth [ , ]. In our system, we provided IT support to all patients with scheduled telehealth visits, but for some patients, more support was needed. Health care systems have also experimented with device loan programs and partnering with community-based organizations to create spaces where patients can connect to the internet [ , , ]. The Veterans Health Administration (VHA) recently developed a partnership with a cellphone carrier to ensure patients could access VHA telehealth apps without affecting their data limits [ ]. Prior studies also suggest that experience with technology is a key predictor of technology use for health care (eg, how often a patient uses the internet or a smartphone) [ ]. Further studies should explore longitudinal models of technology training that move beyond a one-time training and allow for repeated experiences with technology use for health care. Further, our health care team members found that current telehealth apps may not be optimized for patients with disabilities (eg, lack of closed captioning and speech-to-text tools), similar to other studies [ , , ]. To improve access among patients with disabilities and other conditions that could affect human-computer interaction (eg, low literacy), technology vendors should include patients who are affected by the digital divide in co-designing and testing of new products [ , , ]. Future studies should also consider community-based participatory research approaches in the development of digital health technologies to better engage patients affected by the digital divide [ ]. Community-based participatory research has been used in mHealth studies, for example, to increase community participation in app development, usability testing and app refinement, and designing recruitment, implementation, and dissemination strategies [ ].Our study participants noted that access to interpreters was a key ingredient of successful telehealth deployment. Prior studies have documented disparities in telehealth use among patients who prefer English and those who do not prefer English in the United States during the pandemic [
- ]. Similar disparities have been observed in patient portals, which are used by some health care systems to deliver telehealth [ , ]. Qualitative studies have documented barriers, such as limited access to professional interpreters, lack of bilingual HPPs, and lack of translation of COVID-19–related informational materials [ ]. Additional studies are needed to better understand telehealth access among patients who do not prefer English. For example, a previous study demonstrated that factors such as interpreter modality (eg, professional vs ad hoc and video vs in person) affect the accuracy of interpretation for health care visits [ ]. Future studies could compare different models of interpreter services through telehealth and compare interpretation accuracy rates and other outcomes, such as patient satisfaction. Researchers have also recommended that health care organizations develop monitoring systems for evaluating disparities in telehealth uptake by language preference (eg, dashboards), develop telehealth and patient portal trainings in multiple languages, and prioritize the hiring of bilingual HPPs [ ]. More research is needed to develop and test implementation strategies that address disparities in telehealth access based on language preference.Ensuring HPPs Have Sufficient Tools for Telehealth Implementation
HPPs may need additional resources to deliver telehealth effectively. HPP-facing tools [
, ] may be helpful in providing education on “webside manner” and implementation checklists [ , ]. HPPs at Moffitt indicated a need for guidance on the optimal way to conduct a telehealth visit (eg, lighting and eye contact), similar to other studies [ , ]. Health care systems could disseminate available trainings (eg, Academy of Communication in Healthcare training) or develop institution-specific trainings [ ]. Some health care systems have developed implementation checklists for health care team members that include helpful tips, such as confirming a patient’s phone number at the beginning of the virtual visit in case the technology fails [ ]. Patient-facing tools may also improve virtual patient-HPP communication. Some HPPs have developed patient handouts on how to conduct elements of a physical exam during a telehealth visit and how to prepare for the visit (eg, patient positioning) [ ]. Similar approaches could be tested more broadly. Other studies have noted that HPPs may lack sufficient digital health literacy [ , ], which can negatively affect engagement with health IT systems (eg, EHRs) [ ]. This concern was not raised by our participants, but future studies should explore the effects of HPPs’ eHealth literacy on telehealth implementation.Remote Monitoring for Telehealth Implementation
Access to patient data during telehealth visits, such as biometric data and PROs, is also critical to implementation success. A recent study in primary care found that blood pressure assessments declined by 37% during COVID-19, in part due to lack of biometric screening during virtual visits [
, ]. These findings highlight the need for remote monitoring programs, which will require health care systems to invest in complex change (eg, EHR integration of biometric data and optimized data visualization) [ - ]. Prior to COVID-19, there was limited reimbursement for remote monitoring, hindering health care system adoption [ ]. During the pandemic, Medicare has expanded payment policies for remote monitoring with certain restrictions (eg, type of data and minimum amount of data needed) [ ]. Some cancer care systems have started to invest in remote monitoring programs and cited recent changes in Medicare policy as a motivator for adoption [ ]. Cancer care systems will need to evaluate the effectiveness of remote monitoring programs and identify areas in which remote monitoring adds the most value. Like other studies, our research found that older patients may have more privacy concerns about remote monitoring and sharing patient-generated health data, such as images, compared to other patients [ , ]. Prior studies recommended strategies that strengthen patient activation and HPP-patient trust; they also recommended developing patient education programs about how data are being used and protected in order to overcome privacy-related barriers to sharing patient-generated data [ , ]. Other privacy-related barriers (eg, concerns about information security) were not mentioned by participants in our study but deserve consideration in future telehealth research. Organizations have reported examples of “Zoombombing,” or when an intruder joins a Zoom videoconference [ ], raising concerns about information security and telehealth. Researchers have recommended that health care organizations develop multipronged approaches (eg, employee training and simulated cyberattacks) to address information security threats in telehealth [ ]. Further, some have argued that health care organizations should transition from consumer-oriented videoconference tools that were adopted at the onset of the pandemic to health care–specific videoconference tools with additional security features [ ]. More research is needed to identify best practices in information security for telehealth as it grows in usage.Greater Evidence Regarding Effectiveness, Implementation, and Potential Risks of Telehealth
Cancer care systems, payers and insurers, and policy makers will need more evidence for the effectiveness and safety of oncology telehealth to guide future decisions. Research from other health care sectors has demonstrated that telehealth can be equivalent to in-person care for certain conditions and offers a relative advantage over in-person care (eg, reducing rural health care disparities) [
, - ]. Telehealth also has potential risks, such as inappropriate antibiotic prescribing or exacerbating existing health care disparities due to the digital divide [ , - ]. Within oncology, telehealth models for supportive and survivorship care and ancillary services (eg, genetic counseling) have proven effective, but there is limited evaluation of telehealth for other areas of care (eg, screening, diagnosis, treatment, and surveillance) [ - ]. Therefore, it is critical to evaluate telehealth use in these areas and assess impact on patient outcomes, health care quality, costs, equity, and potential risks (eg, inappropriate care). Further, research is needed to evaluate strategies for incentivizing the use of telehealth in a postpandemic landscape [ ]. One strategy may be alternative payment models, which have increased telehealth adoption in other health care sectors but are understudied in oncology [ ]. Further, as adoption decisions move from mandatory to voluntary, studies should test theories of technology adoption (eg, technology acceptance model) to examine what factors help explain sustained telehealth use beyond the pandemic [ , ].Policy Advocacy for Telehealth
Cancer care systems and key stakeholders will need to develop an agenda to ensure that future policies are supportive of oncology telehealth. During the pandemic, many state-level restrictions were lifted that made it easier to implement telehealth, including waiving out-of-state licensure requirements or expanding payment parity [
, ]. Further, many state Medicaid programs and commercial insurers changed telehealth policies in response to COVID-19 (eg, removing cost-sharing requirements) [ , ]. At the federal level, there was a major overhaul of Medicare telehealth payment policies (eg, allowing virtual check-ins through telephone to qualify for telehealth), resulting in 244 temporary regulatory changes [ , , ]. Uncertainty remains regarding which federal- and state-level policies will be retained in the future [ , ]. There has also been federal investment in overcoming the digital divide in the United States [ ]. The Infrastructure Investment and Jobs Act became public law recently and provides funding for expanding broadband access in low-income neighborhoods, reducing practices of digital redlining (ie, limited internet service provision in low-income and high–minority concentration neighborhoods), and expanding internet subsidies for individuals with limited economic resources [ ]. Future research will be needed to monitor program implementation and effectiveness for addressing digital disparities at the federal, state, and local levels. Moving forward, a policy agenda will need to include greater investment in telehealth research, addressing medical licensure and credentialing barriers [ ], and a balanced approach to regulation, one that continues to fuel innovation in telehealth for oncology while safeguarding against potential risks (eg, delivering telehealth for a condition that is not “tele-amenable” [ ]). For instance, among our study participants, some HPPs expressed concerns about using telehealth to deliver information about a new and serious cancer diagnosis. A recent survey among oncologists (n=29) identified similar findings: some oncologists were reluctant to use telehealth for delivering bad news [ ]. Additional studies are needed to determine optimal use of telehealth in oncology to guide future policy.Limitations
This paper has several limitations. First, this is a qualitative study from an NCI-Designated Comprehensive Cancer Center and the findings may not be generalizable to other settings. Second, there are other health care professionals (eg, genetic counselors) who have used telehealth as a part of their practice, and their experiences were not captured in our sample. Third, we limited the interviews to 30 minutes or less to increase participation, and we reduced the number of questions included in the interview guide. Therefore, some topics, such as how HPPs have used telehealth for clinical trials, were not discussed. Fourth, our study excluded residents and fellows who were important stakeholders in telehealth implementation. Future studies should examine the unique experiences of residents and fellows who are simultaneously learning how to deliver care in person and virtually. Finally, it was beyond the scope of this research to capture the patient perspective. Studies have started to assess patient experience with telehealth during the COVID-19 pandemic; however, there has been limited study of this in oncology [
- ]. Future studies should assess the perspectives of patients with cancer regarding telehealth to explore patient satisfaction, barriers and facilitators to telehealth access, and patient preferences for telehealth (eg, whether certain services should be delivered in person vs virtually and preferences for how interpreter services should be implemented).Conclusions
Overall, cancer care frontline HPPs have used innovative and adaptive strategies to rapidly implement telehealth during COVID-19 and are supportive of continuing virtual cancer care delivery beyond the COVID-19 pandemic. HPPs identified several facilitators for telehealth implementation, such as improved care coordination with other HPPs and informal caregivers. HPPs also noted several barriers, such as lack of physical examinations and vital sign information, which limited HPPs’ ability to fully evaluate a patient. To support the rapid growth of oncology telehealth, implementation strategies are needed to overcome the digital divide and ensure that HPPs and patients have the tools necessary to effectively engage in telehealth. Health care systems, policy makers, health insurers, and payers must develop long-term strategies for integrating telehealth into the cancer care continuum, building the evidence base around telehealth in oncology, and developing a policy agenda that will advance telehealth innovation while safeguarding against potential risks.
Authors' Contributions
KT developed the research question and study design, assisted with the qualitative analyses, and drafted the manuscript. MBB, CN, and ON helped refine the interview guide, assisted with conducting qualitative interviews, assisted with conducting the qualitative analyses, reviewed the manuscript draft, and provided feedback. BDG, LBO, ER, JEL, RJF, AAT, KBP, JHJ, NA, YRH, and HSLJ reviewed the summary report of the qualitative data analyses and the manuscript draft and provided feedback. PES helped refine aspects of the study design and methodology, provided input on the interview guide, assisted with participant recruitment, reviewed the manuscript draft, and provided feedback.
Conflicts of Interest
BDG has received funding from SureMed Compliance and KenPharm. JHJ is a consultant for HRA Pharma for unrelated work. HSLJ has received funding from RedHill BioPharma, Janssen Scientific Affairs, and Merck.
Interview guide.
DOCX File , 19 KBReferences
- Fagherazzi G, Goetzinger C, Rashid MA, Aguayo GA, Huiart L. Digital health strategies to fight COVID-19 worldwide: Challenges, recommendations, and a call for papers. J Med Internet Res 2020 Jun 16;22(6):e19284 [FREE Full text] [CrossRef] [Medline]
- Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML. Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US. JAMA Intern Med 2021 Mar 01;181(3):388-391. [CrossRef] [Medline]
- Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML. Variation in telemedicine use and outpatient care during the COVID-19 pandemic in the United States. Health Aff (Millwood) 2021 Feb;40(2):349-358. [CrossRef] [Medline]
- Mehrotra A, Bhatia RS, Snoswell CL. Paying for telemedicine after the pandemic. JAMA 2021 Feb 02;325(5):431-432. [CrossRef] [Medline]
- LeRouge C, Garfield MJ. Crossing the telemedicine chasm: Have the US barriers to widespread adoption of telemedicine been significantly reduced? Int J Environ Res Public Health 2013 Nov 28;10(12):6472-6484 [FREE Full text] [CrossRef] [Medline]
- Rogove HJ, McArthur D, Demaerschalk BM, Vespa PM. Barriers to telemedicine: Survey of current users in acute care units. Telemed J E Health 2012;18(1):48-53. [CrossRef] [Medline]
- Weinstein RS, Lopez AM, Joseph BA, Erps KA, Holcomb M, Barker GP, et al. Telemedicine, telehealth, and mobile health applications that work: Opportunities and barriers. Am J Med 2014 Mar;127(3):183-187. [CrossRef] [Medline]
- Scott KC, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: A systematic review. J Telemed Telecare 2018 Jan;24(1):4-12 [FREE Full text] [CrossRef] [Medline]
- Moffatt JJ, Eley DS. Barriers to the up-take of telemedicine in Australia--A view from providers. Rural Remote Health 2011;11(2):1581 [FREE Full text] [Medline]
- Richard JV, Wilcock AD, Schwamm LH, Uscher-Pines L, Zachrison KS, Siddiqui A, et al. Assessment of telestroke capacity in US hospitals. JAMA Neurol 2020 Aug 01;77(8):1035-1037. [CrossRef] [Medline]
- Zhang D, Wang G, Zhu W, Thapa JR, Switzer JA, Hess DC, et al. Expansion of telestroke services improves quality of care provided in super rural areas. Health Aff (Millwood) 2018 Dec;37(12):2005-2013. [CrossRef] [Medline]
- Pareek P, Vishnoi JR, Kombathula SH, Vyas RK, Misra S. Teleoncology: The youngest pillar of oncology. JCO Glob Oncol 2020 Sep;6:1455-1460 [FREE Full text] [CrossRef] [Medline]
- Shea CM, Turner K, Alishahi Tabriz A, North S. Implementation strategies for telestroke: A qualitative study of telestroke networks in North Carolina. Telemed J E Health 2019 Aug;25(8):708-716 [FREE Full text] [CrossRef] [Medline]
- Weigel G, Ramaswamy A, Sobel L, Salganicoff A, Cubanski J, Freed M. Opportunities and barriers for telemedicine in the US during the COVID-19 emergency and beyond. KFF. Washington, DC: Kaiser Family Foundation; 2020 May 11. URL: https://www.kff.org/womens-health-policy/issue-brief/opportunities-and-barriers-for-telemedicine-in-the-u-s-during-the-covid-19-emergency-and-beyond/ [accessed 2022-01-10]
- Royce TJ, Sanoff HK, Rewari A. Telemedicine for cancer care in the time of COVID-19. JAMA Oncol 2020 Nov 01;6(11):1698-1699. [CrossRef] [Medline]
- Kircher SM, Mulcahy M, Kalyan A, Weldon CB, Trosman JR, Benson AB. Telemedicine in oncology and reimbursement policy during COVID-19 and beyond. J Natl Compr Canc Netw 2020 Sep 30:1-7. [CrossRef] [Medline]
- Kang JJ, Wong RJ, Sherman EJ, Rybkin A, McBride SM, Riaz N, et al. The 3 Bs of cancer care amid the COVID-19 pandemic crisis: "Be safe, be smart, be kind"-A multidisciplinary approach increasing the use of radiation and embracing telemedicine for head and neck cancer. Cancer 2020 Sep 15;126(18):4092-4104 [FREE Full text] [CrossRef] [Medline]
- Knudsen KE, Willman C, Winn R. Optimizing the use of telemedicine in oncology care: Postpandemic opportunities. Clin Cancer Res 2021 Feb 15;27(4):933-936. [CrossRef] [Medline]
- Lonergan PE, Washington III SL, Branagan L, Gleason N, Pruthi RS, Carroll PR, et al. Rapid utilization of telehealth in a comprehensive cancer center as a response to COVID-19: Cross-sectional analysis. J Med Internet Res 2020 Jul 06;22(7):e19322 [FREE Full text] [CrossRef] [Medline]
- Darcourt JG, Aparicio K, Dorsey PM, Ensor JE, Zsigmond EM, Wong ST, et al. Analysis of the implementation of telehealth visits for care of patients with cancer in Houston during the COVID-19 pandemic. JCO Oncol Pract 2021 Jan;17(1):e36-e43. [CrossRef] [Medline]
- Jiang CY, El-Kouri NT, Elliot D, Shields J, Caram MEV, Frankel TL, et al. Telehealth for cancer care in veterans: Opportunities and challenges revealed by COVID. JCO Oncol Pract 2021 Jan;17(1):22-29. [CrossRef] [Medline]
- Medicare telemedicine health care provider fact sheet. CMS. Baltimore, MD: Centers for Medicare and Medicaid Services; 2020 Mar 17. URL: https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet [accessed 2022-01-10]
- Prasad A, Brewster R, Newman JG, Rajasekaran K. Optimizing your telemedicine visit during the COVID-19 pandemic: Practice guidelines for patients with head and neck cancer. Head Neck 2020 Jun;42(6):1317-1321 [FREE Full text] [CrossRef] [Medline]
- Chang PJ, Jay GM, Kalpakjian C, Andrews C, Smith S. Patient and provider-reported satisfaction of cancer rehabilitation telemedicine visits during the COVID-19 pandemic. PM R 2021 Dec;13(12):1362-1368 [FREE Full text] [CrossRef] [Medline]
- Elkaddoum R, Haddad FG, Eid R, Kourie HR. Telemedicine for cancer patients during COVID-19 pandemic: Between threats and opportunities. Future Oncol 2020 Jun;16(18):1225-1227 [FREE Full text] [CrossRef] [Medline]
- Tam S, Wu VF, Williams AM, Girgis M, Sheqwara JZ, Siddiqui F, et al. Disparities in the uptake of telemedicine during the COVID-19 surge in a multidisciplinary head and neck cancer population by patient demographic characteristics and socioeconomic status. JAMA Otolaryngol Head Neck Surg 2021 Feb 01;147(2):209-211. [CrossRef] [Medline]
- Drees J. Moffitt Cancer Center's virtual visits up 5,000% in response to COVID-19. Becker's Hospital Review. Chicago, IL: Becker's Hospital Review; 2020 May 15. URL: https://www.beckershospitalreview.com/oncology/moffitt-cancer-center-s-virtual-visits-up-5-000-in-response-to-covid-19.html [accessed 2022-01-10]
- Oncology. National Cancer Institute. Bethesda, MD: National Cancer Institute URL: https://www.cancer.gov/publications/dictionaries/cancer-terms/def/oncology [accessed 2022-01-10]
- Creswell JW, Miller DL. Determining validity in qualitative inquiry. Theory Pract 2000 Aug;39(3):124-130. [CrossRef]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci 2009;4:50 [FREE Full text] [CrossRef] [Medline]
- Hamilton AB, Finley EP. Qualitative methods in implementation research: An introduction. Psychiatry Res 2019 Oct;280:112516 [FREE Full text] [CrossRef] [Medline]
- Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods 2016 Nov 29;5(1):80-92 [FREE Full text] [CrossRef]
- Boyatzis RE. Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: SAGE Publications, Inc; 1998.
- Crabtree BF, Miller WL. A template approach to text analysis: Developing and using codebooks. In: Crabtree BF, Miller WL, editors. Doing Qualitative Research. 2nd edition. Thousand Oaks, CA: SAGE Publications, Inc; 1999:163-177.
- Brennan PF, Hays BJ. The kappa statistic for establishing interrater reliability in the secondary analysis of qualitative clinical data. Res Nurs Health 1992 Apr;15(2):153-158. [CrossRef] [Medline]
- Tong A, Sainsbury P, Craig J. Consolidated Criteria for Reporting Qualitative Research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357. [CrossRef] [Medline]
- Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. J Palliat Care 1991;7(2):6-9. [Medline]
- Lin TTC, Bautista JR. Understanding the relationships between mHealth apps' characteristics, trialability, and mHealth literacy. J Health Commun 2017 Dec;22(4):346-354. [CrossRef] [Medline]
- Rodriguez JA, Clark CR, Bates DW. Digital health equity as a necessity in the 21st Century Cures Act era. JAMA 2020 Jun 16;323(23):2381-2382. [CrossRef] [Medline]
- Lau J, Knudsen J, Jackson H, Wallach AB, Bouton M, Natsui S, et al. Staying connected in the COVID-19 pandemic: Telehealth at the largest safety-net system in the United States. Health Aff (Millwood) 2020 Aug;39(8):1437-1442. [CrossRef] [Medline]
- Nouri S, Khoong E, Lyles C, Karliner L. Addressing equity in telemedicine for chronic disease management during the Covid-19 pandemic. NEJM Catal Innov Care Deliv 2020 May 04;1(3):1-13 [FREE Full text] [CrossRef]
- Elston Lafata J, Smith AB, Wood WA, Fitzpatrick B, Royce TJ. Virtual visits in oncology: Enhancing care quality while designing for equity. JCO Oncol Pract 2021 May;17(5):220-223. [CrossRef] [Medline]
- DeGuzman PB, Bernacchi V, Cupp CA, Dunn B, Ghamandi BJF, Hinton ID, et al. Beyond broadband: Digital inclusion as a driver of inequities in access to rural cancer care. J Cancer Surviv 2020 Oct;14(5):643-652. [CrossRef] [Medline]
- Potdar R, Thomas A, DiMeglio M, Mohiuddin K, Djibo DA, Laudanski K, et al. Access to internet, smartphone usage, and acceptability of mobile health technology among cancer patients. Support Care Cancer 2020 Nov;28(11):5455-5461. [CrossRef] [Medline]
- Annaswamy TM, Verduzco-Gutierrez M, Frieden L. Telemedicine barriers and challenges for persons with disabilities: COVID-19 and beyond. Disabil Health J 2020 Oct;13(4):100973 [FREE Full text] [CrossRef] [Medline]
- Valdez RS, Rogers CC, Claypool H, Trieshmann L, Frye O, Wellbeloved-Stone C, et al. Ensuring full participation of people with disabilities in an era of telehealth. J Am Med Inform Assoc 2021 Feb 15;28(2):389-392 [FREE Full text] [CrossRef] [Medline]
- Crawford A, Serhal E. Digital health equity and COVID-19: The innovation curve cannot reinforce the social gradient of health. J Med Internet Res 2020 Jun 02;22(6):e19361 [FREE Full text] [CrossRef] [Medline]
- Brewer LC, Fortuna KL, Jones C, Walker R, Hayes SN, Patten CA, et al. Back to the future: Achieving health equity through health informatics and digital health. JMIR Mhealth Uhealth 2020 Jan 14;8(1):e14512 [FREE Full text] [CrossRef] [Medline]
- Rodriguez JA, Saadi A, Schwamm LH, Bates DW, Samal L. Disparities in telehealth use among California patients with limited English proficiency. Health Aff (Millwood) 2021 Mar;40(3):487-495. [CrossRef] [Medline]
- Eberly LA, Kallan MJ, Julien HM, Haynes N, Khatana SAM, Nathan AS, et al. Patient characteristics associated with telemedicine access for primary and specialty ambulatory care during the COVID-19 pandemic. JAMA Netw Open 2020 Dec 01;3(12):e2031640 [FREE Full text] [CrossRef] [Medline]
- Lott A, Campbell KA, Hutzler L, Lajam CM. Telemedicine utilization at an academic medical center during COVID-19 pandemic: Are some patients being left behind? Telemed J E Health 2022 Jan;28(1):44-50. [CrossRef] [Medline]
- Schenker RB, Laguna MC, Odisho AY, Okumura MJ, Burnett H. Are we reaching everyone? A cross-sectional study of telehealth inequity in the COVID-19 pandemic in an urban academic pediatric primary care clinic. Clin Pediatr (Phila) 2022 Jan;61(1):26-33 [FREE Full text] [CrossRef] [Medline]
- Ancker JS, Barrón Y, Rockoff ML, Hauser D, Pichardo M, Szerencsy A, et al. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med 2011 Oct;26(10):1117-1123 [FREE Full text] [CrossRef] [Medline]
- Ancker J, Nosal S, Hauser D, Way C, Calman N. Access policy and the digital divide in patient access to medical records. Health Policy Technol 2017 Mar;6(1):3-11. [CrossRef]
- Kucirek NK, Thomas NJ, Norman JS, Athavale P, Jaradeh K, Hsiang EY, et al. Stories from COVID-19 reveal hospitalized patients with limited English proficiency have always been uniquely prone to social isolation. J Gen Intern Med 2021 Mar;36(3):786-789 [FREE Full text] [CrossRef] [Medline]
- Nápoles A, Santoyo-Olsson J, Karliner L, Gregorich S, Pérez-Stable E. Inaccurate language interpretation and its clinical significance in the medical encounters of Spanish-speaking Latinos. Med Care 2015;53(11):940-947. [CrossRef]
- Rodriguez JA, Casillas A, Cook BL, Marlin RP. The language of equity in digital health: Prioritizing the needs of limited English proficient communities in the patient portal 2.0. J Health Care Poor Underserved 2021;32(2S):211-219. [CrossRef]
- Chua IS, Jackson V, Kamdar M. Webside manner during the COVID-19 pandemic: Maintaining human connection during virtual visits. J Palliat Med 2020 Nov;23(11):1507-1509. [CrossRef] [Medline]
- West HJ. Telemedicine in oncology: Delivering on an overdue promise in the COVID-19 era. Front Oncol 2020;10:578888 [FREE Full text] [CrossRef] [Medline]
- Zulman DM, Verghese A. Virtual care, telemedicine visits, and real connection in the era of COVID-19: Unforeseen opportunity in the face of adversity. JAMA 2021 Feb 02;325(5):437-438. [CrossRef] [Medline]
- Telehealth communication: Quick tips. Academy of Communication in Healthcare. Lexington, KY: Academy of Communication in Healthcare URL: http://www.achonline.org/Telehealth [accessed 2022-01-10]
- Banerjee SC, Staley JM, Howell F, Malling C, Moreno A, Kotsen C, et al. Communicating effectively via tele-oncology (Comskil TeleOnc): A guide for best practices for communication skills in virtual cancer care. J Cancer Educ 2021 Feb 05:1-6 [FREE Full text] [CrossRef] [Medline]
- Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov;8(4):e27 [FREE Full text] [CrossRef] [Medline]
- Kirchberg J, Fritzmann J, Weitz J, Bork U. eHealth literacy of German physicians in the pre-COVID-19 era: Questionnaire study. JMIR Mhealth Uhealth 2020 Oct 16;8(10):e20099 [FREE Full text] [CrossRef] [Medline]
- Kuek A, Hakkennes S. Healthcare staff digital literacy levels and their attitudes towards information systems. Health Informatics J 2020 Mar;26(1):592-612 [FREE Full text] [CrossRef] [Medline]
- Alexander GC, Tajanlangit M, Heyward J, Mansour O, Qato DM, Stafford RS. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020 Oct 01;3(10):e2021476 [FREE Full text] [CrossRef] [Medline]
- Herzer KR, Pronovost PJ. Ensuring quality in the era of virtual care. JAMA 2021 Feb 02;325(5):429-430. [CrossRef] [Medline]
- Dinh-Le C, Chuang R, Chokshi S, Mann D. Wearable health technology and electronic health record integration: Scoping review and future directions. JMIR Mhealth Uhealth 2019 Sep 11;7(9):e12861 [FREE Full text] [CrossRef] [Medline]
- Zhu H, Colgan J, Reddy M, Choe EK. Sharing patient-generated data in clinical practices: An interview study. In: Proceedings of the American Medical Informatics Association Annual Symposium. 2016 Presented at: American Medical Informatics Association Annual Symposium; November 12-16, 2016; Chicago, IL p. 1303-1312 URL: http://europepmc.org/abstract/MED/28269928
- Simblett S, Greer B, Matcham F, Curtis H, Polhemus A, Ferrão J, et al. Barriers to and facilitators of engagement with remote measurement technology for managing health: Systematic review and content analysis of findings. J Med Internet Res 2018 Jul 12;20(7):e10480 [FREE Full text] [CrossRef] [Medline]
- Rodbard D. Continuous glucose monitoring: A review of successes, challenges, and opportunities. Diabetes Technol Ther 2016 Dec;18 Suppl 2:S3-S13 [FREE Full text] [CrossRef] [Medline]
- Abdolkhani R, Gray K, Borda A, DeSouza R. Patient-generated health data management and quality challenges in remote patient monitoring. JAMIA Open 2019 Dec;2(4):471-478 [FREE Full text] [CrossRef] [Medline]
- Health Care: Telehealth and Remote Patient Monitoring Use in Medicare and Selected Federal Programs. GAO-17-365. Washington, DC: Government Accountability Office; 2017 Apr. URL: https://www.gao.gov/assets/gao-17-365.pdf [accessed 2022-01-10]
- Final policy, payment, and quality provisions changes to the Medicare physician fee schedule for calendar year 2021. CMS. 2020 Dec 01. URL: https://www.cms.gov/newsroom/fact-sheets/final-policy-payment-and-quality-provisions-changes-medicare-physician-fee-schedule-calendar-year-1 [accessed 2022-01-10]
- Wicklund E. Mount Sinai extends cancer care to the home with new telehealth program. mHealthIntelligence. Danvers, MA: Xtelligent Healthcare Media; 2021 Jan 20. URL: https://mhealthintelligence.com/news/mount-sinai-extends-cancer-care-to-the-home-with-new-telehealth-program [accessed 2022-01-10]
- Karampela M, Ouhbi S, Isomursu M. Connected health user willingness to share personal health data: Questionnaire study. J Med Internet Res 2019 Nov 27;21(11):e14537 [FREE Full text] [CrossRef] [Medline]
- Serrano KJ, Yu M, Riley WT, Patel V, Hughes P, Marchesini K, et al. Willingness to exchange health information via mobile devices: Findings from a population-based survey. Ann Fam Med 2016;14(1):34-40 [FREE Full text] [CrossRef] [Medline]
- Ling C, Balcı U, Blackburn J, Stringhini G. A first look at Zoombombing. In: Proceedings of the IEEE Symposium on Security and Privacy. 2021 Presented at: IEEE Symposium on Security and Privacy; May 24-27, 2021; San Francisco, CA p. 1452-1467. [CrossRef]
- Jalali MS, Landman A, Gordon WJ. Telemedicine, privacy, and information security in the age of COVID-19. J Am Med Inform Assoc 2021 Mar 01;28(3):671-672 [FREE Full text] [CrossRef] [Medline]
- Kepplinger J, Barlinn K, Deckert S, Scheibe M, Bodechtel U, Schmitt J. Safety and efficacy of thrombolysis in telestroke: A systematic review and meta-analysis. Neurology 2016 Sep 27;87(13):1344-1351. [CrossRef] [Medline]
- Albritton J, Maddox L, Dalto J, Ridout E, Minton S. The effect of a newborn telehealth program on transfers avoided: A multiple-baseline study. Health Aff (Millwood) 2018 Dec;37(12):1990-1996. [CrossRef] [Medline]
- Pekmezaris R, Tortez L, Williams M, Patel V, Makaryus A, Zeltser R, et al. Home telemonitoring in heart failure: A systematic review and meta-analysis. Health Aff (Millwood) 2018 Dec;37(12):1983-1989. [CrossRef] [Medline]
- Dharmar M, Kuppermann N, Romano PS, Yang NH, Nesbitt TS, Phan J, et al. Telemedicine consultations and medication errors in rural emergency departments. Pediatrics 2013 Dec;132(6):1090-1097. [CrossRef] [Medline]
- Mehrotra A, Paone S, Martich GD, Albert SM, Shevchik GJ. A comparison of care at e-visits and physician office visits for sinusitis and urinary tract infection. JAMA Intern Med 2013 Jan 14;173(1):72-74 [FREE Full text] [CrossRef] [Medline]
- Ray KN, Shi Z, Gidengil CA, Poon SJ, Uscher-Pines L, Mehrotra A. Antibiotic prescribing during pediatric direct-to-consumer telemedicine visits. Pediatrics 2019 May;143(5):e20182491 [FREE Full text] [CrossRef] [Medline]
- Rodriguez JA, Betancourt JR, Sequist TD, Ganguli I. Differences in the use of telephone and video telemedicine visits during the COVID-19 pandemic. Am J Manag Care 2021 Jan;27(1):21-26 [FREE Full text] [CrossRef] [Medline]
- Barnett ML, Ray KN, Souza J, Mehrotra A. Trends in telemedicine use in a large commercially insured population, 2005-2017. JAMA 2018 Nov 27;320(20):2147-2149 [FREE Full text] [CrossRef] [Medline]
- Mehrotra A, Jena AB, Busch AB, Souza J, Uscher-Pines L, Landon BE. Utilization of telemedicine among rural Medicare beneficiaries. JAMA 2016 May 10;315(18):2015-2016 [FREE Full text] [CrossRef] [Medline]
- Douglas MD, Xu J, Heggs A, Wrenn G, Mack DH, Rust G. Assessing telemedicine utilization by using Medicaid claims data. Psychiatr Serv 2017 Feb 01;68(2):173-178 [FREE Full text] [CrossRef] [Medline]
- Freeman LW, White R, Ratcliff CG, Sutton S, Stewart M, Palmer JL, et al. A randomized trial comparing live and telemedicine deliveries of an imagery-based behavioral intervention for breast cancer survivors: Reducing symptoms and barriers to care. Psychooncology 2015 Aug;24(8):910-918 [FREE Full text] [CrossRef] [Medline]
- Ferguson RJ, Sigmon ST, Pritchard AJ, LaBrie SL, Goetze RE, Fink CM, et al. A randomized trial of videoconference-delivered cognitive behavioral therapy for survivors of breast cancer with self-reported cognitive dysfunction. Cancer 2016 Jun 01;122(11):1782-1791 [FREE Full text] [CrossRef] [Medline]
- Pang L, Liu Z, Lin S, Liu Z, Liu H, Mai Z, et al. The effects of telemedicine on the quality of life of patients with lung cancer: A systematic review and meta-analysis. Ther Adv Chronic Dis 2020;11:1-12 [FREE Full text] [CrossRef] [Medline]
- van den Brink JL, Moorman PW, de Boer MF, Hop WCJ, Pruyn JFA, Verwoerd CDA, et al. Impact on quality of life of a telemedicine system supporting head and neck cancer patients: A controlled trial during the postoperative period at home. J Am Med Inform Assoc 2007;14(2):198-205 [FREE Full text] [CrossRef] [Medline]
- Schwartz MD, Valdimarsdottir HB, Peshkin BN, Mandelblatt J, Nusbaum R, Huang A, et al. Randomized noninferiority trial of telephone versus in-person genetic counseling for hereditary breast and ovarian cancer. J Clin Oncol 2014 Mar 01;32(7):618-626 [FREE Full text] [CrossRef] [Medline]
- Hilgart JS, Hayward JA, Coles B, Iredale R. Telegenetics: A systematic review of telemedicine in genetics services. Genet Med 2012 Sep;14(9):765-776. [CrossRef] [Medline]
- Agrawal S, Ghandi T. Telehealth should be expanded—If it can address today’s health care challenges. Health Affairs. Bethesda, MD: Project HOPE; 2020 Sep 23. URL: https://www.healthaffairs.org/do/10.1377/hblog20200916.264569/full/ [accessed 2022-01-10]
- Zhao M, Hamadi H, Haley DR, Xu J, White-Williams C, Park S. Telehealth: Advances in alternative payment models. Telemed J E Health 2020 Dec;26(12):1492-1499. [CrossRef] [Medline]
- Kulviwat S, Bruner II GC, Kumar A, Nasco SA, Clark T. Toward a unified theory of consumer acceptance technology. Psychol Mark 2007 Dec;24(12):1059-1084. [CrossRef]
- Lee Y, Kozar KA, Larsen KR. The technology acceptance model: Past, present, and future. Commun Assoc Inf Syst 2003;12:752-780. [CrossRef]
- LaPointe J. Biden administration inherits slew of Medicare regulatory changes. RevCycleIntelligence. Danvers, MA: Xtelligent Healthcare Media; 2021 Jan 26. URL: https://revcycleintelligence.com/news/biden-administration-inherits-slew-of-medicare-regulatory-changes [accessed 2022-01-10]
- H.R.3684 - Infrastructure Investment and Jobs Act. Congress.gov. 2021. URL: https://www.congress.gov/bill/117th-congress/house-bill/3684/text [accessed 2022-01-10]
- Bell DL, Katz MH. Modernize medical licensing, and credentialing, too-Lessons from the COVID-19 pandemic. JAMA Intern Med 2021 Mar 01;181(3):312-315. [CrossRef] [Medline]
- Heyer A, Granberg RE, Rising KL, Binder AF, Gentsch AT, Handley NR. Medical oncology professionals' perceptions of telehealth video visits. JAMA Netw Open 2021 Jan 04;4(1):e2033967 [FREE Full text] [CrossRef] [Medline]
- Singh A, Mountjoy N, McElroy D, Mittal S, Al Hemyari B, Coffey N, et al. Patient perspectives with telehealth visits in cardiology during COVID-19: Online patient survey study. JMIR Cardio 2021 Jan 22;5(1):e25074 [FREE Full text] [CrossRef] [Medline]
- Auchus IC, Jaradeh K, Tang A, Marzan J, Boslett B. Transitioning to telehealth during the COVID-19 pandemic: Patient perspectives and attendance at an HIV clinic in San Francisco. AIDS Patient Care STDS 2021 Jul;35(7):249-254. [CrossRef] [Medline]
- Holcomb D, Faucher M, Bouzid J, Quint-Bouzid M, Nelson DB, Duryea E. Patient perspectives on audio-only virtual prenatal visits amidst the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic. Obstet Gynecol 2020;136(2):317-322. [CrossRef]
Abbreviations
APP: advanced practice provider |
CFIR: Consolidated Framework for Implementation Research |
CMS: Centers for Medicare & Medicaid Services |
EHR: electronic health record |
ESAS: Edmonton Symptom Assessment System |
HPP: health care provider and professional |
IT: information technology |
MA: medical assistant |
mHealth: mobile health |
Moffitt: Moffitt Cancer Center |
NCI: National Cancer Institute |
PAR: Patient Access Representative |
PRO: patient-reported outcome |
VHA: Veterans Health Administration |
Edited by T Leung; submitted 14.04.21; peer-reviewed by T Haddad, T Busse; comments to author 05.05.21; revised version received 07.05.21; accepted 15.12.21; published 19.01.22
Copyright©Kea Turner, Margarita Bobonis Babilonia, Cristina Naso, Oliver Nguyen, Brian D Gonzalez, Laura B Oswald, Edmondo Robinson, Jennifer Elston Lafata, Robert J Ferguson, Amir Alishahi Tabriz, Krupal B Patel, Julie Hallanger-Johnson, Nasrin Aldawoodi, Young-Rock Hong, Heather S L Jim, Philippe E Spiess. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.01.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.