This paper is in the following e-collection/theme issue:

Telehealth and Telemonitoring (1676) JMIR Theme Issue: COVID-19 Special Issue (2453) Outbreak and Pandemic Preparedness and Management (1728)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/44242, first published .
Perceptions and Use of Telehealth Among Diverse Communities: Multisite Community-Engaged Mixed Methods Study

Perceptions and Use of Telehealth Among Diverse Communities: Multisite Community-Engaged Mixed Methods Study

Perceptions and Use of Telehealth Among Diverse Communities: Multisite Community-Engaged Mixed Methods Study

Authors of this article:

Amelia Barwise1, 2 Author Orcid Image ;   Todd Huschka3 Author Orcid Image ;   Cecelia Woo4 Author Orcid Image ;   Jason Egginton3 Author Orcid Image ;   Lily Huang5 Author Orcid Image ;   Jay-Sheree Allen6 Author Orcid Image ;   Matthew Johnson7 Author Orcid Image ;   Kathryn Hamm8 Author Orcid Image ;   Wendy Wolfersteig8 Author Orcid Image ;   Sean Phelan3, 9 Author Orcid Image ;   Megan Allyse1, 10 Author Orcid Image
Article Authors Cited by (6) Tweetations (2) Metrics

    Original Paper

    1Program in Biomedical Ethics Research, Mayo Clinic, Rochester, MN, United States

    2Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States

    3Robert D and Patricia E Kern Center for Science of Health Care Delivery, Mayo Clinic, Rochester, MN, United States

    4Davidson College, Davidson, NC, United States

    5Quantitative Health Sciences, Mayo Clinic, Jacksonville, FL, United States

    6Department of Family Medicine, Rochester, MN, United States

    7Department of Medicine, Mayo Clinic, Rochester, MN, United States

    8Office of Evaluation and Partner Contracts, Southwest Interdisciplinary Research Center, Arizona State University, Phoenix, AZ, United States

    9Division of Health Care Delivery Research, Mayo Clinic, Rochester, MN, United States

    10Department of Obstetrics & Gynecology, Mayo Clinic, Rochester, MN, United States

    *these authors contributed equally

    Corresponding Author:

    Amelia Barwise, MBBCh, PhD

    Division of Pulmonary and Critical Care Medicine

    Mayo Clinic

    200 First Street SW

    Rochester, MN, 55905

    United States

    Phone: 1 507 250 1789

    Email: barwise.amelia@mayo.edu


    Background: Telehealth has been increasingly adopted by health care systems since the start of the COVID-19 pandemic. Although telehealth may provide convenience for patients and clinicians, there are several barriers to accessing it and using it effectively to provide high-quality patient care.

    Objective: This study was part of a larger multisite community-engaged study conducted to understand the impact of COVID-19 on diverse communities. The work described here explored the perceptions of and experience with telehealth use among diverse and underserved community members during COVID-19.

    Methods: We used mixed methods across three regions in the United States (Midwest, Arizona, and Florida) from January to November 2021. We promoted our study through social media and community partnerships, disseminating flyers in English and Spanish. We developed a moderator guide and conducted focus groups in English and Spanish, mostly using a videoconferencing platform. Participants were placed in focus groups with others who shared similar demographic attributes and geographic location. Focus groups were audio-recorded and transcribed. We analyzed our qualitative data using the framework analytic approach. We developed our broader survey using validated scales and with input from community and scientific leaders, which was then distributed through social media in both English and Spanish. We included a previously published questionnaire that had been used to assess perceptions about telehealth among patients with HIV. We analyzed our quantitative data using SAS software and standard statistical approaches. We examined the effect of region, age, ethnicity/race, and education on the use and perceptions of telehealth.

    Results: We included data from 47 focus groups. Owing to our mode of dissemination, we were not able to calculate a response rate for the survey. However, we received 3447 English-language and 146 Spanish-language responses. Over 90% of participants had internet access and 94% had used telehealth. Approximately half of all participants agreed or strongly agreed that telehealth would be beneficial in the future because it better fit their schedules and they would not need to travel. However, approximately half of the participants also agreed or strongly agreed they would not be able to express themselves well and could not be examined when using telehealth. Indigenous participants were especially concerned about these issues when compared to other racial groups.

    Conclusions: This work describes findings from a mixed methods community-engaged research study about telehealth, including perceived benefits and concerns. Although participants enjoyed the benefits of telehealth (eg, not having to travel and easier scheduling), they also had concerns (eg, not being able to express themselves well and not having a physical exam) about telehealth. These sentiments were especially notable among the Indigenous population. Our work highlights the importance of fully understanding the impact of these novel health delivery modalities on the patient experience and actual or perceived quality of care received.

    J Med Internet Res 2023;25:e44242

    doi:10.2196/44242

    Keywords

    telehealth; telemedicine; digital health; community-based research; mixed methods research; COVID-19; health care system; community health; health delivery; patient experience; quality of care 


    Background

    Much has been written about the impact of COVID-19 on health care delivery, especially the increased use of telehealth in expanded clinical situations, including primary care, mental health, prenatal care, substance use disorder, orthodontics, urology, inpatient care, emergency triage, and physical therapy [1-7].

    Although various forms of telehealth existed for many years prior to the pandemic, especially in rural contexts, it was relatively underused or restricted to within health care systems [8-11]. However, within the first 3 months of the COVID-19 pandemic, telehealth policy makers loosened or lifted many legal and practical restrictions, and the use of telehealth increased by 766% while outpatient visits dropped by over 80% in some institutions [12-14]. In the early stages of the COVID-19 pandemic, telehealth was primarily adopted to reduce viral transmission. As a mechanism to preserve personal protective equipment, even inpatients who required consultations with subspecialized teams or interactions with multidisciplinary members of the health care team sometimes received their consults via video conference [15,16].

    Telehealth Versus Telemedicine

    The Federal Communications Commission defines telemedicine as medical services provided with the support of telecommunications technologies, such as diagnostic testing or monitoring a patient’s posttreatment progress. Telehealth includes a broader scope of clinical and nonclinical remote health care services—often provided by nurses—such as patient education, help with medication adherence, and troubleshooting health issues [17]. The World Health Organization has defined telehealth as being health care services provided by any health care professional and telemedicine as being services provided by a physician; however, the two terms are often used interchangeably [18].

    Some observers predict that telemedicine and telehealth should be the delivery paradigm of the future because of their potential to equalize access to health care. Others have highlighted the potential of telemedicine and telehealth to exacerbate current inequality and inequity in health and access to health care [19] by raising barriers for those who do not have internet access, are disadvantaged from a socioeconomic perspective, are less digitally literate, or face language barriers [20-24]. Much attention has also been paid to privacy, data security, and connectivity concerns, which can threaten the rollout and sustainability of telehealth [25,26]. Finally, cultural and relational barriers may hamper the effectiveness of telehealth. Some individuals have reported that a remote discussion with a health care provider does not constitute an effective health care interaction [27].

    Substantial literature has been published during the pandemic documenting the increased uptake and use of telehealth, but there has been less research focus on garnering the views of telehealth users [28,29]. Although some studies have examined the concerns of health care professionals, few have captured the experiences and perceptions of the public about these adaptations and the use of telehealth, including concerns [30-35]. Furthermore, studies capturing the views of diverse and underresourced communities about telehealth have been lacking.

    This study is part of a larger community-engaged research (CEnR) study conducted over a 9-month period that used mixed methods to assess the impact of COVID-19 on marginalized communities across diverse geographic regions. CEnR has been defined by the Centers for Disease Control and Prevention as “the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the wellbeing of those people” [36,37]. CEnR can support the translation of scientific discovery into a reduction in heath disparities as it seeks to understand stakeholder and community needs. CEnR can foster benefits such as influencing public health initiatives and improved health outcomes [38,39].

    This work describes differences in perceptions and experiences of telehealth during COVID-19 across vulnerable and underresourced communities, including apprehensions, perceived quality of care received, and potential advantages.


    Study Setting and Design

    We performed a multisite, multiphase, mixed methods study from January to November 2021 in three geographic regions of the United States: the southwest, southeast, and midwest [40]. We used CEnR methods to explore the needs, assets, and challenges of surrounding communities to understand the impact of COVID-19. Details of the methods are described elsewhere [41] but are briefly outlined below.

    This project is part of a broader effort toward exploring several issues about the impact of COVID-19 on historically marginalized communities. We here focus on the content that informs considerations around telehealth. The study employed quantitative and qualitative methods in three phases. Phase 1 involved qualitative key informant interviews, phase 2 involved focus groups (FGs), and phase 3 employed an anonymous online survey [42]. As stated by Kelle et al [43], mixed methods research can support “mutual validation of data” as well as provide a “more coherent and complete” picture than a single research strategy alone.

    Ethical Considerations

    The qualitative phases of the study were approved as minimal risk by the Mayo Clinic Institutional Review Board (21-001802 and 21-002163). The quantitative phase was hosted by an external survey research company. As verified by Research Compliance, Mayo Clinic had no participant contact, was therefore not engaged in human subjects’ research, and this phase was not eligible for Institutional Review Board review.

    All data analysis was conducted by our study team. All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation and with the Helsinki Declaration of 1975, as revised in 2000.

    During FGs, the moderator began the activity by outlining the elements of informed consent that had been shared prior to scheduling with participants and they were asked to confirm consent verbally. Participants received a US $50 financial remuneration for their time. As with most surveys, respondents are assumed to have provided consent if they complete the survey. Those who completed the survey received US $10 digital financial remuneration. All data from the survey and FGs were deidentified and stored on password-protected computers or in locked cabinets to which only the study team had access.

    We here report the phase 2 perspectives of FG participants about their experience with telehealth and the phase 3 survey participant responses about telehealth.

    FGs and Surveys

    FG Recruitment and Data Collection

    The broader multidisciplinary study team developed the moderator guide, which included questions about the impact of COVID-19 on access and use of health care. The moderator’s guide covered the following COVID impact topics: future hopes, personal, medical, worries, vaccines, household, mental health, and community impacts. We disseminated flyers in Spanish and English through social media and community organizations and convened FGs in all three regions. The purpose of the study, risks and benefits, and the option to decline any question and/or withdraw at any time were outlined to participants before scheduling. Participants were placed in FGs with others who represented similar demographic, residential, or social communities. We also collaborated with local Departments of Health who were conducting FGs about the impact of COVID-19 on communities and were able to negotiate data sharing. Most FGs were conducted via online videoconferencing (participants could also join by calling in on a mobile phone without requiring internet access); some were conducted in person. FGs were scheduled for 1 hour. Two FGs were conducted in Spanish with bilingual moderators.

    Survey Instrument and Sampling

    We developed an electronic survey with the input of community and scientific leaders as well as other key stakeholders. The survey questions about telehealth were adapted from questions developed to evaluate the attitudes of patients living with HIV about telehealth in lieu of face-to-face visits [44]. They included questions about internet access, use of telehealth since the start of the pandemic, likelihood of using telehealth in the future, as well as perceived benefits of telehealth (fit schedule, no need to travel) and concerns with telehealth (doctor could not examine me, cannot express myself, information not safe, excess data usage) (see Multimedia Appendix 1 for full item wording of the survey questions). The survey was pilot-tested extensively, and we solicited comments and suggestions from Community Advisory Boards, existing panels of health equity, and community engagement researchers across the institution as well as volunteers from the community networks of those researchers [45]. The survey was also translated into Spanish by a licensed medical translator and tested by bilingual English/Spanish speakers.

    The survey was distributed through existing networks of community partners and organizations using social media and email lists for 8 weeks in the fall of 2021. A professional research firm hosted, accrued, and deidentified the data.

    Analysis

    FG Qualitative Data Analysis

    We digitally recorded the FGs, and they were transcribed by a Health Insurance Portability and Accountability Act–compliant professional firm and deidentified before analysis. Data from FGs conducted by our team and those conducted by others were stored and analyzed in parallel using identical coding structures, but were not combined into one data set. We used the framework analytic approach to code our data [46] in which the verbatim content of FG transcripts is summarized in a matrix. In this matrix, deductive themes from group moderation guides create the columns and individual FGs create the rows. Inductive—emerging—themes may add additional columns. Transcript page numbers are noted with paraphrased content within the relevant cells. Two trained coders collaborated on all phases of analysis, including initial content paraphrasing, indexing to codebook development, and illustrative quote retrieval.

    Survey Data Analysis

    Statistical analyses were performed using SAS Software 9.04 (SAS Institute Inc, Cary, NC). Categorical variables are reported as counts with percentages and were analyzed with the Pearson χ2 test, Fisher exact test, and general linear models with Tukey options to test simultaneous differences between percentage means where appropriate. Continuous variables are reported as medians with IQR and analyzed with the Wilcoxon rank-sum test. A 2-sided P value of less than .05 was considered statistically significant. When looking at the difference between means with the Tukey honest-significant test, we tested if the difference between means is equal to zero. This may occasionally result in a significant difference between two groups even when the confidence intervals for those two groups overlap.


    FG Participants’ Baseline Characteristics

    In total, data were analyzed for 47 FGs conducted during 2021 by our team and colleagues at Arizona State University. FGs were demographically and geographically discrete, and included separate groups who were uninsured and underinsured, those experiencing homelessness, LGBTQI, Latinx women and men, parents of young children, cancer survivors, Black men and women, Indigenous Americans, immigrants and Spanish-speaking adults, veterans, and older Asian Americans (see Multimedia Appendix 2).

    Survey Respondents’ Baseline Characteristics

    The completed survey data set included 3447 English-language responses and 146 Spanish-language responses. Responses are proportionate to the geographic population of the area surveyed. Due to the mode of dissemination, we cannot calculate a response rate (see Table 1 for respondent characteristics). We conducted site-specific analysis as well as analysis across different demographic groups that included all sites. Of note, the subset identifying their primary identity as Asian was too small to analyze and has been excluded from this paper but included in other work.

    Table 1. Survey respondent characteristics (N=3593).
    CharacteristicsRespondents, n (%)
    Location

    		Arizona2296 (63.9)

    		Florida427 (11.9)

    		Midwest870 (24.2)
    Age (years)

    		18-29739 (20.7)

    		30-392068 (57.9)

    		40-49700 (19.6)

    		50-7568 (1.9)
    Sex at birth

    		Female1164 (67.5)

    		Male2420 (32.5)

    		Woman2292 (65.0)

    		Man1193 (33.9)

    		Nonbinary22 (0.6)
    Prefer not to say gender17 (0.5)
    Ethnicity/race

    		Black Hispanic342 (9.5)

    		Black non-Hispanic427 (11.9)

    		Indigenous269 (7.5)

    		White Hispanic707 (19.7)

    		White non-Hispanic1725 (48.0)

    		Othera123 (3.4)
    Education

    		High school diploma or less835 (23.4)

    		Some college or more2733 (76.6)

    aIncludes Asian.

    Internet Access

    Internet access was high across all sites with over 90% of participants reporting access. Access was the highest among southwest participants, with 98% reporting internet access. Black Hispanics, Indigenous, and white non-Hispanics had lower rates of internet access than other racial and ethnic groups such as Black non-Hispanic and white Hispanic. No significant differences were detected across groups defined by age, sex, gender identity, or education (see Multimedia Appendix 3).

    Use of Telehealth to Date

    A total of 3260 participants answered the question about use of telehealth since the start of the pandemic. Across all sites, any use of telehealth during this time was high with 94% (95% CI 93.2%-94.7%) overall using telehealth at some stage. The proportions (95% CIs) for telehealth use by subgroup are presented in Table 2. All three locations showed statistically significant different usage of telehealth. Florida had the lowest rate, with 88% (95% CI 85.9%-90.6%) having used telehealth. In Arizona, 94% (95% CI 92.3%-94.3%) and in the midwest 97% (95% CI 94.9%-98.2%) had used telehealth since the COVID-19 pandemic started. Black Hispanics had the highest use of telehealth at 97.7% (95% CI 95.1%-99.9%), which was significantly greater than the use of white non-Hispanics at 94.4% (95% CI 92.3%-94.5%). Among those aged 30-39 years, 94.8% (95% CI 93.7%-95.8%) used telehealth, similar to the rate of those aged 40-49 years at 94.3% (95% CI 92.5%-96.1%). This was significantly more than that of participants aged 18-29 years (89.6%, 95% CI 87.8%-91.4%) but was not significantly higher than the use rate of those aged 50-75 years (88.2%, 95% CI 82.4%-94.1%). There were no significant differences in use of telehealth based on education, sex, or gender identity.

    Table 2. Use of telehealth to date according to participant characteristics.
    CharacteristicsUse telehealth, % (±95% CI)Tukey groupings (difference between means)aP value
    Region<.001

    		Midwest96.1 (1.6)A

    		Florida88.3 (2.3)B

    		Arizona93.3 (0.9)C
    Age (years)<.001

    		18-2989.6 (1.8)A

    		30-3994.8 (1.1)B

    		40-4994.3 (1.8)B

    		50-7588.2 (5.8)A, B
    Sex at birth.05

    		Female92.4 (1.4)A

    		Male94.0 (1.0)A
    Ethnicity/race<.001

    		Black Hispanic97.7 (2.5)A

    		Black non-Hispanic94.4 (2.3)A, B

    		Indigenous93.3 (2.8)A, B

    		White Hispanic93.5 (1.8)A, B

    		White non-Hispanic93.4 (1.1)B

    aGroups with different letters have a mean difference significantly different than 0 at the P=.05 level. No significant differences were detected across groups defined by sex, gender identity, or education.

    Perceived Benefits of Telehealth

    Survey questions about the perceived benefits of telehealth included those related to telehealth fitting into people’s schedule better and not needing to travel.

    The proportions (95% CIs) for perceived telehealth benefits by subgroup are presented in Table 3. Over half of all respondents (56.1%, 95% CI 54.5%-57.8%) agreed or strongly agreed that telehealth fit their schedule better than in-person visits. Indigenous respondents were significantly more likely to feel this way than any other groups except non-Hispanic white individuals, at 65.8% (95% CI 59.9%-71.7%) agreement. Those with moderate levels of education (59.0%, 95% CI 57.1%-60.8%) were significantly more likely than those with lower levels (48.5%, 95% CI 45.2%-51.9%) to report that telehealth fit their schedule better (see Table 3).

    Those with moderate levels of education were significantly more likely to agree that telehealth would reduce the need to travel than those with lower levels of education (49.0%, 95% CI 47.2%-50.9% vs 38.8%, 95% CI 35.4%-42.2%). Additionally, the Indigenous population was significantly more likely to agree that not traveling was a benefit than any other group, at 64.3% (95% CI 58.4%-70.2%). There was a marginal age difference, with those aged 50-75 years being least likely to agree (33.8%, 95% CI 22.0%-45.7%). This was significantly lower than that for participants aged 40-49 years (50.7%, 47.0%-54.4%), but not for the other age groups (see Table 3).

    Table 3. Perceived benefits of telehealth.
    CharacteristicsFit schedule betterWould not need to travel

    		Agree, % (± 95% CI)Tukey groupings (difference between means)aP valueAgree, % (±95% CI)Tukey groupings (difference between means)aP value
    Region

    		.004

    		.05

    		Midwest57.2 (3.3)A
    		43.1 (3.3)A

    		Florida48.9 (4.7)B
    		48.7 (4.7)A

    		Arizona57.5 (2.0)A
    		47.6 (2.0)A
    Age (years)

    		<.001

    		.01

    		18-2950.2 (3.6)A
    		44.2 (3.6)A, B

    		30-3958.1 (2.1)B
    		46.6 (2.1)A, B

    		40-4959.6 (3.7)B
    		50.7 (3.7)A

    		50-7545.6 (11.8)A, B
    		33.8 (11.8)B
    Ethnicity/race

    		.002

    		<.001

    		Black Hispanic50.6 (5.3)A
    		43.6 (5.3)A

    		Black non-Hispanic55.0 (4.7)A
    		40.5 (4.7)A

    		Indigenous65.8 (5.9)B
    		64.3 (5.9)B

    		White Hispanic53.6 (3.7)A
    		46.7 (3.7)A

    		White non-Hispanic57.0 (2.3)B
    		44.8 (2.3)A
    Education

    		<.001

    		<.001

    		High school or less48.5 (3.4)A
    		38.8 (3.4)A

    		Some college or more59.0 (1.9)B
    		49.0 (1.9)B

    aGroups with different letters have a mean difference significantly different than 0 at the P=.05 level. No significant differences were detected across groups defined by sex or gender identity.

    Perceived Concerns With Telehealth

    Survey questions about the perceived concerns of telehealth included those related to not being examined properly and not being able to express oneself.

    The proportions (95% CIs) for perceived telehealth concerns by subgroup are presented in Table 4. Overall, approximately half of all respondents (48.2%, 95% CI 46.5%- 49.8%) felt that telehealth would hinder the ability of the provider to examine them well. This pattern was consistent across all demographic groups with some notable differences. Those aged 30-39 years and 40-49 years were the most concerned about this issue at 49.4% (95% CI 47.2%-51.5%) and 53.4% (95% CI 49.7%- 57.1%), respectively, which was significantly higher than the rate of agreement for those aged 18-29 years at 43.4% (95% CI 39.8%-47.0%). Those with moderate levels of education were significantly more likely to be concerned about this issue (49.9%, 95% CI 48.1%-51.8%) than those with lower levels of education (38.8%, 95% CI 35.4%-42.2%). Indigenous respondents were statistically the most concerned about lack of sufficient examination through telehealth, with 63.9% (95% CI 58.2%-69.7%) agreeing or strongly agreeing that “the doctor would not be able to examine me well.” In all other racial and ethnic groups, only about half of all respondents agreed or strongly agreed with the statement. No significant differences were detected across groups defined by sex or gender identity (see Table 4).

    Overall, 45.2% (95% CI 43.5%-46.8%) of participants agreed or strongly agreed that they “would not be able to express themselves well.” The participants in the southeast differed from those of other sites, with only 38.4% (95% CI 33.7%-43.1%) concerned about the issue of expressing themselves. Indigenous respondents had more concerns than other demographic groups, with 60.2% (95% CI 54.3%-66.1%) agreeing or strongly agreeing that they “would not be able to express themselves well.” No statistically significant differences between age groups, sex, gender, or education level were noted (see Table 4).

    Table 4. Perceived concerns with telehealth.
    CharacteristicsCould not be examinedCould not express myself

    		Agree, % (±95% CI)Tukey groupings (difference between means)aP valueAgree, % (±95% CI)Tukey groupings (difference between means)aP value
    Region

    		.06

    		.001

    		Midwest49.0 (3.3)A, B
    		49.2 (3.3)A

    		Florida43.6 (4.7)B
    		38.4 (4.7)B

    		Arizona49.7 (2.0)A
    		45.8 (2.0)A
    Age (years)

    		.002

    		.30

    		18-2943.4 (3.6)A
    		43.0 (3.6)A

    		30-3949.4 (2.2)B
    		47.1 (2.1)A

    		40-4953.4 (3.7)B
    		45.3 (3.7)A

    		50-7547.1 (11.9)A, B
    		45.6 (11.8)A
    Ethnicity/race

    		<.001

    		<.001

    		Black Hispanic44.7 (5.3)A
    		40.0 (5.2)A

    		Black non-Hispanic41.9 (4.7)A
    		39.3 (4.7)A

    		Indigenous63.9 (5.9)B
    		60.2 (5.9)B

    		White Hispanic45.8 (3.7)A
    		43.6 (3.7)A

    		White non-Hispanic48.9 (2.3)A
    		46.0 (2.3)A
    Education

    		.02

    		.66

    		High school or less45.3 (3.4)A
    		45.1 (3.4)A

    		Some college or more49.9 (1.9)B
    		46.0 (1.9)A

    aGroups with different letters have a mean difference significantly different than 0 at the P=.05 level. No significant differences were detected across groups defined by sex or gender identity.

    FG Data

    FGs discussed the broad issue of the impact of COVID-19 on medical care. Telehealth was one of several impactful topics within this realm. Some FG participants supported the concept of the convenience of telehealth, reflecting the findings from the survey data. These views were found in different geographic locations in which FGs were conducted. Participants found telehealth easier, convenient, and helpful, as it avoided the need to travel and park at a clinic, facilitated health care discussions, and was less disruptive than going in person. Textbox 1 provides representative quotes from the FG discussions highlighting these benefits.

    Representative quotes from focus group discussions on the benefits of telehealth during the COVID-19 pandemic.

    “Everything I have done is virtual which honestly for my lifestyle is more convenient for me.” [Florida, Black millennials/Black women focus group]

    “Fortunately enough, I’ve had…help and virtual doctors where I can tell them my symptoms …, and then they’ll prescribe me medicine, and I can just pick it up. That’s been really helpful.” [Arizona, LGBTQ+ focus group]

    “…we now have access to telehealth. …and I think that, in the end, it’s been positive for those people who don’t have access to go to a clinic personally.” [Florida, Spanish-speaking populations focus group]

    “We actually prefer virtual, just so then we don’t actually have to get over there and drive and pay for parking and go back.” [Midwest, pregnancy focus group]

    “My doctors are in [a different location] and it makes it easier for us to talk to them virtually.” [Arizona, Gila Bend, Maricopa County focus group]

    “There’s a discretionary process to see should you even be in-person, or can we do this virtually. I had an ear infection a couple of months ago... (they said) ‘we'll put you on with a nurse and you can describe all your symptoms. If it’s actually an ear infection, we’ll put in a prescription for you and we won’t make you come in’.” [Florida, Black millennials/Black women focus group]

    Textbox 1. Representative quotes from focus group discussions on the benefits of telehealth during the COVID-19 pandemic.

    However, as in the survey findings, several FG participants had concerns about telehealth. These views included worries about not being assessed adequately, lack of trust in the sufficiency of telehealth, feeling uncomfortable, and sometimes just wanting to see a clinician in person. These views were found across all geographic locations in which FGs were conducted. Textbox 2 provides representative quotes about these concerns.

    Representative quotes from focus group discussions on the concerns of telehealth during the COVID-19 pandemic.

    “During COVID, everything was via video call. … I did not feel comfortable about it because I wanted to see …the doctor personally.” [Florida, Spanish-speaking populations focus group]

    “It’s awkward. I prefer being touched and having them look at my eyes to see how I am. They can’t do that over the internet.” [Florida, unhoused people focus group]

    “Well, it’s hard to diagnose anybody or see anybody, how they’re really doin’ or whatever when you don’t see them physically. You don’t really see what’s goin’ on.” [Arizona, Guadalupe, Maricopa County focus group]

    “I feel like remote visits to the doctor are fairly worthless. … You’re just telling them your symptoms. They can’t look at you and assess you in person. I feel like I just paid $100 for nothing…” [Arizona, parents with young children focus group]

    “Theoretically, you have everything you need there, but you don’t. You're missing a third dimension.” [Arizona, veterans focus group]

    “You just don’t feel satisfaction (with video calls). We had our annual check-up, for the year, 2020, using telehealth. I know my wife didn’t feel that a thorough evaluation of her health condition with the general practitioner got done. Obviously, I feel somewhat similar too.” [Arizona, Asian American 65+ focus group]

    Textbox 2. Representative quotes from focus group discussions on the concerns of telehealth during the COVID-19 pandemic.

    Furthermore, some FG participants articulated a concern with not being listened to or understood, including themes of minimization of symptoms, potential for medical errors, and lack of high-quality care.

    "It didn’t feel like they were actually hearin’ or seein’ like they would in person. It’s like, “Okay, take an aspirin and call me in the mornin’. See how you feel.” That was my opinion of it, and I didn’t feel I was gettin’ the personal care that I needed from my doctors."
    [Florida, unhoused people FG]
    "you get a lot of different opinions. It’s virtual. It’s not real in person. You get misdiagnosed or told different things from different doctors."
    [Arizona, LGBTQ+ FG]

    Notably, sometimes within the same FG, divergent opinions and perceptions of telehealth were voiced. For example, participants in the LGBTQ+ and the Spanish-speaking FGs expressed both positive and negative perceived views of telehealth. Those who were unhoused articulated negative views about telehealth overall.


    Principal Findings

    This CEnR, mixed methods study used a community survey and FGs to collect data from diverse and underserved populations across three regions of the United States about the use and perceptions of telehealth from January to November 2021 of the COVID-19 pandemic. Although telehealth was widely used by all demographic groups and in all states, perceived benefits and concerns differed. Despite frequently voiced apprehension in the news media about connectivity and its impact on telehealth, over 90% of our participants had internet access.

    What was most striking from the survey data findings was the coexisting perceived benefits and concerns among Indigenous respondents. Indigenous respondents, like other ethnic groups, enjoyed the convenience of telehealth as they did not need to travel and it fit in their schedule; however, they also had greater concerns than other groups about telehealth, including the ability of their clinicians to examine them properly and for them to be able to express themselves to their clinicians. Evidence from studies conducted prepandemic suggested that lack of physical examination has been a consistent concern for patients. Guidelines to help determine the appropriateness of a virtual visit as well as guidance for conducting virtual physical examinations have been developed as a result [47,48].

    TechQuity is a recently developed term that incorporates an antiracism and proequity ethos to the use of technology in health care. While telehealth is a small part of the increasing use of technology within the health care realm, we should be mindful that telehealth does not exacerbate structural racism and inequities and take note of these findings [49]. Other potential barriers to the use of telehealth not assessed in our study relate to digital literacy and comfort with technology, which may be worse among older adults. This group may benefit substantially from avoiding in-person visits owing to their susceptibility to contagion and mobility challenges [50,51]. However, they faced challenges with online scheduling of vaccination appointments and may require assistance to use telehealth, which may not always be available [52,53]. Language barriers and poor health literacy may also be obstacles to the use of telehealth [54,55]. Just as the above factors substantially impacted vaccine uptake due to difficulties in accessing patient portal messages and online scheduling, it is likely that a telehealth visit without sufficient support from a caregiver or family member would not meet patient needs sufficiently and detract from the patient experience [53,56].

    Since the previously ubiquitous reimbursement and licensure hurdles that hampered adoption have been largely overcome during the pandemic, telehealth is appealing to institutions as a mechanism to efficiently deliver some forms of health care [10]. The exponential increase in the use of telehealth during the pandemic has encouraged health care institutions to maintain telehealth as a permanent modality for delivering certain types of care [57-59]. While telehealth has some advantages for convenience as highlighted in our findings, concerns remain and warrant further exploration, particularly as they relate to the patient experience.

    Study Strengths

    We leveraged CEnR methodology as an important tool to understand and address equity issues associated with telehealth in our communities [37,38]. The study utilized a mixed methods approach within a limited time frame across multiple states and sites safely during the pandemic.

    We disseminated a community-wide electronic survey among diverse and underserved populations using our community networks, partnerships, and organizational connections, utilizing social media to reach those not usually surveyed by academic institutions. We did not use a convenience sample of empaneled patients. The survey was translated into Spanish using a licensed medical translator and it was pilot-tested by bilingual English/Spanish speakers. Participants needed only a mobile phone to participate in both the surveys and FGs. We combined our quantitative survey methods with qualitative research approaches using FGs, providing us with a deeper contextual understanding of the data [40]. We had a multidisciplinary study team who had expertise in survey methods, qualitative methods, and mixed methods, ensuring all phases of the research were conducted in a scientifically robust manner [60].

    Another strength of this study is the use of the framework method to analyze the FG data. This approach has several advantages over other qualitative analytic methods. It can be deployed in multidisciplinary research teams where some members have limited qualitative experience but can still engage in sense-making under the guidance of an experienced qualitative researcher. Furthermore, the framework analysis supported a structure for cross-disciplinary analysis of matrices we developed over time as well as a robust audit trail for our study team [46].

    Study Limitations

    As with all surveys, response bias is a potential concern. It is possible that those with strong opinions are more likely to respond. We are unable to calculate a response rate for the survey due to the social media mode of dissemination, and while this was assessed to be the most successful mode for capturing diverse and underserved community voices, people who do not use social media may have been less likely to be aware of the study. Nonusers of social media may also be less familiar with digital technology; thus, these findings may underestimate some of the potential concerns. We conducted most FGs through Zoom technology, which likely supported engagement among most participants and was less burdensome than traveling for an in-person FG, but it is possible that this deterred some from participating or fully engaging. Even with broad-reaching recruitment strategies, we received few survey responses from those identifying as Asian. Our findings elucidate important technological considerations about telehealth, including internet access, but also other less explored worries that patients may have about whether telehealth is as effective as in-person clinic visits.

    It is likely that health care systems will have differing priorities to patients when decisions about telehealth access and utilization are discussed [61-63]. Health care systems that invested time, staff, and other resources into supporting telehealth have been able to overcome barriers such as reimbursement and now hope to promote telehealth in many health care settings as a viable return on investment [63]. As health care institutions roll out and sustain telehealth, considerations need to include not only internet access and connectivity but also ease with use of telehealth based on education, language, technology savvy and health literacy, and the patient experience.

    Conclusion

    This manuscript describes findings from a mixed methods CEnR study about telehealth, including perceived benefits and concerns. As well as being thoughtful about the logistical elements of implementing new health care delivery modalities and processes, our work also highlights the imperative to understand the impact of these novel modalities on the patient experience and actual or perceived quality of care received. Although telehealth is popular among health care systems, overall, patients have concerns that include not being able to express themselves and not having a physical exam.

    Acknowledgments

    We would like to thank the Mayo Clinic Center for Health Equity and Community-Engaged Research (CHCR), which provided funding for this initiative. We would also like to thank Ms. Laura Pacheco-Spann MS, MSM, and Ms. Monica Albertie, MHA, for their logistical support and assistance with all aspects of this study. The focus group data collection in Maricopa County Department of Public Health was funded under a contract with the Arizona Board of Regents, DBA Arizona State University, Arizona State University, Funding Proposal 00026920, Award 00035628; Maricopa County Department of Public Health contract #PHPQ21010 (WW, principal investigator).

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Survey questions related to telehealth.

    DOCX File , 16 KB
    ‎
    Multimedia Appendix 2

    Focus group participant baseline characteristics.

    DOCX File , 13 KB
    ‎
    Multimedia Appendix 3

    Internet access according to demographic characteristics of survey participants.

    DOCX File , 21 KB
    1. Chang JE, Lai AY, Gupta A, Nguyen AM, Berry CA, Shelley DR. Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID era. Milbank Q 2021 Jun;99(2):340-368 [FREE Full text] [CrossRef] [Medline]
    2. Hanlon P, Daines L, Campbell C, McKinstry B, Weller D, Pinnock H. Telehealth interventions to support self-management of long-term conditions: a systematic metareview of diabetes, heart failure, asthma, chronic obstructive pulmonary disease, and cancer. J Med Internet Res 2017 May 17;19(5):e172 [FREE Full text] [CrossRef] [Medline]
    3. Molfenter T, Roget N, Chaple M, Behlman S, Cody O, Hartzler B, et al. Use of telehealth in substance use disorder services during and after COVID-19: online survey study. JMIR Ment Health 2021 Feb 08;8(2):e25835 [FREE Full text] [CrossRef] [Medline]
    4. Wang H, Poehler JL, Ziegler JL, Weiler CC, Khan SA. Patient care rounds in the intensive care unit during COVID-19. Jt Comm J Qual Patient Saf 2020 Oct;46(10):600-601 [FREE Full text] [CrossRef] [Medline]
    5. Zhou X, Snoswell CL, Harding LE, Bambling M, Edirippulige S, Bai X, et al. The role of telehealth in reducing the mental health burden from COVID-19. Telemed J E Health 2020 Apr 01;26(4):377-379. [CrossRef] [Medline]
    6. Fryer K, Delgado A, Foti T, Reid CN, Marshall J. Implementation of obstetric telehealth during COVID-19 and beyond. Matern Child Health J 2020 Sep 20;24(9):1104-1110 [FREE Full text] [CrossRef] [Medline]
    7. Aziz A, Zork N, Aubey JJ, Baptiste CD, D'Alton ME, Emeruwa UN, et al. Telehealth for high-risk pregnancies in the setting of the COVID-19 pandemic. Am J Perinatol 2020 Jun 12;37(8):800-808 [FREE Full text] [CrossRef] [Medline]
    8. Marcin JP, Shaikh U, Steinhorn RH. Addressing health disparities in rural communities using telehealth. Pediatr Res 2016 Jan 14;79(1-2):169-176 [FREE Full text] [CrossRef] [Medline]
    9. Hirko KA, Kerver JM, Ford S, Szafranski C, Beckett J, Kitchen C, et al. Telehealth in response to the COVID-19 pandemic: Implications for rural health disparities. J Am Med Inform Assoc 2020 Nov 01;27(11):1816-1818 [FREE Full text] [CrossRef] [Medline]
    10. Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff 2014 Feb;33(2):207-215. [CrossRef] [Medline]
    11. Gajarawala SN, Pelkowski JN. Telehealth benefits and barriers. J Nurse Pract 2021 Feb;17(2):218-221 [FREE Full text] [CrossRef] [Medline]
    12. Weigel GR, Ramaswamy A, Sobel L, Salganicoff A, Cubanski J, Freed M. Opportunities and barriers for telemedicine in the U.S. during the COVID-19 emergency and beyond. KFF Women's Health Policy. 2020 May 11.   URL: https:/​/www.​kff.org/​womens-health-policy/​issue-brief/​opportunities-and-barriers-for-telemedicine-in-the-u-s-during-the-covid-19-emergency-and-beyond/​ [accessed 2022-09-27]
    13. Shaver J. The state of telehealth before and after the COVID-19 pandemic. Prim Care 2022 Dec;49(4):517-530 [FREE Full text] [CrossRef] [Medline]
    14. Xu S, Glenn S, Sy L, Qian L, Hong V, Ryan DS, et al. Impact of the COVID-19 pandemic on health care utilization in a large integrated health care system: retrospective cohort study. J Med Internet Res 2021 Apr 29;23(4):e26558 [FREE Full text] [CrossRef] [Medline]
    15. Barwise A, Tschida-Reuter D, Sutor B. Adaptations to interpreter services for hospitalized patients during the COVID-19 pandemic. Mayo Clin Proc 2021 Dec;96(12):3184-3185 [FREE Full text] [CrossRef] [Medline]
    16. Humphreys J, Schoenherr L, Elia G, Saks NT, Brown C, Barbour S, et al. Rapid implementation of inpatient telepalliative medicine consultations during COVID-19 pandemic. J Pain Symptom Manage 2020 Jul;60(1):e54-e59 [FREE Full text] [CrossRef] [Medline]
    17. Nelson R. Telemedicine and telehealth: the potential to improve rural access to care. Am J Nurs 2017;117(6):17-18. [CrossRef]
    18. World Health Organization. Telemedicine: opportunities and developments in member states: report on the Second Global Survey on eHealth 2009 (Global Observatory for eHealth Series, Volume 2). Geneva: World Health Organization; 2012.
    19. Nouri S, Khoong EC, Lyles CR, Karliner L. Addressing equity in telemedicine for chronic disease management during the Covid-19 pandemic. NEJM Catalyst Innov Care Deliv 2020 May 04;1(3):1-13 [FREE Full text]
    20. Cherewka A. The digital divide hits US immigrant households disproportionately during the COVID-19 pandemic. Migration Policy Institute. 2020 Sep 03.   URL: https://www.migrationpolicy.org/article/digital-divide-hits-us-immigrant-households-during-covid-19 [accessed 2022-09-27]
    21. Khilnani A, Schulz J, Robinson L. The COVID-19 pandemic: new concerns and connections between eHealth and digital inequalities. J Inf Commun Ethics Soc 2020 Jun 25;18(3):393-403. [CrossRef]
    22. Rodriguez JA, Betancourt JR, Sequist TD, Ganguli I. Differences in the use of telephone and video telemedicine visits during the COVID-19 pandemic. Am J Manag Care 2021 Jan 14;27(1):21-26 [FREE Full text] [CrossRef] [Medline]
    23. Hsueh L, Huang J, Millman AK, Gopalan A, Parikh RK, Teran S, et al. Disparities in use of video telemedicine among patients with limited English proficiency during the COVID-19 pandemic. JAMA Netw Open 2021 Nov 01;4(11):e2133129 [FREE Full text] [CrossRef] [Medline]
    24. Tan-McGrory A, Schwamm LH, Kirwan C, Betancourt JR, Barreto EA. Addressing virtual care disparities for patients with limited English proficiency. Am J Manag Care 2022 Jan 14;28(1):36-40 [FREE Full text] [CrossRef] [Medline]
    25. Hall JL, McGraw D. For telehealth to succeed, privacy and security risks must be identified and addressed. Health Aff 2014 Feb;33(2):216-221. [CrossRef] [Medline]
    26. Jalali MS, Landman A, Gordon WJ. Telemedicine, privacy, and information security in the age of COVID-19. J Am Med Inform Assoc 2021 Mar 01;28(3):671-672 [FREE Full text] [CrossRef] [Medline]
    27. Gordon HS, Solanki P, Bokhour BG, Gopal RK. "I'm not feeling like I'm part of the conversation" Patients' perspectives on communicating in clinical video telehealth visits. J Gen Intern Med 2020 Jun 03;35(6):1751-1758 [FREE Full text] [CrossRef] [Medline]
    28. Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health 2020 Aug 01;20(1):1193 [FREE Full text] [CrossRef] [Medline]
    29. Fisk M, Livingstone A, Pit SW. Telehealth in the context of COVID-19: changing perspectives in Australia, the United Kingdom, and the United States. J Med Internet Res 2020 Jun 09;22(6):e19264 [FREE Full text] [CrossRef] [Medline]
    30. Marcus J, MacMillan D, Ketchie M, Clark PC. Nurses' perceptions of technology used in language interpretation for patients with limited English proficiency. Am J Nurs 2020 Sep;120(9):48-58. [CrossRef] [Medline]
    31. Goldberg EM, Jiménez FN, Chen K, Davoodi NM, Li M, Strauss DH, et al. Telehealth was beneficial during COVID-19 for older Americans: a qualitative study with physicians. J Am Geriatr Soc 2021 Nov 20;69(11):3034-3043 [FREE Full text] [CrossRef] [Medline]
    32. Goldberg EM, Lin MP, Burke LG, Jiménez FN, Davoodi NM, Merchant RC. Perspectives on telehealth for older adults during the COVID-19 pandemic using the quadruple aim: interviews with 48 physicians. BMC Geriatr 2022 Mar 08;22(1):188 [FREE Full text] [CrossRef] [Medline]
    33. Bashir A, Bastola DR. Perspectives of nurses toward telehealth efficacy and quality of health care: pilot study. JMIR Med Inform 2018 May 25;6(2):e35 [FREE Full text] [CrossRef] [Medline]
    34. Venville A, O'Connor S, Roeschlein H, Ennals P, McLoughlan G, Thomas N. Mental health service user and worker experiences of psychosocial support via telehealth through the COVID-19 pandemic: qualitative study. JMIR Ment Health 2021 Aug 12;8(8):e29671 [FREE Full text] [CrossRef] [Medline]
    35. Andrews E, Berghofer K, Long J, Prescott A, Caboral-Stevens M. Satisfaction with the use of telehealth during COVID-19: an integrative review. Int J Nurs Stud Adv 2020 Nov;2:100008 [FREE Full text] [CrossRef] [Medline]
    36. CDC/ATSDR Committee on Community Engagement. Principles of community engagement. Atlanta, GA: Centers for Disease Control and Prevention; 1997.
    37. Cook J, Michener JL, Lyn M, Lobach D, Johnson F. Practice profile. Community collaboration to improve care and reduce health disparities. Health Aff 2010 May;29(5):956-958. [CrossRef] [Medline]
    38. Showstack R, Duque S, Woods NK, López A, Chesser A. Lifting the voices of Spanish-speaking Kansans: a community-engaged approach to health equity. Multilingua 2021;41(4):489-517. [CrossRef]
    39. Kirk CM, Johnson-Hakim S, Anglin A, Connelly C. Putting the community back into community health needs assessments: maximizing partnerships via community-based participatory research. Prog Community Health Partnersh 2017;11(2):167-173. [CrossRef] [Medline]
    40. Creswell JW, Klassen AC, Plano Clark VL, Clegg Smith K, Office of Behavioral and Social Sciences Research (OBSSR). Best practices for mixed methods research in the health sciences. Bethesda, MD: National Institutes of Health; 2011:541-545.
    41. Barwise AK, Egginton J, Pacheco-Spann L, Clift K, Albertie M, Johnson M, et al. Community engaged research to measure the impact of COVID-19 on vulnerable community member's well-being and health: a mixed methods approach. Wien Klin Wochenschr 2022 Dec 05;1-7:online ahead of print [FREE Full text] [CrossRef] [Medline]
    42. Huang L, Cleveland T, Clift K, Egginton JS, Pacheco-Spann L, Johnson MG, et al. Key stakeholder perspectives of community engagement efforts and the impact of the Covid-19 pandemic. J Prim Care Community Health 2022 Nov 04;13:21501319221133076 [FREE Full text] [CrossRef] [Medline]
    43. Kelle U. Combining qualitative and quantitative methods in research practice: purposes and advantages. Qual Res Psych 2006;3(4):293-311. [CrossRef]
    44. Dandachi D, Dang BN, Lucari B, Teti M, Giordano TP. Exploring the attitude of patients with HIV about using telehealth for HIV care. AIDS Patient Care STDS 2020 Apr 01;34(4):166-172. [CrossRef] [Medline]
    45. Formea CM, Mohamed AA, Hassan A, Osman A, Weis JA, Sia IG, et al. Lessons learned: cultural and linguistic enhancement of surveys through community-based participatory research. Prog Community Health Partnersh 2014;8(3):331-336 [FREE Full text] [CrossRef] [Medline]
    46. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013 Sep 18;13(1):117 [FREE Full text] [CrossRef] [Medline]
    47. Benziger CP, Huffman MD, Sweis RN, Stone NJ. The telehealth ten: a guide for a patient-assisted virtual physical examination. Am J Med 2021 Jan;134(1):48-51 [FREE Full text] [CrossRef] [Medline]
    48. Reeves JJ, Ayers JW, Longhurst CA. Telehealth in the COVID-19 era: a balancing act to avoid harm. J Med Internet Res 2021 Feb 01;23(2):e24785 [FREE Full text] [CrossRef] [Medline]
    49. Rhee K, Dankwa-Mullan I, Brennan V, Clark C. What is TechQuity? J Health Care Poor Underserved 2021;32(2S):xiii-xviii. [CrossRef]
    50. Vergara J, Parish A, Smallheer B. Telehealth: opportunities in geriatric patient care during COVID-19. Geriatr Nurs 2020 Sep;41(5):657-658 [FREE Full text] [CrossRef] [Medline]
    51. Kruse C, Fohn J, Wilson N, Nunez Patlan E, Zipp S, Mileski M. Utilization barriers and medical outcomes commensurate with the use of telehealth among older adults: systematic review. JMIR Med Inform 2020 Aug 12;8(8):e20359 [FREE Full text] [CrossRef] [Medline]
    52. Kalicki AV, Moody KA, Franzosa E, Gliatto PM, Ornstein KA. Barriers to telehealth access among homebound older adults. J Am Geriatr Soc 2021 Sep 13;69(9):2404-2411 [FREE Full text] [CrossRef] [Medline]
    53. Prescott GM, Prescott WA. Health information technology utilization and impact on COVID-19 vaccination. J Am Pharm Assoc 2021 Jul;61(4):e230-e232 [FREE Full text] [CrossRef] [Medline]
    54. Rodriguez JA, Saadi A, Schwamm LH, Bates DW, Samal L. Disparities In telehealth use among California patients with limited English proficiency. Health Aff 2021 Mar 01;40(3):487-495. [CrossRef] [Medline]
    55. Yin L, Ng F, Rutherford-Rojas M, Williams M, Cornes S, Fernandez A, et al. Assessing medical student readiness to navigate language barriers in telehealth: cross-sectional survey study. JMIR Med Educ 2022 Aug 11;8(3):e36096 [FREE Full text] [CrossRef] [Medline]
    56. Fuchs JR, Fuchs JW, Tietz SE, Lum HD. Older adults with limited English proficiency need equitable COVID-19 vaccine access. J Am Geriatr Soc 2021 Apr 24;69(4):888-891 [FREE Full text] [CrossRef] [Medline]
    57. Bakken S. Telehealth: simply a pandemic response or here to stay? J Am Med Inform Assoc 2020 Jul 01;27(7):989-990 [FREE Full text] [CrossRef] [Medline]
    58. Gardenier D, Ballard-Hernandez J, O'Donnell M. Is telehealth here to stay? J Nurse Pract 2021 Mar;17(3):265-266 [FREE Full text] [CrossRef] [Medline]
    59. Wamsley CE, Kramer A, Kenkel JM, Amirlak B. Trends and challenges of telehealth in an academic institution: the unforeseen benefits of the COVID-19 global pandemic. Aesthet Surg J 2021 Jan 01;41(1):109-118 [FREE Full text] [CrossRef] [Medline]
    60. Eckhardt AL, DeVon HA. The MIXED framework: a novel approach to evaluating mixed-methods rigor. Nurs Inq 2017 Oct 09;24(4):e12189. [CrossRef] [Medline]
    61. Botrugno C. Towards an ethics for telehealth. Nurs Ethics 2019 Mar 15;26(2):357-367. [CrossRef] [Medline]
    62. Kaplan B, Litewka S. Ethical challenges of telemedicine and telehealth. Camb Q Healthc Ethics 2008 Oct 01;17(4):401-416. [CrossRef]
    63. Snoswell CL, North JB, Caffery LJ. Economic advantages of telehealth and virtual health practitioners: return on investment analysis. JMIR Perioper Med 2020 May 21;3(1):e15688 [FREE Full text] [CrossRef] [Medline]

    ‎
    CEnR: community-engaged research
    FG: focus group

    Edited by A Mavragani; submitted 21.11.22; peer-reviewed by B Wolfe-Piggott, D Valdes; comments to author 10.01.23; revised version received 31.01.23; accepted 31.01.23; published 28.03.23

    Copyright

    ©Amelia Barwise, Todd Huschka, Cecelia Woo, Jason Egginton, Lily Huang, Jay-Sheree Allen, Matthew Johnson, Kathryn Hamm, Wendy Wolfersteig, Sean Phelan, Megan Allyse. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.03.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.