This paper is in the following e-collection/theme issue:

Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (782) Participatory Medicine & E-Patients (758) Information Seeking, Information Needs (419) Participatory Research Protocols and Proposals (127) Medicine 2.0: Social Media, Open, Participatory, Collaborative Medicine (1552) Peer-to-Peer Support and Online Communities (592)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48550, first published .
Online Forums as a Tool for Broader Inclusion of Voices on Health Care Communication Experiences and Serious Illness Care: Mixed Methods Study

Online Forums as a Tool for Broader Inclusion of Voices on Health Care Communication Experiences and Serious Illness Care: Mixed Methods Study

Online Forums as a Tool for Broader Inclusion of Voices on Health Care Communication Experiences and Serious Illness Care: Mixed Methods Study

Authors of this article:

Carine Davila1, 2 Author Orcid Image ;   Stephanie H Chan3, 4 Author Orcid Image ;   Anna Gosline3, 4 Author Orcid Image ;   Zamawa Arenas5 Author Orcid Image ;   Jane Kavanagh3, 6 Author Orcid Image ;   Brian Feltz5, 7 Author Orcid Image ;   Elizabeth McCarthy5, 8 Author Orcid Image ;   Tyrone Pitts9 Author Orcid Image ;   Christine Ritchie1, 2, 10 Author Orcid Image
Article Authors Cited by Tweetations (4) Metrics

    Original Paper

    1Division of Palliative Care and Geriatric Medicine, Massachusetts General Hospital, Boston, MA, United States

    2Department of Medicine, Harvard Medical School, Boston, MA, United States

    3Massachusetts Coalition for Serious Illness Care, Boston, MA, United States

    4Blue Cross Blue Shield of Massachusetts, Boston, MA, United States

    5Flowetik, Boston, MA, United States

    6Ariadne Labs, Boston, MA, United States

    73D Research Partners LLC, Harvard, MA, United States

    8Elizabeth M McCarthy Consulting, Boston, MA, United States

    9The Coalition to Transform Advanced Care, Washington, DC, United States

    10Center for Aging in Serious Illness, Mongan Institute, Boston, MA, United States

    Corresponding Author:

    Carine Davila, MPH, MD

    Division of Palliative Care and Geriatric Medicine

    Massachusetts General Hospital

    55 Fruit St

    Austen 600

    Boston, MA, 02114

    United States

    Phone: 1 617 724 9197

    Email: cdavila@mgh.harvard.edu


    Background: Existing health care research, including serious illness research, often underrepresents individuals from historically marginalized communities. Capturing the nuanced perspectives of individuals around their health care communication experiences is difficult. New research strategies are needed that increase engagement of individuals from diverse backgrounds.

    Objective: The aim of this study was to develop a mixed methods approach with qualitative online forums to better understand health communication experiences of individuals, including people from groups historically marginalized such as Black and Latino individuals; older adults; and people with low income, disability, or serious illness.

    Methods: We used a multiphase mixed methods, community-informed research approach to design study instruments and engage participants. We engaged a diverse group of collaborators with lived experience of navigating the health care system who provided feedback on instruments, added concepts for testing, and offered guidance on creating a safe experience for participants (phase 1). We conducted a national quantitative survey between April and May 2021 across intrapersonal, interpersonal, and systems-level domains, with particular focus on interpersonal communication between patients and clinicians (phase 2). We conducted two asynchronous, qualitative online forums, a technique used in market research, between June and August 2021, which allowed us to contextualize the learnings and test concepts and messages (phase 3). Using online forums allowed us to probe more deeply into results and hypotheses from the survey to better understand the “whys” and “whats” that surfaced and to test public messages to encourage action around health.

    Results: We engaged 46 community partners, including patients and clinicians from a Federally Qualified Health Center, to inform study instrument design. In the quantitative survey, 1854 adults responded, including 50.5% women, 25.2% individuals over 65 years old, and 51.9% individuals with low income. Nearly two-thirds identified as non-Hispanic white (65.7%), 10.4% identified as non-Hispanic Black, and 15.5% identified as Hispanic/Latino. An additional 580 individuals participated in online forums, including 60.7% women, 17.4% individuals over 65 years old, and 49.0% individuals with low income. Among the participants, 70.3% identified as non-Hispanic white, 16.0% as non-Hispanic Black, and 9.5% as Hispanic/Latino. We received rich, diverse input from our online forum participants, and they highlighted satisfaction and increased knowledge with engagement in the forums.

    Conclusions: We achieved modest overrepresentation of people who were over 65 years old, identified as non-Hispanic Black, and had low income in our online forums. The size of the online forums (N=580) reflected the voices of 93 Black and 55 Hispanic/Latino participants. Individuals who identify as Hispanic/Latino remained underrepresented, likely because the online forums were offered only in English. Overall, our findings demonstrate the feasibility of using the online forum qualitative approach in a mixed methods study to contextualize, clarify, and expound on quantitative findings when designing public health and clinical communications interventions.

    J Med Internet Res 2023;25:e48550

    doi:10.2196/48550

    Keywords

    serious illness care; serious illness communication; mixed methods research; community-engaged design; equity in research; online forum; health care experiences; internet; illness; marginalized community; efficacy; communication; engagement; quantitative survey; health care 


    The COVID-19 pandemic raised the US health care community’s acknowledgement of both the historic and current inequities in health care access, treatment, and outcomes. The pandemic therefore highlighted the need for increased engagement of diverse communities to increase the validity of research studies [1-3]. This is true for all of health care, and serious illness research is no exception [3-7]. Serious illness communication (SIC) describes conversations that occur between patients with serious illness and clinicians to understand the patient’s goals, values, preferences, and priorities so that health care can be aligned with those priorities [8]. SIC is a type of shared decision-making and part of the broad set of activities known as advance care planning (ACP). SIC represents an important tool for the creation of therapeutic alliance and has the potential to align goals and clinical decisions. Such conversations are enhanced when clinicians within care systems have the cultural skills, attitudes, behaviors, and interactional styles to promote effective SIC [9,10]. However, there are documented disparities by race, ethnicity, and income in SIC and ACP-related activities, including health care proxy completion rates, conversations with clinicians and family about wishes for care, and the kinds of language that worked best when it comes to encouraging these activities [11-18].

    Earlier work from the Massachusetts Coalition for Serious Illness Care (MCSIC) focused on how to best promote SIC and ACP to the public, especially to historically marginalized communities most likely to experience poor health outcomes [19]. Some of the open-ended qualitative research found confusion and misunderstanding about the language used to describe the many activities that are collectively referred to as ACP. It also became clear that many individuals associate the entire field of SIC and ACP with the very end of life and death. No matter how items were phrased, people continuously assumed that the topic was related specifically to do-not-resuscitate orders, “pull the plug” decisions, and what has come to be referred to as “true” end-of-life planning, such as estate or funeral planning [20]. Our goal for this research was to better contextualize people’s beliefs and attitudes about serious illness, SIC, and ACP within the larger canvas of their overall health care experiences. Unlike our prior work, we were not aiming to directly encourage SIC or ACP, but instead to further understand how SIC and ACP might align with the challenges and needs as seen by patients and to establish an approach to generate new insights into how and when ACP and SIC should be introduced and encouraged. Toward this end, we sought new approaches to obtain broader and more nuanced input from historically marginalized communities and ask new questions that solicited a more holistic focus on the health care journey, rather than exclusively on the serious illness journey or end of life journey. Accordingly, our specific question was as follows: How do we engage a wide range of insights on these issues to ensure that the perspectives of a small number of individuals are not extrapolated to reflect entire communities?

    Our research had four key aims. The first aim was to understand individuals’ experiences with the health care system that shape care expectations and attitudes toward the system, specifically with regard to medical decision-making, recognition of and support for social determinants of health, and trust in and respect by clinicians and health care systems. Second, we sought to understand the greatest perceived medical, social, and financial needs when it comes to improving serious illness care in the United States. Third, we wanted to understand how individuals perceive sample language that clinicians may use to engage and support individuals under their care and understand what authentically resonates with them. Finally, we wanted to obtain input on how best to contextualize and frame different public messages to encourage action around health, including SIC and ACP. We sought to explicitly understand these perspectives of individuals from historically marginalized communities, who are often underrepresented in research.

    Here, we describe our overarching approach to optimize representation in the research, demonstrate the breadth of engagement in our quantitative survey and qualitative online forums, and highlight participants’ satisfaction with the online forum engagement tool.


    Overview

    We designed a mixed methods community-informed research study. The study took place in three phases from August 2020 to August 2021 (see Figure 1). Guided by our aims, we sought to understand: What are people’s lived experiences in health care settings? What are the challenges faced by people with serious illness and caregivers? How does this impact what we should prioritize saying, doing, and asking people to do?

    ‎
    Figure 1. Research timeline spanning three phases in the mixed methods study, August 2020-August 2021. NORC: National Opinion Research Center.

    Ethical Approval

    This study received approval through the Harvard Longwood Campus Institutional Review Board (phase 2, IRB21-0398) and Massachusetts General Brigham’s Institutional Review Board (phase 3, 2021P003549).

    Phase 1: Engage Community Partners in Study Design

    Our goal was to gather input from community partners to ensure that our research objectives, approach, and framing were aligned with the needs of people from historically marginalized communities. We wanted to understand what matters to people around serious illness care, trying to discard our biases and assumptions. We engaged people through various channels, including focus groups with clinicians at a Federally Qualified Health Center (FQHC), in-depth telephone interviews with Black and Latino low-income older adults who participate in a Program for All-Inclusive Care for the Elderly at the FQHC, and an online survey with input from leading serious illness care organizations across the country. All participants were compensated for their time and engagement. We asked FQHC participants what situations and circumstances impact the type of health care Black and Latino patients receive, what they are most concerned about regarding their health and health care, and what they thought needed to change to improve people’s health care experiences. We asked clinicians and serious illness care leaders what and how to ask about people’s health care experiences, including SIC and ACP, and what needs to change to improve these health care experiences. These insights informed the development of study instruments for both our quantitative survey (phase 2) and qualitative online forum guide (phase 3). By the end of phase 1, we had a small diverse group of collaborators with lived experience of the challenges people face navigating the health care system, who provided more detailed and focused feedback on each instrument, added new concepts for testing, and offered guidance on creating a safe and caring experience for our participants. This group included additional MCSIC members and other national leaders in serious illness care, representing a diverse group of voices as listed in the Acknowledgments.

    Phase 2: Quantitative Survey

    Participant Recruitment

    Survey respondents were included and invited to participate via the National Opinion Research Center (NORC) at the University of Chicago through their national AmeriSpeak panel. The AmeriSpeak panel is a probability-based household panel that uses a multistaged probability-based sampling method through which NORC achieved an estimated sample frame coverage of 97% of the residential United States, including a supplemental list of rural households not recorded on the US Postal Service Computerized Delivery Sequence file but identified through NORC in-person fieldwork. Households are sampled with a known, nonzero probability of selection from the NORC National Frame and recruited through a rigorous process that uses mail, telephone, and in-person recruitment by field interviewers to ensure that even hard-to-reach populations are represented in the panel [21-23]. Enrollment targets were set at a minimum of 100 respondents for specific groups: people with low income (less than US $50,000/year), Black respondents with low income, Latino respondents with low income, age greater than 65 years, people with disability (self-identified and/or answering “yes” to any of six questions about function from the American Community Survey, hereafter described as ACS-6 [24,25]), and people with serious illness (as per the “Identifying People With Serious Illness” subsection below). No specific exclusion criteria were designated. In addition to the national sample described here, a nonprobability-based sample of Massachusetts-based residents was administered through Lucid for separate state-specific analyses [26].

    Quantitative Survey Development

    The design of the quantitative survey was informed by extensive review of the literature, engagement with community partners in phase 1, and the social ecological model [27]. The survey instrument covered topics across intrapersonal, interpersonal, and systems-level domains (Figure 2). We particularly focused on the interpersonal domain between patients and clinicians, covering topics such as whether patients feel engaged in shared decision-making, trust clinicians to do what is right, feel treated with dignity and respect, feel afraid to speak up and ask questions, or feel talked down to or made to feel inferior. Complementing these domains, we iteratively engaged our ongoing collaborators on question topics and language, including how we identified people with serious illness (see below).

    ‎
    Figure 2. Quantitative survey domains, based on the social ecological model; phase 2 of the mixed methods study.

    NORC conducted eight cognitive interviews from AmeriSpeak panelists by video to qualitatively understand how the survey questions were interpreted and make recommendations on alternative word choices. These interviews yielded meaningful recommendations to simplify language, add in clarifiers, provide additional answer choices that respondents thought were missing, and add appropriate prompts to facilitate the flow of the survey. The full quantitative survey instrument is included in Multimedia Appendix 1.

    Identifying People With Serious Illness

    We iteratively engaged with our research and community partners and determined a need to distinguish people with serious illness from people with chronic disability. We identified people with serious illness based on whether participants self-reported “yes” to two questions: (1) have you ever been diagnosed with any of the following (diabetes; asthma, lung disease, emphysema, or chronic obstructive pulmonary disease; heart disease or had a stroke; cancer; Alzheimer disease, dementia, or memory loss; depression, anxiety, or other serious mental health problems; or chronic kidney disease or kidney failure)?, and (2) over the last 12 months, would you say that you have been feeling sicker and that it’s been getting harder to do your normal levels of work and activity?

    Survey Fielding

    NORC AmeriSpeak panelists were invited to participate in the survey between April 20, 2021, and May 17, 2021. The survey was offered in English and Spanish and via both the web and telephone to adults aged 18 years and older. Web-mode panelists were sent up to five email reminders to encourage participation and phone-mode panelists were called throughout the field period. Panelists were offered the cash equivalent of US $3 for completing the survey.

    Phase 3: Qualitative Online Forums

    Description

    To supplement a traditional quantitative survey, we elected to use a series of asynchronous, qualitative online forums, a technique commonly used in market research to gain deeper understanding of why individuals think, believe, and feel how they do. We engaged participants who were part of online forums to delve deeper into understanding people’s health care experiences, testing sample public messages with participants, and framing different public messages that encourage action around SIC and ACP in the context of people’s lived experiences (see Figure 3). Online forums allow engagement of a wide variety of individuals with sample sizes large enough to ensure varied perspectives even within groups that are historically underrepresented and marginalized.

    ‎
    Figure 3. Online forum sample message to encourage action; phase 3 of the mixed methods study.
    Participant Recruitment

    Participants were recruited by Full Circle Research Co, an independent online participant sample provider [28]. Individuals participating with the sample provider completed a screening questionnaire including demographics to determine eligibility. Inclusion criteria included availability to participate during scheduled online forum dates, aged 18 years and older, and having convenient access to a computer or smartphone with a high-speed or broadband internet connection. Exclusion criteria included lack of appropriate technology (such as convenient access to a device and the internet and an up-to-date internet browser such as Microsoft Internet Explorer Version 7.0 or higher, Mozilla Firefox Version 3.0 or higher, Safari Version 2.0.4 or higher, or any version of Google Chrome), inability to participate in the online forum in English (requiring ability to speak English and ability to type free-text responses), and less than high school education.

    Participants were recruited to represent a mix of individuals across age, gender/gender identity, geography/state, education, marital status, employment status, race/ethnicity, income, and self-described health status. Individuals from the following groups were intentionally oversampled: people with low income (less than US $50,000/year), Black respondents with low income, Latino respondents with low income, age greater than 65 years, people with disability, people living with disability (as per phase 2 and ACS-6 [24,25]), people with serious illness (as per the identification criteria outlined in phase 2), and caregiver (defined as helping someone close to them who has a lot of medical or health needs or conditions). We sought to oversample these groups to ensure diversity of perspectives informing our findings.

    Online Forum Activity Guide Design

    Structured activity guides (see Multimedia Appendix 2 and Multimedia Appendix 3) were designed to engage participants in the online forum. Activity guide design was deeply informed by input from community partner engagement in phase 1 and coincided with receiving preliminary results from the phase 2 quantitative survey. Therefore, some activity prompts were created to probe deeper into specific questions, results, or hypotheses generated from the quantitative survey, allowing us to better understand the “whys” in addition to the “whats” that surfaced in the quantitative survey. Each online forum began with “getting to know you” activities to quickly establish trust and inspire participants to see their responses as more than answering research questions. Most activities were open-ended conversation prompts. To encourage continued engagement and reduce response fatigue, other activity formats were used, including answering multiple-choice questions; asking participants to tell us why they chose certain responses; reacting to definitions of quality care; rating the perceived impact of specific language clinicians may use to engage individuals in health care decision-making; and reading through sample public messages on actions one can take to address health and well-being with a highlighter tool to indicate words and phrases that stood out as being positive, negative, or neutral.

    Online Forum Fielding

    Qualified individuals were invited to participate in the online forums via itracksBoard, an online qualitative research platform [29]. Each week’s activities were designed to take around 2 hours to complete over the course of a 5-day week, with new activities available twice per day for a total of 7 or 8 activities per week (see Figure 4 for a sample activity). Participants responded to prompts asynchronously. A moderator (EM) published these activity prompts and then was present in the forum by asking probing, follow-up questions to the participants to encourage more conversation.

    ‎
    Figure 4. Sample activity on the itracksBoard online forum platform; phase 3 of the mixed methods study.

    The online forums were administered in two waves. Wave 1 was conducted for 2 weeks from June 21, 2021, to July 1, 2021, targeting 250 participants in two communities (targeting no more than 125-150 individuals in each community). The primary objective of the first wave was to explore consumers’ health care experiences, both good and bad; the drivers that contributed to these experiences; and reactions to sample messaging clinicians could use to engage patients in their care. Wave 2 was 1-week long from August 9, 2021, to August 13, 2021, targeting 250 participants with three communities (to allow testing with three different messaging frames). The second wave explored how well the participants feel known by their clinicians, their concerns about future care if they became ill, and their reactions to messaging encouraging them to act as one of three options: as a public health campaign, a simple doctor’s letter, and a doctor’s office letter with additional framing language. This sample messaging was developed and informed by the quantitative survey findings. The forums were purposefully smaller in the second wave to test different messages with distinct populations. Participants who completed all activities in wave 1 received a US $125 gift card and those that completed the activities in wave 2 received a US $100 gift card.

    Analysis

    Phase 2: Quantitative Survey

    To account for differences in nonresponse, NORC applied statistical weighting to adjust to Current Population Survey totals associated with age, gender, education, race/Hispanic ethnicity, housing tenure, telephone status, and census division. Additional sampling weights were applied to account for the interactions of age and gender, age and race/ethnicity, and race/ethnicity and gender. The weighted data, which reflects the US population of adults aged over 18 years, were used in subsequent analyses (not shown here). Descriptive statistics were calculated using SPSS version 29 (IBM Corp, Armonk, NY, USA).

    Phase 3: Qualitative Online Forums

    Two authors (EM and ZA) conducted qualitative content analysis of online forum data [30]. We conducted the qualitative analysis in a hybrid inductive-deductive manner based on a combination of the domains from the activity guides and the information that emerged from the online responses. This involved separately combing through daily participant responses verbatim, which comprised responses to research questions intentionally fielded from the activity guide and the organic conversation participants engaged in, all within the “walls” of the online forum, and identifying observations deemed consistent, insightful, and worthy of more exploration. We then compared high-level observations and grouped these into key themes that addressed the overall research questions. We created summaries of participants’ responses by activity, identifying important points, and highlighting key themes for review.


    Phase 1: Engage Community Partners

    We engaged with 46 community partners through various channels. We conducted two focus groups with seven clinicians and care navigators from an FQHC community health center, six in-depth telephone interviews with Black and Latino low-income older adults who receive care at an FQHC, and 12 respondents to an online survey with representatives from leading serious illness care organizations across the country. We also received additional input from 21 MCSIC members and other national leaders in serious illness care.

    Our conversations from care providers and people with lived serious illness experience yielded many common experiences and concerns with health care systems that impact how historically marginalized groups experience serious illness care (see Textbox 1). People shared challenges in navigating the health care system; the unaffordability of care; experiences with bias and discrimination; the geographic inaccessibility of care; and the limited ability clinicians have to address more pressing social challenges impacting their health, including housing, food insecurity, immigration status, and loss of income. These findings led us to focus our study instruments on people’s prior health care experiences and how their current care fits into other life priorities.

    Textbox 1. Key themes from community partners (N=46), comprising US adults engaged from February to July 2021, in phase 1 of the mixed methods study.
    • Difficulty navigating the health care system, such as managing referrals to multiple specialists, accessing home care, and getting prescriptions refilled.
    • Unaffordability of care (including the cost of health insurance and medications) as having a real impact on well-being.
    • Experiencing bias and discrimination based on a variety of factors, including race, immigration status, age, disability, insurance coverage, and more.
    • Lack of high-quality care geographically close by and transportation challenges getting to appointments far from where they live.
    • Clinicians are not trained to help beyond immediate medical needs and those other needs are often much more pressing, such as housing, food insecurity, immigration status, and loss of income.

    Phase 2: Quantitative Survey

    The survey was fielded to 6126 households with a survey completion rate of 30.3%. The margin of error was ±3.08 percentage points and the median duration to complete the survey was 12 minutes. The population consisted of 1854 adults (955 women, 51.5%; mean age 48.4, SD 17.5 years). Table 1 shows the demographics of the survey respondents, with both unweighted and weighted values. Of 1854 surveys, 94.5% occurred online (not by phone) and 2% occurred in Spanish. The respondents were 65.7% white (non-Hispanic), 10.4% Black (non-Hispanic), and 15.5% Hispanic. Approximately 1 in 5 (19.8%) respondents were categorized as having a serious illness and 18.9% self-identified as having a disability.

    Table 1. Quantitative survey population characteristics (N=1854), comprising a nationally representative sample of US adults fielded from April to May 2021, in phase 2 of the mixed methods study.
    CharacteristicUnweighted value, n (%)Weighted value, %
    Gender: woman955 (51.5)51.9
    Age (years)

    		18-2470 (3.8)7.0

    		25-34415 (22.4)22.5

    		35-44292 (15.7)17.3

    		45-54255 (13.8)12.8

    		55-64354 (19.1)18.7

    		65-74325 (17.5)14.9

    		75+143 (7.7)6.8
    Race/ethnicity

    		White, non-Hispanic1218 (65.7)61.2

    		Black, non-Hispanic193 (10.4)12.0

    		Asian, non-Hispanic50 (2.7)6.5

    		>2 races or other, non-Hispanic105 (5.7)3.7

    		Hispanic288 (15.5)16.7
    Annual household income (US $)

    		<30,000437 (23.6)17.8

    		30,000 to <60,000526 (28.4)25.1

    		60,000 to <100,000464 (25.0)30.1

    		≥100,000427 (23.0)26.9
    Health insurance status

    		Employer or spouse’s employer807 (43.5)44.9

    		Medicare412 (22.2)19.9

    		Other447 (24.1)23.7

    		No insurance188 (10.1)11.5
    Current employment status: working1079 (58.2)59.1
    Education

    		Less than high school58 (3.1)9.8

    		High school graduate or equivalent317 (17.1)27.8

    		Vocational/technical school/some college/associates degree794 (42.8)27.6

    		Bachelor’s degree386 (20.8)19.6

    		Postgraduate study/professional degree299 (16.1)15.2
    People with disability351 (18.9)15.6
    Region

    		Northeast256 (13.8)17.3

    		Midwest491 (26.5)20.7

    		South628 (33.9)38.0

    		West479 (25.8)23.9

    Phase 3: Online Forums

    A total of 5191 individuals received the screening questionnaire, with 1512 qualifying individuals invited to participate. Of the 1512 individuals invited to participate in the forums, 644 (42.6%) joined and 90.0% (n=580) completed every activity and comprised the participant pool. Online forum participants’ demographics are outlined in Table 2. Approximately 60.7% of participants identified as women, compared to 50.5% of the US population [31]. An estimated 70.3% of participants were white (non-Hispanic), 16.0% Black (non-Hispanic), 9.5% Hispanic, and 1.9% Asian, compared to census estimates of 59.3%, 13.6%, 18.9%, and 6.1%, respectively [31]. Approximately 15.9% of the participants were categorized as having a serious illness, 15.2% self-identified as having a disability, and 18.4% self-identified as a caregiver.

    Table 2. Qualitative online forum participant characteristics (N=580), comprising a national sample of US adults fielded from June to August 2021, in phase 3 of the mixed methods study.
    CharacteristicsParticipants, n (%)a
    Gender: woman352 (60.7)
    Age (years)

    		18-2417 (2.9)

    		25-3483 (14.3)

    		35-4485 (14.7)

    		45-54107 (18.4)

    		55-64162 (27.9)

    		65-7590 (15.5)

    		75+11 (1.9)

    		Unknown25 (4.3)
    Race/ethnicity

    		White, non-Hispanic408 (70.3)

    		Black, non-Hispanic93 (16.0)

    		Asian, non-Hispanic11 (1.9)

    		>2 races or other, non-Hispanic13 (2.2)

    		Hispanic55 (9.5)
    Annual household income (US $)

    		<25,000111 (19.1)

    		25,000 to <50,000173 (29.8)

    		50,000 to <100,000168 (29.0)

    		≥100,000103 (17.8)

    		Unknown25 (4.3)
    People with disability88 (15.2)
    Region

    		Northeast187 (32.2)

    		Midwest94 (16.2)

    		South217 (37.4)

    		West82 (14.1)

    aPercentages may not all add to 100% due to rounding.

    Furthermore, participants highlighted that they enjoyed the opportunity to engage in manner through online forums, that they learned from one another and their experiences, and that some even felt motivated to take action in their own health journey (see Textbox 2).

    Textbox 2. Reflections from online forum participants, comprising a national sample of US adults fielded from June to August 2021, in phase 3 of the mixed methods study.

    Attitudes

    • “I enjoyed participating in this study. Thank you for the opportunity to voice my opinions.” [middle-aged multiracial woman with serious illness and disability]
    • “Thank you for this fun group. I learned a lot from the other members and was glad I was able to contribute my thoughts. I really felt heard.” [young Latina woman with comorbid illness]

    Knowledge

    • “This was a great group to be in. I appreciated everyone’s thoughts/comments. Some of them were some real eye openers. Thank you for allowing me to be a part of this group. I appreciated it.” [older white woman with disability]
    • “It has been not only a pleasure but also a learning experience for me as well. It made me think of health issues that could and will arise as time goes by and it’s good to have plans and invest time in surrounding yourself and family with a health care environment that will be proactive about your care and well being. Again my thanks to you Beth and the Coalition.” [older Black man]

    Behaviors (engagement/activation)

    • “It has inspired me to not be afraid to talk up and give my opinions regarding whatever situation but especially health care. Only because I used to be so nervous walking into a doctor’s office. Now, I’ve learned to put my fears aside and ask questions and express myself because that is the only way my doctor is able to know me better and prescribe my treatments.” [older woman with serious illness]
    • “For a while I have been procrastinating about designating a proxy to speak for me regarding health care decisions…and getting a will. Why? Because like most human beings I want to concentrate on life in the present instead of worrying too much about sickness and death in the future. It is a mechanism of defense we human beings have. The discussions here addressing these topics made me realize that I have to take action now, grab the bull by the horns as they say. These decisions and actions are not easy but necessary because even if we are healthy and alive today we never know what can happen tomorrow. I have to take action. For starters, I have to think carefully about who will be my health care proxy and I have to work on getting a will.” [younger man with comorbid illness]

    Understanding people’s health care communication experiences in the past is important as these experiences will influence their willingness to seek health care services [32] and in turn engage in future serious illness conversations. SIC strategies improve quality of life, enhance communication quality, reduce psychological distress, and promote positive patient and clinician experiences [33-38]. This community-informed mixed methods research study utilized online forums to better understand how SIC and ACP fit in the context of people’s prior experiences and life priorities. This study presents a unique approach with online engagement that is feasible and offers benefits over other traditional research approaches. We engaged 46 individual community partners, including patients, clinicians, and serious illness national experts, in our first phase of research, obtaining rich and compelling feedback that we incorporated in the design of study instruments. In our quantitative second phase, we used an intentionally recruited census-representative cohort of US adults through the NORC AmeriSpeak panel to reach over 1850 adults, which is in line with the recent large Kaiser Family Foundation National Serious Illness Care Survey with 2000 respondents [15]. Our survey population included nearly 20% respondents who have a serious illness, which is a greater proportion than included in the Kaiser Family Foundation’s survey, and 19% who have a disability, which is below the 25.7% estimated prevalence of US adults who live with at least one disability [39]. Our survey population also included 1386 (74.7%) younger and middle-aged adults (less than 65 years), who are often not captured relative to health care lived experience, beliefs, and concerns related to serious illness care. In our qualitative phase, we used asynchronous online forums as a method of engaging 580 US adults, a much larger number of people than traditional qualitative method styles can obtain, to ensure that the voices of a few are not used to represent entire historically marginalized populations. Recruitment for the online forums focused on engaging individuals from historically marginalized communities that are often underincluded in research efforts. In recruitment of diverse populations, it is important to assure compensation [40]. In all phases of our work, including online forums, we provided reimbursement for participation. With intentional efforts to oversample people from historically marginalized communities, we achieved overrepresentation of individuals who are over 65 years old, identify as non-Hispanic Black, and who have an annual income below the median US household income in the online forum population. We remained underrepresented in terms of individuals who identify as Hispanic/Latino and non-Hispanic Asian in our online forum population.

    Our study achieved mixed results in terms of diverse racial/ethnic participant engagement in the quantitative and qualitative phases. In the quantitative phase, where we relied on the prerecruited panel of potential participants, we achieved lower than census levels of representation from non-Hispanic Black (10.4% vs 13.6% in the census), Hispanic/Latino (15.5% vs 18.9% in the census), and non-Hispanic Asian (2.7% vs 6.1% in the census) individuals [31]. These findings reflect that even when utilizing a census-representative survey sampling population, additional efforts may be needed to achieve census-level representation of historically marginalized racial/ethnic communities. In the qualitative phase, where we conducted intentional oversampling of Black and Latino individuals, we achieved higher than census levels of representation of non-Hispanic Black individuals (16.0% vs 13.6% in the census) and lower than census levels of representation of Hispanic/Latino (9.5% vs 18.9% in the census) and Asian (1.9% vs 6.1% in the census) individuals [31]. Importantly, the large size of the online forum population (N=580) reflected 93 Black and 55 Hispanic/Latino participants who were included in the study, ensuring that the perspectives of only a small handful of individuals are not extrapolated to reflect an entire historically marginalized community. This online forum recruitment highlights that intentional oversampling can achieve overrepresentation of some populations, including non-Hispanic Black individuals, but additional effort to offer participation in Spanish may improve engagement of Hispanic/Latino individuals, as two out of three Latino individuals report Spanish as their language preference [41], and Asian individuals, who have also been historically underincluded and often grouped even though they reflect a wide variation of languages, cultures, and backgrounds [42].

    Prior studies have qualitatively examined patients’ health care experience and others have utilized publicly available online forum comments for qualitative analysis [43,44]. There has been some use of online forums to qualitatively collect information from patients [43-47], although here we describe the benefit of using online forums that allowed for tailored recruitment and greater representation from groups that are historically marginalized. This mixed methods approach, especially with such a flexible design for how to engage participants in online forums, allowed for rich, people-centered results in a scalable format. The recruitment approach engaged individuals from specific subgroups whose voices are not always elevated in research, including individuals from some historically marginalized communities (eg, 93 Black individuals and 55 Hispanic/Latino individuals) although notably not all (eg, only 11 Asian individuals participated). This helped to ensure that we heard varied perspectives within groups as well as between groups. The size and interactive nature of the forums allowed participants to surface themes that we did not know to ask about, such as weight bias, and probe concurrently through the moderator as they arose. We were also able to explore survey questions we needed further insights on to interpret the quantitative survey responses.

    Finally, participants shared that they were engaged and satisfied with the online forum engagement process, some even sharing that they felt inspired to do new or different things (eg, finding a new doctor or speaking up more at a visit) given the content they were exposed to during the forums through the structured exercises as well as from interacting with each other. This feedback informally suggests that participation in this study increased their patient activation, although formal assessment of the Patient Activation Measure was not conducted. Prior research by Hibbard and Greene [48] outlined that higher patient activation is linked with improved health outcomes. Furthermore, the same research group highlighted that when patient activation increases, it is frequently associated with improved health outcomes and lower costs [49]. Participants’ positive reflections on their online forum experience suggest that this is a feasible way to engage a wide range of perspectives in qualitative research.

    There are also study limitations that should be mentioned. The quantitative survey was fielded in English and Spanish, as well as electronically online and by telephone, although respondents were limited to those participating in NORC’s AmeriSpeak panel, which may inherently represent a population typically more willing to engage in survey research. The survey completion rate was 30%, which is in line with probability-based survey panel completion rates using the NORC AmeriSpeak panel [50]. Statistical weighting was used to overcome sampling bias by adjusting to the Current Population Survey and for interactions. In our qualitative online forums, participation was likely limited by the forums being conducted only in English, online only, and requiring typing for many of the activities. This likely contributed to underrepresentation of Hispanic/Latino and non-Hispanic Asian participants and possibly contributed to less engagement from people with disability, who may experience challenges with eyesight and typing ability required to engage in this forum. Future studies could consider offering a parallel forum in Spanish and ensuring browser/forum compatibility with text-to-voice and voice-to-text software for individuals with eyesight or typing challenges.

    There are important opportunities for future research. In addition to highlighting the recruitment of individuals from historically marginalized communities in this mixed methods study, further research highlighting the differences in people’s prior health care experiences and engagement in SIC and ACP across individual characteristics and identities (eg, race/ethnicity, income, serious illness, and disability) is important. Future qualitative research could consider using a parallel online forum offered in Spanish to engage perspectives from Hispanic/Latino individuals who prefer Spanish. It would also be helpful to better understand the barriers and facilitators for people from historically marginalized communities to engage in serious illness conversations, building on prior work exploring barriers and facilitators to SIC among Black individuals and increasing understanding for Hispanic/Latino and Asian individuals [51].

    Here, we outline a community-informed mixed methods approach to understanding people’s prior health care experiences and testing the framing of public messages to encourage engagement with clinicians and action around health, in service of finding new strategies to improve serious illness care. Our approach of engaging the community throughout the study design and execution focused our study, enriched our study instruments, and yielded important findings. This study can serve as a feasible model of choosing methodologies that allow for engagement with target communities and in-depth exploration of research aims, which in our case is to better understand how to improve serious illness care and communication.

    Acknowledgments

    This work was funded by sponsors of Massachusetts Coalition for Serious Illness Care, notably Blue Cross Blue Shield of Massachusetts, and with dedicated grant support from The John A Hartford Foundation and the Cambia Health Foundation, who together made this research possible as part of their innovative MessageLab program. We are indebted to Tony Back and Marian Grant, who lead MessageLab. We are grateful to the organizations forming part of MessageLab who provided guidance and support throughout the study. These organizations include: the American Academy of Hospice and Palliative Medicine (AAHPM), Ariadne Labs, the Center to Advance Palliative Care (CAPC), the Coalition to Transform Advanced Care (C-TAC), the National Coalition for Hospice and Palliative Care, the National Hospice and Palliative Care Organization (NHPCO), National POLST (Portable Medical Orders), Respecting Choices, The Conversation Project, and Vital Talk. CD also received funding and support from the Commonwealth Fund Fellowship in Minority Health Policy at Harvard University. We also want to thank the voices from our community collaborators who took a deep dive with us and helped ensure that we were asking the right questions, in the right way, with the right emphasis to capture what really matters to people. Thank you to Manny Lopes and the Federally Qualified Health Center (FQHC) East Boston Neighborhood Health Center for its staff participation in focus groups and connecting us to older adults for in-depth interviews. Thank you to those that provided feedback through our online partner survey, to Kate DeBartolo and her team at The Conversation Project, and to Jon Broyles and Adriana Krasinsky at C-TAC for their supportive outreach in connecting us to members in their communities. Thank you especially to Shirley Roberson (C-TAC) for the many hours spent working on the questions, approaches, and ideas. We benefited greatly from the expertise, experience, and previous research done by colleagues and are so grateful for the time they spent commenting, brainstorming, and supporting this work. Our thanks to Rebekah Angove (Patient Advocate Foundation), Rebecca Kirch (National Patient Advocate Foundation), Diane Meier and Lisa Morgan (CAPC), Liz Hamel (Kaiser Family Foundation), Rebecca Sudore (University of California, San Francisco), Glynn Elwyn (The Dartmouth Institute), and Cynthia Carter Perrilliat (Alameda County Care Alliance). We also benefited greatly from the voices, networks, and expertise of our Massachusetts Coalition collaborators: Rachel Broudy (Ariadne Labs), Ellen DiPaola (Honoring Choices Massachusetts), Lachlan Forrow (retired Beth Israel Lahey Health), Arlene Germain (Massachusetts Advocates for Nursing Home Reform), Joan Reede and Emorcia Hill (Harvard Medical School Office for Diversity Inclusion & Community Partnerships), Lisa Iezzoni (The Mongan Institute), Vicki Jackson (Massachusetts General Hospital), Paul Lanzikos (Dignity Alliance Massachusetts), Alexis Levitt (Massachusetts Chapter of the National Academy of Elder Law Attorneys), James Lomastro (Dignity Alliance Massachusetts), Nathalie McIntosh (Massachusetts Health Quality Partners), and Sandy Novack (Dignity Alliance Massachusetts). We also thank Zhimeng Jia and Joanna Paladino for their critical review and feedback of this manuscript. Generative artificial intelligence was not used in the preparation of this manuscript.

    Data Availability

    The data sets generated during and/or analyzed during this study are not publicly available due to terms of consent for study participation but are available from the corresponding author on reasonable request.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Phase 2: quantitative survey instrument, March 2021.

    DOCX File , 38 KB
    Multimedia Appendix 2

    Phase 3: online forum activity guide, June 2021.

    DOCX File , 50 KB
    Multimedia Appendix 3

    Phase 3: online forum activity guide, August 2021.

    DOCX File , 39 KB
    1. Armstrong K, Ritchie C. Research participation in marginalized communities - overcoming barriers. N Engl J Med. Jan 20, 2022;386(3):203-205. [CrossRef] [Medline]
    2. George S, Duran N, Norris K. A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans, and Pacific Islanders. Am J Public Health. Feb 2014;104(2):e16-e31. [CrossRef] [Medline]
    3. Barrett NJ, Hasan M, Bethea K, Johnson KS. The fierce urgency of now: addressing racial and ethnic disparities in serious illness care. N C Med J. Jul 08, 2020;81(4):254-256. [FREE Full text] [CrossRef] [Medline]
    4. Mack JW, Paulk ME, Viswanath K, Prigerson HG. Racial disparities in the outcomes of communication on medical care received near death. Arch Intern Med. Sep 27, 2010;170(17):1533-1540. [FREE Full text] [CrossRef] [Medline]
    5. Johnson KS. Racial and ethnic disparities in palliative care. J Palliat Med. Nov 2013;16(11):1329-1334. [FREE Full text] [CrossRef] [Medline]
    6. Frydman JL, Gelfman LP, Morillo J, Allen OS, Bickell NA, Kwon D, et al. Racial/ethnic disparities in serious illness communication for patients with cancer. J Clin Oncol. Jun 01, 2022;40(16_suppl):6540-6540. [CrossRef]
    7. Rhodes RL, Barrett NJ, Ejem DB, Sloan DH, Bullock K, Bethea K, et al. A review of race and ethnicity in hospice and palliative medicine research: representation matters. J Pain Symptom Manage. Nov 2022;64(5):e289-e299. [CrossRef] [Medline]
    8. Sanders JJ, Paladino J, Reaves E, Luetke-Stahlman H, Anhang Price R, Lorenz K, et al. Quality measurement of serious illness communication: recommendations for health systems based on findings from a symposium of national experts. J Palliat Med. Jan 01, 2020;23(1):13-21. [CrossRef] [Medline]
    9. Shim JK. Cultural health capital: a theoretical approach to understanding health care interactions and the dynamics of unequal treatment. J Health Soc Behav. Mar 24, 2010;51(1):1-15. [CrossRef] [Medline]
    10. Epstein RM, Street Jr RL. Patient-centered communication in cancer care: promoting healingreducing suffering. Patient-centered communication in cancer care. National Cancer Institute. 2007. URL: https://cancercontrol.cancer.gov/sites/default/files/2020-06/pcc_monograph.pdf [accessed 2020-10-19]
    11. Clark MA, Person SD, Gosline A, Gawande AA, Block SD. Racial and ethnic differences in advance care planning: results of a statewide population-based survey. J Palliat Med. Aug 2018;21(8):1078-1085. [CrossRef] [Medline]
    12. Huang IA, Neuhaus JM, Chiong W. Racial and ethnic differences in advance directive possession: role of demographic factors, religious affiliation, and personal health values in a national survey of older adults. J Palliat Med. Feb 2016;19(2):149-156. [FREE Full text] [CrossRef] [Medline]
    13. Smith AK, McCarthy EP, Paulk E, Balboni TA, Maciejewski PK, Block SD, et al. Racial and ethnic differences in advance care planning among patients with cancer: impact of terminal illness acknowledgment, religiousness, and treatment preferences. J Clin Oncol. Sep 01, 2008;26(25):4131-4137. [FREE Full text] [CrossRef] [Medline]
    14. Harrison KL, Adrion ER, Ritchie CS, Sudore RL, Smith AK. Low completion and disparities in advance care planning activities among older Medicare beneficiaries. JAMA Intern Med. Dec 01, 2016;176(12):1872-1875. [FREE Full text] [CrossRef] [Medline]
    15. DiJulio B, Hamel L, Wu B, Brodie M. Serious illness in late life: the public's views and experiences. Kaiser Family Foundation. Nov 2017. URL: https://www.kff.org/other/report/serious-illness-in-late-life-the-publics-views-and-experiences/ [accessed 2020-10-19]
    16. Nouri S, Lyles CR, Rubinsky AD, Patel K, Desai R, Fields J, et al. Evaluation of neighborhood socioeconomic characteristics and advance care planning among older adults. JAMA Netw Open. Dec 01, 2020;3(12):e2029063. [FREE Full text] [CrossRef] [Medline]
    17. Nouri SS, Barnes DE, Volow AM, McMahan RD, Kushel M, Jin C, et al. Health literacy matters more than experience for advance care planning knowledge among older adults. J Am Geriatr Soc. Oct 19, 2019;67(10):2151-2156. [FREE Full text] [CrossRef] [Medline]
    18. Massachusetts Survey on Advance Care Planning and Serious Illness Care, Spring 2018 Survey of Massachusetts Residents. Massachusetts Coalition for Serious Illness Care. 2018. URL: https://tinyurl.com/bdf6hmwu [accessed 2020-11-15]
    19. Advancing the language of advance care planning: a messaging research project. Massachusetts Coalition for Serious Illness Care. Nov 2019. URL: https://tinyurl.com/33mb4wh6 [accessed 2020-10-10]
    20. 2017 consumer research: deep dive on conversations. Massachusetts Coalition for Serious Illness Care. URL: https://tinyurl.com/yc8aempu [accessed 2020-11-15]
    21. AmeriSpeak. URL: https://amerispeak.norc.org/ [accessed 2023-04-26]
    22. NORC at the University of Chicago. URL: https://www.norc.org/Pages/default.aspx [accessed 2023-04-26]
    23. Technical overview of the AmeriSpeak® panel NORC's probability-based household panel. AmeriSpeak and NORC and the University of Chicago. URL: https://tinyurl.com/bdete8wa [accessed 2022-11-15]
    24. HHS implementation guidance on data collection standards for race, ethnicity, sex, primary language, and disability status. Assistant Secretary for Planning and Evaluation (ASPE). Oct 2011. URL: https:/​/aspe.​hhs.gov/​reports/​hhs-implementation-guidance-data-collection-standards-race-ethnicity-sex-primary-language-disability [accessed 2023-04-12]
    25. Population surveys that include the Standard Disability Questions. Centers for Disease Control and Prevention. Sep 16, 2020. URL: https://www.cdc.gov/ncbddd/disabilityandhealth/datasets.html [accessed 2023-04-12]
    26. Lucid, a Cint Group company. URL: https://luc.id/page/2/?et_blog [accessed 2023-04-26]
    27. Crabtree BF, Miller WL. Doing qualitative research. Newbury Park, CA. SAGE Publications, Inc; 2012.
    28. Full Circle Research. 2021. URL: https://www.ilovefullcircle.com/ [accessed 2023-04-26]
    29. itracks: Insight for growth. URL: https://www.itracks.com/board/ [accessed 2023-04-26]
    30. Neergaard MA, Olesen F, Andersen RS, Sondergaard J. Qualitative description - the poor cousin of health research? BMC Med Res Methodol. Jul 16, 2009;9:52. [FREE Full text] [CrossRef] [Medline]
    31. QuickFacts United States. US Census Bureau. URL: https://www.census.gov/quickfacts/fact/table/US/SEX255221 [accessed 2023-04-19]
    32. Butler SM, Sheriff N. How poor communication exacerbates health inequities – and what to do about it. Brookings Institution. Feb 22, 2021. URL: https://tinyurl.com/3e95pa3x [accessed 2023-09-05]
    33. Curtis JR, Downey L, Back AL, Nielsen EL, Paul S, Lahdya AZ, et al. Effect of a patient and clinician communication-priming intervention on patient-reported goals-of-care discussions between patients with serious illness and clinicians: a randomized clinical trial. JAMA Intern Med. Jul 01, 2018;178(7):930-940. [FREE Full text] [CrossRef] [Medline]
    34. Bernacki R, Paladino J, Neville BA, Hutchings M, Kavanagh J, Geerse OP, et al. Effect of the serious illness care program in outpatient oncology: a cluster randomized clinical trial. JAMA Intern Med. Jun 01, 2019;179(6):751-759. [FREE Full text] [CrossRef] [Medline]
    35. Kumar P, Wixon-Genack J, Kavanagh J, Sanders JJ, Paladino J, O’Connor NR. Serious illness conversations with outpatient oncology clinicians: understanding the patient experience. JCO Oncol Pract. Dec 2020;16(12):e1507-e1515. [CrossRef]
    36. Paladino J, Koritsanszky L, Nisotel L, Neville BA, Miller K, Sanders J, et al. Patient and clinician experience of a serious illness conversation guide in oncology: a descriptive analysis. Cancer Med. Jul 04, 2020;9(13):4550-4560. [FREE Full text] [CrossRef] [Medline]
    37. You J, Singh J, Simon J, Ma IW, Paladino J, Swinton M, et al. A quality improvement initiative to implement the Serious Illness Care Program on hospital medical wards. Can J Gen Int Med. Feb 08, 2022;17(1):29-51. [CrossRef]
    38. Wright AA, Zhang B, Ray A, Mack JW, Trice E, Balboni T, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. Oct 08, 2008;300(14):1665-1673. [FREE Full text] [CrossRef] [Medline]
    39. Prevalence of disability and disability types by urban-rural county classification - United States, 2016. Centers for Disease Control and Prevention. Oct 27, 2019. URL: https://www.cdc.gov/ncbddd/disabilityandhealth/features/disability-prevalence-rural-urban.html [accessed 2023-04-19]
    40. Bierer BE, White SA, Gelinas L, Strauss DH. Fair payment and just benefits to enhance diversity in clinical research. J Clin Transl Sci. Jul 14, 2021;5(1):e159. [FREE Full text] [CrossRef] [Medline]
    41. American Community Survey B16001: Language Spoken at Home 2019. United States Census Bureau. URL: https://data.census.gov/table?q=Language+Spoken+at+Home&tid=ACSST1Y2019.S1601 [accessed 2023-04-12]
    42. Trinh-Shevrin C, Islam NS, Rey MJ. Asian American communities and health: context, research, policy, and action. 1st edition. San Francisco, CA. Jossey-Bass; 2009.
    43. Pearson SE, Taylor J, Hoare DJ, Patel P, Baguley DM. Exploring the experiences of cancer patients with chemotherapy-induced ototoxicity: qualitative study using online health care forums. JMIR Cancer. Mar 14, 2019;5(1):e10883. [FREE Full text] [CrossRef] [Medline]
    44. Yi EG, Adamek ME. Alzheimer's caregivers' experience with and perceptions of the Affordable Care Act: thematic analysis of online discussion forums. J Appl Gerontol. Dec 14, 2021;40(12):1796-1806. [CrossRef] [Medline]
    45. Horgan A, McCarthy G, Sweeney J. An evaluation of an online peer support forum for university students with depressive symptoms. Arch Psychiatr Nurs. Apr 2013;27(2):84-89. [FREE Full text] [CrossRef] [Medline]
    46. Im EO, Lee B, Chee W, Dormire S, Brown A. A national multiethnic online forum study on menopausal symptom experience. Nurs Res. 2010;59(1):26-33. [FREE Full text] [CrossRef] [Medline]
    47. Ferrante JM, Friedman A, Shaw EK, Howard J, Cohen DJ, Shahidi L. Lessons learned designing and using an online discussion forum for care coordinators in primary care. Qual Health Res. Nov 11, 2016;26(13):1851-1861. [FREE Full text] [CrossRef] [Medline]
    48. Hibbard JH, Greene J. What the evidence shows about patient activation: better health outcomes and care experiences; fewer data on costs. Health Aff. Feb 2013;32(2):207-214. [CrossRef] [Medline]
    49. Greene J, Hibbard JH, Sacks R, Overton V, Parrotta CD. When patient activation levels change, health outcomes and costs change, too. Health Aff. Mar 2015;34(3):431-437. [CrossRef] [Medline]
    50. Technical Overview of the AmeriSpeak® Panel NORC'S Probability-Based Household Panel. AmeriSpeak. Feb 08, 2022. URL: https://tinyurl.com/bdete8wa [accessed 2023-04-12]
    51. Sanders JJ, Johnson KS, Cannady K, Paladino J, Ford DW, Block SD, et al. From barriers to assets: rethinking factors impacting advance care planning for African Americans. Palliat Support Care. Jun 2019;17(3):306-313. [CrossRef] [Medline]

    ‎
    ACP: advance care planning
    ACS-6: American Community Survey
    FQHC: Federally Qualified Health Center
    MCSIC: Massachusetts Coalition for Serious Illness Care
    NORC: National Opinion Research Center
    SIC: serious illness communication

    Edited by A Mavragani; submitted 28.04.23; peer-reviewed by D Chao, N Finn; comments to author 20.08.23; revised version received 10.09.23; accepted 28.09.23; published 06.12.23.

    Copyright

    ©Carine Davila, Stephanie H Chan, Anna Gosline, Zamawa Arenas, Jane Kavanagh, Brian Feltz, Elizabeth McCarthy, Tyrone Pitts, Christine Ritchie. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.12.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.