Patients Contributing to Visit Notes: Mixed Methods Evaluation of OurNotes

Introduction

Background

Patient engagement is essential for improving patient experience, reducing the cost of care, and improving population health [1]. With the proliferation of secure patient portals over the past decade, patients have gained a powerful engagement tool [2,3]. Millions of patients and their care partners now go on the web to communicate with members of the care team and review their medical records. Beginning in April 2021, as part of the 21^st Century Cures Act, patients nationwide have access to virtually all information in their electronic health records, including the notes of their clinicians [4].

Hundreds of provider organizations implemented open notes on their patient portals following publication of the first open notes study in 2012 [5,6]. Patients rate note reading as very important for remembering what happened in their office visits, remembering their care plan, taking care of their health, and feeling in control of their care [7-10]. Patients have also expressed interest in doing more than passively reading their notes. In the first study of open notes, 60% wanted to be able to comment on notes, and 35% wanted to be able to approve their notes [5]. Though few primary care physicians thought patients should approve the content of notes, approximately one-third agreed they should be able to comment. Researchers have since shown that reading notes and inviting patients to comment on their notes can benefit patient safety [11-16]. Though many clinicians face time pressures, a majority report spending the same or less time writing open notes [17]. The National Academy of Medicine’s Vital Directions for Health and Health Care initiative recommends that patients and authorized family caregivers be able to modify their health records [18]. Though many ambulatory care practices collect information from patients in brief questionnaires, we know of no efforts inviting them to actively write in their records. As a step in this direction, we decided to mount a pilot inviting patients to contribute to their visit notes, thus transforming the official record of their care into cogenerated OurNotes.

We designed, implemented, and evaluated portal mechanisms for patients and their care partners to contribute to notes of providers documenting their ambulatory visits. Following advice from expert interviews, we determined to focus on previsit communications by asking patients to (1) write a brief history since the last visit and (2) to identify up to three priorities for the upcoming visit [19].

Objectives

Our objectives were to have providers incorporate this previsit input into their visit notes, thereby taking a first step toward OurNotes, and to assess patient and provider experiences with this pilot intervention. We hypothesized that clinicians and patients would find it beneficial to submit the information before visits, especially patients with a heavy burden of illness, and that the intervention would be time-neutral or possibly save time for clinicians. This paper reports the principal findings from 12-month pilots of this intervention in 4 sites.

Methods

Overview

Primary care practices associated with 4 sites across the United States participated: Beth Israel Deaconess Medical Center (BIDMC) in Boston, UCHealth (UCH) in Aurora, Colorado (including a women’s health clinic), Dartmouth-Hitchcock Medical Center (DHMC) in Lebanon, New Hampshire, and University of Washington Medicine (UW) in Seattle (including a safety net clinic and a clinic that serves primarily privately insured patients). Each site has well-established mechanisms for offering open notes to patients. The OurNotes and site project teams developed an overall intervention design and evaluation plan. The study protocols were approved by the institutional review boards of the participating institutions.

Intervention

The intervention included primary care providers and patients who were registered on the patient portal. During the 12-month pilot period, these patients were invited via a secure portal message or email to complete a form addressing two questions before a scheduled visit: How have you been since your last visit?What are the most important things you would like to discuss at your visit? (list up to 3). Answers were limited to 2000 and 300 characters, respectively. Clinicians could view the completed forms before and during the visit, and when they wrote the visit note, they could decide whether and to what degree to incorporate the patient’s words.

Between mid-2017 and early 2019, each site built its own implementation process and supported the information technology infrastructure. Although all asked the same two questions up to 1 week before the visit, the sites differed in their approaches (Multimedia Appendix 1). Notably, 2 sites implemented practice-wide approaches with many clinicians (UC and UW), and 2 recruited volunteer clinicians (BIDMC and DHMC). Two sites (UC and UW) used Epic (Epic Systems Corporation, Verona, Wisconsin) to facilitate the insertion of patients’ submissions into notes. They were able to launch 9 months earlier than the other 2 sites.

In all sites, invitations to patients (Multimedia Appendix 2) included links to the form (Multimedia Appendix 3). The invitation described the intervention, explained that completing the form was voluntary, and noted that the form might not be read if they did not keep the appointment. At BIDMC, UC, and UW, automated processes sent the previsit invitations; physicians at DHMC selected patients from the weekly schedule, and clinic staff or the physician sent the invitation message.

Evaluation

Pilot interventions were conducted between June 2018 and April 2020. Three sites (BIDMC, UC, and UW) used portal tracking data to monitor the use of the form throughout the pilot, and clinicians at DHMC made periodic estimates. The evaluation plan also included surveys of both clinicians and patients at each site and moderated discussions with the selected groups. However, the COVID-19 pandemic led to restrictions on research activities and diversion of information systems resources to support clinical care. Therefore, we were unable to complete all parts of the planned evaluation. We completed both patient and clinician surveys at the 2 sites that finished their 12-month pilots in 2019 (UC and UW), and surveyed clinicians at the other 2 sites (BIDMC and DHMC) in Spring 2020. However, without information systems resources to identify and track participants in early 2020, we were unable to conduct patient surveys at BIDMC or DHMC. We were also unable to convene focus groups.

The research team developed survey items based on pilot objectives, informal comments, and feedback from each site. We asked clinicians about the usefulness of patient responses, effects on workflow, and ideas for the future. We asked patients about the difficulties and benefits of answering the questions and about ideas for future improvements. To assess the burden of illness, we asked the patients if they had a chronic illness. The surveys included brief sets of sociodemographic items and both closed-ended and free-text questions. The questionnaires are available on request.

We invited clinicians who had received at least three completed previsit forms in the previous 6 months to complete the survey. We invited patients at UC and UW who had submitted at least one form during the pilot study. Clinicians in the 4 sites were contacted between June 2019 and July 2020, and UC and UW patients were contacted between September and October 2019. All participants received an invitation and up to two reminders. Each site offered an incentive of US $10 to US $50, following local guidelines. The surveys were conducted using Research Electronic Data Capture (REDCap, Vanderbilt).

Analysis

Descriptive statistics were used to summarize response rates for the survey, survey responses, and the proportion of visits for which patients returned forms (using portal tracking data at BIDMC, UC, and UW, and clinician estimates at DHMC). In addition, patient results were stratified according to whether they reported having a chronic illness.

We completed thematic analyses of the free-text comments from the surveys. Patients were asked two questions: one for ways to improve the OurNotes previsit process, and one for other comments. To analyze responses, 2 authors first scanned the responses and noted that both questions drew comments related to both OurNotes and to other topics; therefore, we combined the comments for analysis. We reviewed 30 comments independently and drafted six topics, then independently reviewed 30 new comments and defined the final set of themes for coding: comments about the two questions, the form, workflow, patient engagement, whether the submitted form was read by clinicians, and others. Using REDCap, both authors coded all responses with up to three codes each, and 1 selected representative examples.

Clinicians were asked a single question about how to improve the previsit OurNotes process and any other comments. Two authors reviewed 30 comments to establish a list of nine themes, then 1 author reviewed all comments, assigned 1 or more themes to each, and selected representative thematic examples.

Results

Overview

In the 3 sites with portal tracking, patients were invited to complete forms before 58,652 scheduled visits, 5365 (9.1%) forms were submitted, and in the site without portal tracking, forms were returned before approximately 50% (260/520) visits (Multimedia Appendix 1). We invited 174 clinicians to complete the evaluation survey, and 76 (43.7%) responded. Two respondents did not remember receiving any completed forms, leaving an analytic data set of 74 clinicians. We contacted 1962 patients who had submitted at least one form and 525 (26.8%) responded; 204 did not remember completing a form, leaving an analytic data set of 321 patients.

Clinician Results

Among the 74 clinician respondents, 49 (66%) were women, 61% (45/74) obtained their licenses since 2000, and 54% (40/74) reported 21 or more patient visits per week. On average, clinicians reported receiving between 1 and 3 patient forms per week in the previous 3 months (Table 1). Nearly all respondents agreed that having the information from the forms was a good idea, and 70% (52/74) reported that they usually or always incorporated responses of patients in the visit notes. Almost 90% (66/74) of clinicians reported that having the previsit information of patients either saved time or did not change the visit length.

The process of entering the responses of a patient to the record varied among the participating sites. In 2 sites where the previsit information could be automatically inserted into the visit note, 92% (45/48) preferred this approach. At the other 2 sites, most clinicians copied and pasted the information, 48% (11/23), read or typed all or part of the words of patients into the note, 13% (3/23), or simply referred to having read what the patient wrote, 30% (7/23).

Clinicians were asked to describe their experiences with the intervention and to suggest changes or improvements. Their 57 text responses centered on better notifications about completed forms, the need for easy insertion of patient text into the note, and ways to support patient participation (Textbox 1). Clinicians suggested that patients could be encouraged to be more engaged and taught about how to complete the form so that it would be more helpful for the visit.

Patient Results

Among the 321 patients who completed the survey from 2 sites (UC and UW), 57.9% (186/321) were women, 79.7% (255/320) were White, and 73.1% (234/320) had completed a 4-year college degree (Table 2). Half (158/320, 49.3%) of the respondents reported having three or more visits to their primary care provider in the previous 12 months, 68.2% (219/321) reported having one or more major chronic conditions, and 77.2% (247/320) reported their health as excellent, very good, or good. There were very few differences between respondents at the 2 sites (data not shown), other than more women responding in Colorado than in Washington (74% and 47% of respondents, respectively), and a higher proportion of respondents reporting White race in Colorado versus Washington (88% vs 74%, respectively).

More than half of the patients (170/321, 52.9%) completed more than one previsit form (Table 3). In general, few patients reported any difficulty, and more than 80.6% (259/321) had no difficulty answering the questions. About 68.8% (221/321) reported that answering the questions helped them prepare for the visit. Among the 85% (273/321) who reported reading their notes after the visit, 41% (112/273) found that the note included some or all that they had written on the previsit form, whereas more than one-third did not know whether what they had written was included in their visit notes. Patients overwhelmingly agreed that sending their commentary was a good idea (296/321, 92.2%). Most patients preferred to receive the previsit form 2-7 days before their visit. We found no substantive differences in responses between patients who did and did not report having a chronic illness (data not shown).

The patients wrote 282 comments in free text (Textbox 2). The coders agreed on codes for 161 responses and reached a consensus through discussion on the remaining 121. The comments most often (N=55) addressed the two questions in the form; about half wrote that they liked the questions as they were, and half found the questions too general. More specific comments related to the question asking for an interim history, with many making suggestions for more detailed queries, or requesting guidance on how to respond. The respondents wrote 33 comments about other aspects of the form and suggested some improvements. About half of the respondents commented on the space available for answers thought it adequate, and half wanted more room. Respondents saw the form as a tool supporting their engagement with care (27 comments), most often as a mechanism and incentive for collecting their thoughts before a visit. The 20 comments related to workflow often focused on improved visit efficiency, although several mentioned finding it inefficient when they were asked the same questions again at the visit; 27 comments referenced perceptions that the submitted forms were not read by their clinicians. Several suggested alerts to let the provider know the form was available, and to let the patient know it had been opened. Among other topics mentioned were concerns about privacy, patients for whom the form might present difficulties, and appreciation for specific staff, clinicians, portals, or institutions.

Discussion

Principal Findings

To our knowledge, this is the first study published on patients and clinicians cogenerating visit notes, or OurNotes. Patients were asked to prepare both interim histories about their conditions and their goals for an upcoming scheduled primary care visit, and participating patients and clinicians were strongly supportive. Consistent with our prime hypothesis, patients found completing the forms largely beneficial, and our hypothesis that OurNotes would have a neutral or positive impact on the time of clinicians appears to have been substantiated. We did not find additional benefits for patients with chronic illnesses.

In this pilot inquiry, participants offered many suggestions on how to improve the process. Patient respondents described how thinking through and providing the information supported their own engagement with care and helped visits flow more efficiently, but many wrote that they needed guidance about what to write. Clinicians often included the submissions of patients in their visit notes, suggested technical changes that would facilitate the process, and encouraged future efforts to teach patients to write interim histories that are more informative. Overall, both patients and clinicians suggest that this provides an important opportunity for better care. Based on this early proof of concept, all 4 sites decided after the 12-month pilot to continue the intervention, while considering also how to enhance this new approach to transparent patient-clinician communication and patient engagement.

Challenges in the Pilot Implementation

Patients, in aggregate, submitted forms for fewer than 10% of eligible visits. We have no direct evidence from nonparticipants about why they chose not to send forms, and demographic data to analyze differences between them and participants were not available. A number of factors could explain this low level of participation, including insufficient communication with patients about the project before the intervention began, and the stated desire of patients in survey comments for guidance in answering the questions. The factors discussed below may have also contributed to this. Although evaluating reasons for nonparticipation was not among the objectives of this pilot study, it will be important to ascertain this in future research.

Although patients generally liked the idea of submitting information before visits, if such submissions are not readily available to providers before or during the visit, it could prove hurtful, as some patients may feel their input is being overlooked. Another challenge is how to support patients who may feel intimidated by being asked to write free text. Perhaps emphasizing the acceptability of writing phrases rather than complete sentences, allowing dictation functions such as those on smartphones, or offering other ways to record a verbal history may help. For patients to prepare and for clinicians to respond, designing a variety of effective approaches will require input from clinicians, patients, researchers, and information systems specialists.

Incorporating patient-written text into visit notes was an entirely new functionality in all 4 sites, and each designed and implemented an approach within the timeframe of the pilot to enable the most important steps in the process while attempting to minimize the impact on clinicians’ workflow. All sites experienced occasional glitches; clinicians commented particularly about sometimes not knowing that a patient had submitted a form until after the visit, and about issues with getting text of patients into their notes. After 12 months, each practice clearly required technical changes to facilitate various steps in the process. Although most clinicians reported no negative impact on their time, this important factor needs further study.

Three Recommendations for Future Implementations

First, both patients and clinicians wanted a closed communication loop for previsit information. Clinicians expressed frustration about sometimes not knowing patients had submitted forms, or not being able to find them, and patients were disappointed or frustrated after taking time to complete the forms and then having their clinicians seemingly ignore what they had done. In a closed loop, clinicians would always know before an appointment that the patient had sent information, and patients would know whether the clinician had viewed their submissions. Using automated notifications for both patients and providers, technology can create such a system. Completed forms could be delivered directly to clinicians, although they worry about keeping up with an ever-growing in-box and potential medicolegal liability in unread items. Including links to submitted forms on daily schedules of clinicians could be one way to avoid additional messaging. Providers may also consider deadlines or messaging to encourage early patient submissions, in consideration for clinicians who review patients’ data the day or evening before a session, rather than at the time of the visit.

Second, both patients and clinicians explicitly asked for education that would help patients write informative interim histories. Some patients wrote of simply not knowing how to approach a request for a history. Lacking educational interventions, clinicians found the histories to be less useful than the visit priorities patients formulated. A foundational question concerns whether to ask patients to use a semistructured format, in addition to or in place of free text. In an informal meeting with members of one patient family advisory council, participants suggested that patients should be offered both options. Alternatively, might we prompt for interim histories by problem, or make forms that solicit unstructured histories before primary care visits, whereas those to subspecialists might be better suited to structured queries and responses?

Third, it is clear from the patient participation rate that we must learn more about how to encourage patients to take advantage of new opportunities to provide information to their clinicians outside of time-pressured encounters. For many, this involves registering more patients on patient portals, particularly those from vulnerable populations, and ensuring that they have adequate access, education, and support to use the portal [20,21]. Today, many patients fill out forms on paper or tablet computers in the waiting room [22]. Completing a web-based form at home facilitates timely review of prior open notes and may invite more consideration and thought, but it may also raise privacy concerns. Some survey respondents were intimidated by the form. Many of these concerns can be addressed with education, testimonials by patients and clinicians, and other outreach strategies. An appealing interface, dictation function, and seamless submission process may be helpful. Using some principles of behavioral economics, such as incentives for both patients and clinicians, may also improve rates of participation [23-25]. Encouraging patient engagement is always a multi-pronged, long-term undertaking.

Limitations

This pilot study had important limitations. Most importantly, the proportion of patients who returned previsit forms was modest, and we were unable to conduct postimplementation evaluations with patients in 2 of the 4 sites because of the COVID-19 pandemic. Although we found no difference between the experiences of patients with and without chronic illness, we did not examine records to more fully ascertain their burden of illness. Response rates were within the usual range for web-based surveys [26], but nearly 40% of the patients did not remember submitting a previsit form, a function perhaps of having completed a single form many months before the survey, or of the ever-increasing amount of electronic communication with which patients contend. Finally, we did not collect quantitative information about the proportion of submitted forms that were actually used in the visits.

Conclusions

Overall, in 4 different sites, each with a common goal but many degrees of freedom to design their own intervention, providers and patients liked the idea of patients providing information in advance of their visits and incorporating their observations into visit notes. Further development and research will be needed to assess barriers to patient participation, establish reliable notifications about information submitted by patients, and enable seamless incorporation of patient-generated text into notes.

This exploration is just beginning, and many questions arise. For both patients and clinicians, how should one balance time and effort before an encounter with time spent during a visit? Can cogenerating notes improve a crucial element of care: supporting mutual trust among patients, families, and clinicians? Could cogeneration of notes have particular benefits for those with multiple chronic conditions and those nearing the end of life? Can we effectively bring care partners into the process? Should we ask patients to think more broadly about their goals for health care, addressing issues germane to far more than an individual, time-constrained visit? As care delivery evolves in response to the COVID-19 pandemic, might OurNotes prove an effective component of telemedicine [27]? Future exploration may provide useful insights.