Fifteen Years’ Use of Patient-Reported Outcome Measures at the Group and Patient Levels: Trend Analysis

Introduction

From the time of Hippocrates, information originating from the patient has been considered indispensable. Today, few diagnoses can be established, and few treatments monitored sufficiently, solely by using paraclinical data without explicit information from the patient. However, until recently, such patient inputs were always shortened and interpreted by a clinician. With the introduction of the term health-related quality of life, systematic measurement was adopted for research in several clinical specialties [1]. The potential of its many applications was further boosted by the US Food and Drug Administration’s definition of a patient-reported outcome (PRO) as a measurement based on:

Any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else [2].

The first draft of this term appeared in 2006 [3] and the final version in 2009 [2]. Since 2004, we have collected PRO data (although the term PRO data was not coined at that time) from several Danish patient populations. The aim of this paper is to highlight trends during the last 15 years with respect to patient inclusion, the collection of PRO data, the processing of the data, and the actual applications and use of the PRO measurements.

Methods

Overview

All PRO data have been collected using the AmbuFlex/WestChronic PRO system, which was developed by the author in 2004 and has been continuously updated since. The system has been described in detail elsewhere [4,5]. The first version of the generic PRO system, WestChronic, was developed for mixed-mode (Web and paper) collection of PRO data for research purposes in clinical epidemiological studies with repetitive measurements. Based on experienced feasibility and high response rates, it was decided in 2007 to develop this system into a flexible, multipurpose PRO system. The goal was to use clinical PRO data as the basis for outpatient follow-up in selected patient groups.

AmbuFlex: Telehealth Patient-Reported Outcomes as the Basis for Follow-Up in Chronic Diseases

In AmbuFlex, outpatients report their symptoms from home at regular intervals instead of attending fixed visits at the outpatient clinic. The PRO measures are used to decide whether a patient needs or wishes an outpatient visit, and were developed to promote better symptom monitoring, more patient-centered care, and more efficient use of resources [5]. Specific questionnaires have been developed for each diagnostic group. The AmbuFlex concept consists of three generic elements: PRO data collection, PRO-based automated decision algorithm, and PRO-based graphical overview for clinical decision support.

The AmbuFlex/WestChronic Patient-Reported Outcomes System

The AmbuFlex/WestChronic system supports dynamic mixed-mode data collection with the use of the internet or paper forms, as well as automated communication to the patient and the clinician via personalized letters, emails, text messages, and secure electronic communication. All information regarding implemented projects, items and questionnaires, communication, clinical users, and patients resides in tables in a Structured Query Language database residing in the server park of Region Central Denmark. All administration of projects, questionnaires, users, and patients is supported by the system’s software and managed in browser windows.

The system has several integrations (Table 1 and Figure 1). All Danish citizens are assigned a unique 10-digit number (Civil Personal Registration [CPR] number), and continuously updated information on their current postal address and vital status is available from the Danish Civil Registration System [6]. This information is automatically collected online prior to any approach to patients. On-demand printing of questionnaires and letters, as well as scanning of incoming questionnaires with subsequent optical character recognition, is controlled by the system software, and results about all variables end up in result tables for the individual implemented projects in the same database. This occurs irrespective of whether Web or paper forms are used, and all results are instantaneously accessible. WestChronic may implement an arbitrary number of PRO projects with individual protocols, questionnaires, patients, and clinical users. For the patient and clinician, each implemented project appears as a unique PRO project with its own logo, domain, website, accompanying letters, contact information, etc. A new and rewritten version is underway.

‎View this figure

Data Analysis

The data for the present paper were generated by an ad hoc application in the system and transferred for analysis in the R package (R Foundation for Statistical Computing, Vienna, Austria). Patients who had been referred but not yet answered at least one questionnaire were not included. The data collection in the first project started in September 2004, and during 2004 a total of 301 questionnaires were collected from 241 patients; this period was considered a pilot phase and not included in the analysis.

The analysis of trends was based on a generic model applicable for any clinical health data [7]. Each application of PRO data is divided into four processes: patient identification (eg, registers, consecutive or individual referral), data collection (logistics and organization), data aggregation (at patient or at some specified group level) and the actual data use (Figure 2). In the present paper, the last two processes will be described together. The patient identification process involves the identification of patient(s) from whom data are to be collected, the data collection process involves the actual collection of health data, including logistic procedures, the data aggregation process involves the management and organization of collected data for the data use process, and the data use process involves the use of the health data for the purpose of the specified activity. Each process may be repeated or may take place simultaneously with the previous process (from [7]).

‎View this figure

Results

Primary Findings

An overview of selected projects is given in Table 2. In the first cohort studies, emphasis was on response rate, which was promoted by up to three reminders using email and letters and by giving the patient a free choice between Web and paper questionnaires. The maximum response rate was reached in patients with prostate cancer (Table 2), where the first questionnaire was answered by 95.9% of the patients. The PRO-based algorithm was first introduced in a screening program for patients with coronary heart diseases, and the AmbuFlex principle was fully unfolded in the implantation in epilepsy patients (Table 2, Figures 1 and 3).

Figure 4 and Table 3 show the development of the number of projects, patients, and received questionnaires. When considering that 2019 is only partly included, an exponential model of number of received questionnaires per calendar year explains 94% of the variation (Figure 4). At item level, a total of 13,538,760 responses have been received in the period. Until 5 years ago, web- and paper-based questionnaires were used with similar frequencies, but during the last 4 years web-based questionnaires have almost completely taken over. Projects where data was used at group level dominated until 2014, but PRO data used at the patient level is now by far the most frequent (Figure 5). Most projects and about one-third of the questionnaires now have an attached algorithm (AmbuFlex) by which the incoming answers are classified into two or three groups (Table 3 and Figure 3).

Patient Identification

In the first part of the 15-year period, all projects were register-based, epidemiologic cohort studies with defined inclusion criteria, while from 2015 onwards PRO data for clinical use at the patient level has dominated the scene (Table 3 and Figure 5). In the latter case, patients are individually referred by a clinician. Patient cohorts are still created at group level and are used in research for topics like outcome measures, validation, and studies of determinants of referral. In 2019, 16,062 new patients were included, of which 73.8% (11,854/16,062) were referred individually by a clinician while the rest were included as part of a group by means of batch processing.

Patient-Reported Outcome Data Collection

Fundamental changes have taken place with respect to PRO data collection over the course of very few years. Until 2014, paper-based communication was used in more than 90% of the approaches (Figure 6), while electronic communication was predominantly used in epidemiological studies with multiple measurements where about half of the communication was electronic (emailing and web-based answering) (Figures 4 and 6). A minor revolution took place November 1, 2014. From this date, all communication between the Danish authorities (including all public hospitals), and Danish citizens had to use a national, secure electronic mail system, called e-boks. All citizens are provided an e-boks, but while its use was previously voluntary, only people with disabilities can now be granted a dispensation and continue to receive paper letters from places like hospitals. After the AmbuFlex/WestChronic system gained access to e-boks (Table 1), this channel rapidly became the most important communication method to reach patients (Figure 6). These changes in communication methods to and from patients had consequences for turnaround time (ie, the time from asking the patient to answer until the questionnaire is received) (Figure 7). A total of 75% of the answers were received within 17 days in 2005 due to the homogeneous cohorts, but turnaround time increased slowly until 2014. At this time, the Danish Mail decreased their services due to economic problems caused by the steep fall in paper-based mail. The historic low turnaround time in 2019 is caused by a combination of e-boks communication and the web-answering method. Today, the first answers are received within minutes. Use of ordinary, unsecure email ended in 2018. Text messaging is still used for reminders in some projects but is mainly used as a 2-factor authentication tool, where the patient, after entering their credentials (CPR number), receives a 6-digit cipher code as a text message.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

Patient-Reported Outcome Data Aggregation and Data Use

While data from group-level projects are aggregated and used in the analysis phases of each epidemiological study, data in patient-level projects are, or should be, aggregated and used the date the questionnaire is received; however, this is not always the case. If PRO data are used as an add-on in connection with ordinary organized outpatient follow-up, it is up to each clinician to decide to open the PRO overview or not. In a multicenter project covering eight Danish hospitals, the patients treated for lung cancer answered a PRO questionnaire each time they went to the hospital for a follow-up visit, with the idea being that the PRO data should be used to enforce patient-clinician communication during the consultation (Table 2). For legal reasons, the AmbuFlex/WestChronic system keep automated log files of when and to whom the PRO data are shown. In total, a minority of responses (47%) were seen by a clinician (Figure 8). There were major differences between departments, ranging from 14-93% [34]. Similar figures have been found in other projects where the PRO data are an add-on to the existing outpatient set-up. In AmbuFlex, however, the PRO data are not an add-on but the basis for the follow-up. Each time a questionnaire is received, it is either handled automatically (green response, Figure 3) or included on a list where it remains until a clinician has accessed it and made a clinical decision regarding whether the patient should be contacted or not. Therefore, virtually no AmbuFlex questionnaires remain unnoticed.

‎View this figure

Discussion

During the 15-year period, we have experienced an exponential growth in the number of answered questionnaires and several marked changes have taken place. The creation of cohorts for clinical epidemiological research purposes has shifted towards cohorts defined by clinical use of PRO data at the patient level, and PRO-based follow-up (AmbuFlex) has replaced fixed appointments in many patient groups. Overall, the system is currently in use in 47 International Statistical Classification of Diseases and Related Health Problems (ICD)–groups, covering 16,000 patients and 94 departments throughout Denmark. In the first part of the period, a combination of paper- and web-based questionnaires secured response rates above 90% and low attrition rates. Back in 2005 66.5% questionnaires were paper based, but this is only the case for 4.3% in 2019. In AmbuFlex even higher response-rates have been reached, but the numbers are not directly comparable because each patient is individually referred and has given consent to participate. In addition, the approach methods for questionnaires and reminders have changed from letters, emails, and text messaging to a national, secure, electronic mail system through which 93.2% of communication to patients took place in 2019. The combination of secure email and web-based answering has resulted in a low turnaround time, where half of all responses are received within 5 days.

Our experiences probably reflect general changes that have taken place during the 15-year period. First, the demand for PRO measurements has increased, quantitatively as well as qualitatively. PRO measures have been used at the group level for many years, even before the PRO term was coined, when it was termed Quality of Life measures among other things (particularly in clinical trials and observational studies). PRO measures have also been used at the group level to provide evidence for drug and device approval and, in some countries, used to evaluate quality of care and health service provider performance. By contrast, using PRO data systematically at the individual patient level might be relatively new, but it is now by far our predominant activity. This increased demand is driven by a wish among patients, as well as clinicians, to use the potential in PRO data to promote better symptom assessment, more patient-centered care, and more efficient use of resources. In Denmark, hospital administrators and health ministerial civil servants quickly realized PRO measures’ potential to prioritize resources to the outpatients who actually want or need clinical attention.

Second, substantial technological changes have occurred in the period, which has created new opportunities. For example, an electronic health record (EHR) system has been implemented that covers the whole Region Central Denmark, and AmbuFlex obtained an early online connection that allowed clinicians to access it because it was a part of the EHR. In addition, the national secure electronic mail system (e-boks) has had a high impact on reducing response time.

Finally, several legal changes have occurred. The rationale for implementation of the General Data Protection Regulation (GDPR) is sound because there is a real risk of patients’ information being accessed and used by people for whom it was not initially intended. However, much time is used on details and bureaucratic documentation procedures, which doubly enhance data security. Another legal change is the European Union’s medical device regulation. Questionnaires with an attached algorithm, like those used in AmbuFlex, are classified as a medical device and as such they must be certified with a certification (Conformité Européenne) marking. Patient safety is a cornerstone, but since we are dealing with outpatients who are instructed to contact their family doctor or the department directly in case of exacerbation it makes little sense to treat a questionnaire with the same rules as electronic medical equipment. At present, AmbuFlex is granted a dispensation for current projects until the marking is in place, but not allowed to launch new projects. If AmbuFlex were to start today, it is unlikely that we would ever have surfaced.

One issue, which is often overlooked, is whether collected PRO data are used sufficiently [7,41]. Most patients are very careful when filling in their questionnaires, and it should be our obligation to promote as much use of the data as possible, irrespective of if data are originally collected for research, quality assessment, or clinical use at the patient level. In the future, clinical settings will probably be the most important source of PRO data, but also for other purposes like research and quality surveillance, so new ways to conduct complementary data collection will be necessary [7]. The current legal restrictions on the joint use of health data imposed by the GDPR make no distinction between use and misuse, and steps should be taken to alleviate these legal restrictions on the joint use of PRO data.