Electronic Implementation of Patient-Reported Outcome Measures in Primary Health Care: Mixed Methods Systematic Review

Introduction

The World Health Organization advocates for universal access to primary health care (PHC), a critical first point of contact within health care systems in the Organization for Economic Co-operation and Development countries, despite ongoing access limitations [1-4].

Chronic diseases, primarily managed in this setting, continue to be leading causes of mortality and morbidity in these countries [5,6]. The Canadian Institute of Health Research promotes the use of patient-reported outcome measures (PROMs) to enhance patient experience, clinical outcomes, and health care efficiency [7]. While PROMs have been widely implemented in hospital settings, their use in primary care remains underexplored. PROMs are assessments of a patient’s health status based on their own perceptions, without input from a third party. These reports are collected using validated questionnaires that quantify aspects such as quality of life, disease management, daily functioning, and symptoms [8]. For more than a decade, governments in several countries have funded initiatives to develop, implement, and use PROMs in hospitals [9] and PHC settings [10]. Optimal implementation and use of PROMs are associated with clinical benefits, such as improved communication with patients, better adaptation of health care to patient needs, and shorter consultation times [9].

The emergence of digital tools has a great potential for the implementation and use of PROMs in health care settings [11]. Digital methods, compared to traditional pen-and-paper approaches, enhance data collection quality, reduce costs, support clinical decision-making, and are better received by patients [12]. Several systematic reviews have identified the barriers and facilitators associated with their implementation in health care systems and evaluated the effectiveness of digitally collected electronic PROMs (ePROMs) [8,13,14]. In oncology, the main barriers to implementation are increased workload and inadequate technological infrastructure [8,14]. In terms of effects, these systematic reviews note that there is a fairly wide divergence between the results of the studies that have been identified. However, in oncology and pediatrics, the implementation of ePROMs is associated with an increase in quality of life and patient satisfaction [8,13]. While systematic reviews have identified impacts, barriers, and facilitators for ePROMs implementation in specialized settings, these findings have limited transferability to PHC due to differences in patient populations and clinical workflows; for example, the wider age range in primary care makes adapting ePROMs to varying levels of digital literacy more challenging. The implementation of ePROMs in PHC has however the potential to provide unique benefits, particularly by supporting self-management of chronic disease symptoms [15].

This review aims to evaluate the implementation and effectiveness of ePROMs for chronic disease management in PHC, identifying associated barriers and facilitators.

Methods

Ethical Considerations

We conducted a systematic review of the literature according to the Cochrane Methods Group and in compliance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for its reporting (Multimedia Appendix 1) [16,17]. We registered the study protocol with the PROSPERO Systematic Review Registry (ID: CRD42022333513) [18] and have also published it [19].

Synthesis Questions

The synthesis questions were (1) What are the effective strategies to implement ePROMs in PHC? (2) What are the challenges and barriers and facilitators to successful implementation of ePROMs in PHC? and (3) What are the outcomes of ePROMs in PHC chronic disease management?

Eligibility Criteria

We included all types of evidence matching the following PICOS (Population, Intervention, Comparison, Outcomes, and Setting) [16]:

• Population: All studies including the implementation of an ePROM among adults for chronic disease management
• Interventions or phenomena of interest: No restrictions. We included all types of implementations, theoretical models, structures, or PROMs.
• Comparator: No restrictions
• Outcomes: We considered all outcomes reported in the studies. We sought outcomes related to patients, caregivers, health care providers, policy makers, barriers, facilitators, acceptability, feasibility, adoption, fidelity, morbidity, mortality, quality of life, satisfaction, cost, and cost-effectiveness.
• Setting: We included studies conducted exclusively in PHC settings, regardless of location, and extracted information on implementation.

We included all types of empirical studies (qualitative, quantitative, and mixed methods) published in French, English, or Spanish.

Search Strategy

Using an iterative process, the search strategy was developed in collaboration with an experienced information specialist (FB). On August 15, 2022, we searched the following databases: MEDLINE (OVID), Embase (via Embase), CINAHL (EBSCO), and Web of Science. Considering the large number of results, we decided not to consult gray literature sources (eg, GreyNet, Grey Matters, Google, websites, and ProQuest Dissertations & Theses), as suggested in our protocol. We applied no restrictions to the search strategy, including time limit, as mentioned in our protocol due to the lack of effect in reducing the number of results. The full search strategy is available in Multimedia Appendix 2.

Data Collection and Screening

We exported all citations in the web-based collaboration tool Covidence, where duplicates were removed with the automated function [20]. Pairs of reviewers screened the titles and abstracts independently. We retained ambiguous or incomplete abstracts to be reviewed in full. We searched and obtained all the full texts of the selected references and imported the PDF files in Covidence. Pairs of reviewers then independently applied the inclusion criteria using the full texts following a pilot testing using the process outlined above. At any moment in the screening process, the first author helped resolve any discrepancy. All the reasons for exclusions were recorded in Covidence, and a PRISMA flowchart describes study identification, screening, inclusions, and exclusions (Figure 1) [21].

‎

Data Extraction and Appraisal

We extracted descriptive data (title, year of publication, authors, funding, conflicts of interests, and country), study types (published or gray literature), methodological data (design, sample size, measure constructs, and name of the instrument), setting data (clinical setting, type health professionals, and patient population), characteristics of the ePROM tools (functionality, interface and delivery platform, and integration approach), implementation data (description of implementation strategies, facilitators, and barriers), outcomes (patient health, providers workflow, and cost), and outcomes type (qualitative and quantitative). The quality of the included studies was evaluated using the Mixed Methods Appraisal Tool (MMAT), which is designed for systematic reviews that synthesize data from qualitative, quantitative, and mixed methods studies [22].

Data Synthesis

We used the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework as a data analysis framework. RE-AIM has been developed to evaluate the public health impacts of interventions and has been used in systematic reviews to help structure the assessment of the different implementation factors at play in complex contexts and settings [23,24]. This framework includes 5 dimensions: reach (how willing the targeted population is to participate in the intervention; ie, ePROMs), efficacy (what is the impact of the intervention on outcomes), adoption (can this be adopted by new groups with ease and minimal changes), implementation (what are the special issues and barriers), and maintenance (can the intervention be maintained and the impact continued). The use of RE-AIM enabled us to give an overview of the parameters strengthening (review questions 1 and 2) the efficiency (review question 3) of ePROMs’ integration in PHC and its impact on outcomes.

We used a 2-phase sequential mixed methods synthesis design, that is, conduct a qualitative synthesis and use its results to inform the quantitative synthesis [25]. For phase 1 (qualitative), we summarized and described methods and approaches designed to implement and integrate ePROMs in PHC, using a thematic synthesis procedure [26]. The qualitative data synthesis produced narrative summaries of main themes, which were then classified according to the RE-AIM framework. We summarized study characteristics and methodological differences and similarities to highlight the following points: strengths and weaknesses of each implementation method, main outcomes of implementation, main resources used and their impacts, and if any trade-offs are described and their effect on the results of the study.

Results

Overview

The PRISMA diagram (Figure 1) shows the results of the search strategy and study selection process. Our search strategy identified 12,525 references. The exclusion of duplicates (n=3998) and the first selection stage (titles and abstracts) led us to retain 761 references. A further 749 references were excluded when the texts were read in full. In all, 22 studies met the selection criteria and were therefore selected.

Study Characteristics

Table 1 summarizes descriptive data from the 22 selected studies; of these, 5 (23%) are mixed-design studies, 6 (27%) are qualitative studies, and 11 (50%) are quantitative studies. These 22 studies were published between 2015 and 2022, and the majority were conducted in the United States (n=9, 41%) and Canada (n=8, 36%). The number of participants included in these studies ranged between 2334 and 8, and only 3 studies had >300 participants. The aims of the studies selected align around improving clinical outcomes for patients with chronic diseases and enhancing health care delivery through better integration of patient-reported data into clinical practice. In most studies, the average age of the population was >50 years. Gender representation varied from study to study, but in the majority (16/22, 73%), women represented >50% of participants. Additional information about the intervention are presented in Multimedia Appendix 3.

The main chronic diseases population targeted are asthma (4/22, 18%), cardiovascular diseases (3/22, 14%), multimorbidity (5/22, 23%), mental health (3/22, 14%), diabetes (2/22, 9%), or complex cases of chronic disease (2/22, 9%). Regarding ePROMs, the digital tool most frequently implemented was a mobile app (9/22, 41%). The constructs of ePROMs varied between 3 main categories: general health (eg, quality of life and health status), symptom monitoring (eg, for sleep, pain, anxiety, and depression), and self-management (eg, self-efficacy and patient activation).

Assessment of Studies’ Methodological Quality

The evaluation of the studies’ methodological quality shows that the majority (15/22, 68%) received a score of ≥80%. Specifically, 38% (8/22) of the studies scored 100%, 33% (7/22) of the studies scored between 80% and 95%, and 19% (4/22) of the studies scored between 60% and 75%. Only 2 studies scored ≤40% (Table 2). While most of the included studies (15/22, 68%) met a high standard of methodological rigor (≥80% on the MMAT), this does not entirely eliminate concerns about heterogeneity, smaller sample sizes, and diverse outcome measures. Higher-quality studies generally provided robust justifications for their choice of ePROM tools and specified clear implementation processes (eg, training sessions and technical support). In contrast, studies with lower MMAT scores often lacked detail on how they integrated ePROMs into existing workflows or offered minimal information on training protocols. These distinctions may influence both the internal validity and generalizability of the findings.

Qualitative Results

Overview

We present the entire thematic organization and representatives quotes of the qualitative studies structured according to the RE-AIM model in Table 3.

Digital Literacy and Training

The data highlight challenges with the lack of digital literacy among patients and providers, alongside insufficient training for providers [38,43]. These issues affected various aspects of the RE-AIM framework, notably impacting reach, adoption, and implementation. Addressing digital literacy through targeted training sessions for both patients and providers emerges as a facilitator by enhancing the usability and acceptance of ePROMs tools [39,41].

Patient-Provider Communication and Relationship

The dynamics of communication and the relationship between patients and providers are identified as both barriers and facilitators [41,45]. The lack of patient-provider interaction and inadequate feedback are significant barriers, while strong relationships, interactivity, and timely communication facilitate implementation [41].

Personalization and Integration Into Clinical Workflow

The necessity for ePROMs tools to be personalized and seamlessly integrated into clinical workflows is a common theme across barriers and facilitators. Studies showed the reluctance to adopt new practices due to personalization and integration issues [38,41], contrasting with facilitators that advocate for straightforward data presentation and interoperability [42].

Technical Challenges and Support

Studies identified technical difficulties, including data entry, information retrieval, and a lack of technical support, as significant barriers [39]. Facilitators highlighted in the studies were ongoing technical support and user-friendly interfaces [47].

Workload and Treatment Quality

Concerns related to the impact of ePROMs on workload and treatment quality are highlighted barriers [40,43]. The increased workload for providers and potential distractions from acute health issues pose major barriers, while the potential for improved treatment quality through enhanced patient contexts offers a significant advantage [45].

Quantitative Results

Overview

We present an overview of all quantitative results in Table 4.

Clinical Outcomes and Quality of Life

A tablet-based electronic screening tool significantly improved the identification of behavioral health problems, offering an efficient approach to behavioral health screening and follow-up health care [27]. Overall, 3 studies showed no improvement on symptoms [29,34,35,43], while another study indicated a positive association [32]. The use of ePROMs in asthma management showed improvements within group in asthma-related quality of life and symptom control but were not statistically significant between groups [15]. The adoption of an ePROMs tool did not demonstrate statistical differences in overall or subscale scores of quality of life, patient activation, or satisfaction with chronic pain health care across both control and intervention groups [29,43,45,47]. Quality of life was used as the dependent variable by 5 studies, 3 showed no improvements [30,35,45], 1 showed improvement [15], and 1 showed decline [31].

Self-Management and Health Behaviors

Some studies showed significant improvement in self-management [37], skill and technique acquisition, and social integration [30], as well as adherence to diet recommendations and mental health outcomes over time [34]. Two studies showed no improvements in patient activation [45] and self-management [33].

System-Level Outcomes

Two studies indicated that implementing ePROMs does not reduce rehospitalizations [30,37]. Moreover, compared to usual care, the implementation of ePROMs is associated with a drop in cost-effectiveness because the data collection targets all patients and not specific cases. In addition, differences in patients’ digital literacy may also explain this drop in cost-effectiveness [31]. The study by Staeheli et al [27] showed an improvement in follow-up care.

Feasibility and Acceptability

Four studies indicated that ePROMs’ implementation is feasible and achieves a good level of acceptability [15,32,33,46], while 2 studies showed no association with satisfaction [28,29]. One study focusing on mental health outcomes also highlighted significant feasibility and acceptability as well as high engagement [46]. However, it also reported a steep attrition in long-term data collection [46]. One study implementing an audio computer-assisted self-interview system showed minimal time burden and high feasibility for patients regardless of age, language, and computer literacy [36].

Discussion

Principal Findings

It is difficult to draw clear conclusions about the effects of ePROM implementation in PHC. However, combining insights from both qualitative and quantitative data on the implementation and effects of ePROMs yields a nuanced understanding of their potential and challenges. A lack of digital literacy and engagement seems to be a key barrier to effectiveness, and in quantitative studies, ePROMs were well received when researchers emphasized feasibility and acceptability and provided training and support. This suggests that improving digital literacy and ensuring user-friendly design and adequate support are crucial for enhancing patient engagement with ePROMs and their effectiveness. Qualitative findings highlighted that the lack of personalization and patient-provider communication were impactful. Quantitatively, interventions that allowed for personalized feedback, goal setting, and self-management support showed positive effects. This indicates that ePROMs that are personalized and facilitate or enhance communication between patients and providers can lead to better health outcomes and patient experiences.

Qualitative data provided insights into perceived benefits, such as increased confidence and motivation and improved person-centered health care, even when quantitative outcomes showed null effects regarding clinical metrics such as health status and quality of life. This suggests a complex relationship between perceived benefits and measurable health outcomes, indicating that ePROMs may impact aspects of health care and patient experience not fully captured by quantitative measures. It also underscores the nuanced effects of digital and behavioral interventions on self-management and behavioral outcomes, highlighting the importance of personalized, interactive approaches and the potential for digital platforms to support but not fully substitute for comprehensive health care strategies aimed at enhancing patient engagement and self-management capabilities [30,33,34,45]. Qualitative insights suggest that despite the enthusiasm for ePROMs and digital interventions, there are concerns about their long-term impact and cost-effectiveness, with quantitative data echoing these concerns. While qualitative data highlight benefits such as increased confidence and motivation and improved patient-centered care, these effects do not always translate into measurable clinical outcomes. This highlights the need for more comprehensive research into the long-term outcomes, cost-effectiveness, and broader societal utility of digital health interventions.

Our review found that most studies were conducted in the United States (9/22, 41%) and Canada (8/22, 36%). The relative emphasis on these regions means that the findings may be influenced by North American health care financing models and policy environments. For instance, in the United States, fragmented insurance coverage and variable reimbursement structures can influence the adoption of digital health tools. In Canada, publicly funded health care and provincial eHealth strategies might support ePROM implementation by offering centralized funding or infrastructure but can also slow adoption. Other high-income countries may have distinct funding mechanisms, regulatory requirements, or national digital health agendas that shape ePROM uptake differently. With most included studies conducted in North America, our results have limited generalizability. Health care systems worldwide vary in digital infrastructure, reimbursement models, and regulatory frameworks, influencing ePROMs adoption. Future research should explore cross-cultural differences and assess implementation in diverse settings to better inform global policy and practice.

The results of this study corroborate those of other systematic reviews. First, systematic reviews carried out in pediatric health care [13], oncology [48,49], or breast cancer treatment [14] also present divergent results about patient satisfaction, quality of health care, health outcomes, patient management, and patient health behavior. The authors of these studies conclude that differences in the context of health care and the quality of ePROM implementation make it difficult to generalize the results of the studies that have been identified. Ishaque et al [48] also associates this divergence to the low statistical power of most of the studies reviewed. Second, the authors also conclude that ePROM implementations have the potential to improve patient health, provided they consider the barriers and facilitators specific to each health care setting [14,48,50].

Some of our results also differ from those of systematic reviews that have looked at other health care contexts. Indeed, systematic reviews indicate a positive, clear, and well-supported relationship between the implementation of ePROMs and health benefits. For example, in pediatrics, the integration of ePROMs is associated with an increase in health-related quality of life and patient satisfaction [13], while in oncology, the detection of unidentified problems and the monitoring of treatment response are improved by the implementation of ePROMs [49]. This difference in results from our study can probably be explained by the greater heterogeneity of populations and treatments in PHC, pointing at the significant challenge to widespread ePROMs implementation and uptake in this setting.

Limitations

Our study has several limitations. The heterogeneity of study designs and outcome measures precluded meta-analysis, necessitating a narrative synthesis. The quantitative studies included in our sample assessed a diverse range of health indexes using scales that were not based on a common metric, making it impossible to extract standardized results for quantification. Consequently, we interpreted their findings using the vote-counting method [51]. However, the vote-counting method has been criticized [51], particularly because it does not account for effect sizes, limiting comparability and resulting in lower precision than meta-analysis. Our qualitative data analysis was limited to a descriptive approach. To accurately capture the net effect of ePROM implementation, future studies should aim to minimize the influence of barriers while maximizing the impact of facilitators. To advance this understanding, we plan to conduct a meta-synthesis of the identified qualitative studies, offering a comprehensive analysis of barriers, facilitators, and their interrelationships.

In terms of public decision-making, we can question the real size of the social benefits of implementing ePROMs. Indeed, only 1 of the studies in our sample carried out a cost-benefit analysis [31], and none of them did so in an ecological manner. No study has compared the benefits of implementing ePROMs to those of another intervention with the same level of financial investment. Future research should conduct cost-benefit analyses in a real-world context to identify which sector would yield the greatest overall benefit from financial investment.

Success in implementing ePROMs in PHC appears to hinge on addressing digital literacy, ensuring personalization and meaningful patient-provider interactions, carefully integrating technology into clinical workflows, and conducting thorough research on their long-term impacts and cost-effectiveness. ePROM implementation research could conduct large‐scale, multisite randomized trials or pragmatic trials to compare ePROM implementation strategies; investigate long-term outcomes and sustainability (eg, cost‐utility analyses and ecological cost‐benefit comparisons); focus on diverse patient populations, especially underserved communities, to address health equity and digital literacy gaps; and use standardized outcome measures or validated ePROM instruments to facilitate cross‐study comparisons. By focusing on these areas, future efforts could better assess the benefits of digital health technologies for patients, providers, and health care systems.