Review
Abstract
Background: Managing chronic diseases remains a critical challenge in primary health care (PHC) across the Organization for Economic Co-operation and Development countries. Electronic patient-reported outcome measures (ePROMs) are emerging as valuable tools for enhancing patient engagement, facilitating clinical decision-making, and improving health outcomes. However, their implementation in PHC remains limited, with significant variability in effectiveness and adoption.
Objective: This systematic review aimed to assess the implementation and effectiveness of ePROMs in chronic disease management within PHC settings and to identify key barriers and facilitators influencing their integration.
Methods: A mixed methods systematic review was conducted following the Cochrane Methods and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We included studies that implemented ePROMs among adults for chronic disease management in PHC. The extracted data included patient health outcomes, provider workflow implications, implementation factors, and cost considerations. The reach, effectiveness, adoption, implementation, and maintenance framework was used.
Results: Our search yielded 12,525 references, from which 22 (0.18%) studies were included after screening and exclusions. These studies, primarily conducted in the United States (n=9, 41%) and Canada (n=8, 36%), covered various chronic diseases and used diverse ePROM tools, predominantly mobile apps (n=9, 41%). While some studies (n=10, 45%) reported improvements in patient health outcomes and self-management, others (n=12, 55%) indicated no significant change. Key barriers included digital literacy gaps, integration challenges within clinical workflows, and increased provider workload. Facilitators included strong patient-provider relationships, personalized interventions, and technical support for users. While some studies (n=10, 45%) demonstrated improved patient engagement and self-management, long-term cost-effectiveness and sustainability remain uncertain.
Conclusions: Success in implementing ePROMs in PHC appears to hinge on addressing digital literacy, ensuring personalization and meaningful patient-provider interactions, carefully integrating technology into clinical workflows, and conducting thorough research on their long-term impacts and cost-effectiveness. Future efforts should focus on these areas to fully realize the benefits of digital health technologies for patients, providers, and health care systems.
Trial Registration: PROSPERO CRD42022333513; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022333513
International Registered Report Identifier (IRRID): RR2-10.2196/48155
doi:10.2196/63639
Keywords
Introduction
The World Health Organization advocates for universal access to primary health care (PHC), a critical first point of contact within health care systems in the Organization for Economic Co-operation and Development countries, despite ongoing access limitations [-].
Chronic diseases, primarily managed in this setting, continue to be leading causes of mortality and morbidity in these countries [,]. The Canadian Institute of Health Research promotes the use of patient-reported outcome measures (PROMs) to enhance patient experience, clinical outcomes, and health care efficiency []. While PROMs have been widely implemented in hospital settings, their use in primary care remains underexplored. PROMs are assessments of a patient’s health status based on their own perceptions, without input from a third party. These reports are collected using validated questionnaires that quantify aspects such as quality of life, disease management, daily functioning, and symptoms []. For more than a decade, governments in several countries have funded initiatives to develop, implement, and use PROMs in hospitals [] and PHC settings []. Optimal implementation and use of PROMs are associated with clinical benefits, such as improved communication with patients, better adaptation of health care to patient needs, and shorter consultation times [].
The emergence of digital tools has a great potential for the implementation and use of PROMs in health care settings []. Digital methods, compared to traditional pen-and-paper approaches, enhance data collection quality, reduce costs, support clinical decision-making, and are better received by patients []. Several systematic reviews have identified the barriers and facilitators associated with their implementation in health care systems and evaluated the effectiveness of digitally collected electronic PROMs (ePROMs) [,,]. In oncology, the main barriers to implementation are increased workload and inadequate technological infrastructure [,]. In terms of effects, these systematic reviews note that there is a fairly wide divergence between the results of the studies that have been identified. However, in oncology and pediatrics, the implementation of ePROMs is associated with an increase in quality of life and patient satisfaction [,]. While systematic reviews have identified impacts, barriers, and facilitators for ePROMs implementation in specialized settings, these findings have limited transferability to PHC due to differences in patient populations and clinical workflows; for example, the wider age range in primary care makes adapting ePROMs to varying levels of digital literacy more challenging. The implementation of ePROMs in PHC has however the potential to provide unique benefits, particularly by supporting self-management of chronic disease symptoms [].
This review aims to evaluate the implementation and effectiveness of ePROMs for chronic disease management in PHC, identifying associated barriers and facilitators.
Methods
Ethical Considerations
We conducted a systematic review of the literature according to the Cochrane Methods Group and in compliance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for its reporting () [,]. We registered the study protocol with the PROSPERO Systematic Review Registry (ID: CRD42022333513) [] and have also published it [].
Synthesis Questions
The synthesis questions were (1) What are the effective strategies to implement ePROMs in PHC? (2) What are the challenges and barriers and facilitators to successful implementation of ePROMs in PHC? and (3) What are the outcomes of ePROMs in PHC chronic disease management?
Eligibility Criteria
We included all types of evidence matching the following PICOS (Population, Intervention, Comparison, Outcomes, and Setting) []:
- Population: All studies including the implementation of an ePROM among adults for chronic disease management
- Interventions or phenomena of interest: No restrictions. We included all types of implementations, theoretical models, structures, or PROMs.
- Comparator: No restrictions
- Outcomes: We considered all outcomes reported in the studies. We sought outcomes related to patients, caregivers, health care providers, policy makers, barriers, facilitators, acceptability, feasibility, adoption, fidelity, morbidity, mortality, quality of life, satisfaction, cost, and cost-effectiveness.
- Setting: We included studies conducted exclusively in PHC settings, regardless of location, and extracted information on implementation.
We included all types of empirical studies (qualitative, quantitative, and mixed methods) published in French, English, or Spanish.
Search Strategy
Using an iterative process, the search strategy was developed in collaboration with an experienced information specialist (FB). On August 15, 2022, we searched the following databases: MEDLINE (OVID), Embase (via Embase), CINAHL (EBSCO), and Web of Science. Considering the large number of results, we decided not to consult gray literature sources (eg, GreyNet, Grey Matters, Google, websites, and ProQuest Dissertations & Theses), as suggested in our protocol. We applied no restrictions to the search strategy, including time limit, as mentioned in our protocol due to the lack of effect in reducing the number of results. The full search strategy is available in .
Data Collection and Screening
We exported all citations in the web-based collaboration tool Covidence, where duplicates were removed with the automated function []. Pairs of reviewers screened the titles and abstracts independently. We retained ambiguous or incomplete abstracts to be reviewed in full. We searched and obtained all the full texts of the selected references and imported the PDF files in Covidence. Pairs of reviewers then independently applied the inclusion criteria using the full texts following a pilot testing using the process outlined above. At any moment in the screening process, the first author helped resolve any discrepancy. All the reasons for exclusions were recorded in Covidence, and a PRISMA flowchart describes study identification, screening, inclusions, and exclusions (
Figure 1) [].
Data Extraction and Appraisal
We extracted descriptive data (title, year of publication, authors, funding, conflicts of interests, and country), study types (published or gray literature), methodological data (design, sample size, measure constructs, and name of the instrument), setting data (clinical setting, type health professionals, and patient population), characteristics of the ePROM tools (functionality, interface and delivery platform, and integration approach), implementation data (description of implementation strategies, facilitators, and barriers), outcomes (patient health, providers workflow, and cost), and outcomes type (qualitative and quantitative). The quality of the included studies was evaluated using the Mixed Methods Appraisal Tool (MMAT), which is designed for systematic reviews that synthesize data from qualitative, quantitative, and mixed methods studies [].
Data Synthesis
We used the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework as a data analysis framework. RE-AIM has been developed to evaluate the public health impacts of interventions and has been used in systematic reviews to help structure the assessment of the different implementation factors at play in complex contexts and settings [,]. This framework includes 5 dimensions: reach (how willing the targeted population is to participate in the intervention; ie, ePROMs), efficacy (what is the impact of the intervention on outcomes), adoption (can this be adopted by new groups with ease and minimal changes), implementation (what are the special issues and barriers), and maintenance (can the intervention be maintained and the impact continued). The use of RE-AIM enabled us to give an overview of the parameters strengthening (review questions 1 and 2) the efficiency (review question 3) of ePROMs’ integration in PHC and its impact on outcomes.
We used a 2-phase sequential mixed methods synthesis design, that is, conduct a qualitative synthesis and use its results to inform the quantitative synthesis []. For phase 1 (qualitative), we summarized and described methods and approaches designed to implement and integrate ePROMs in PHC, using a thematic synthesis procedure []. The qualitative data synthesis produced narrative summaries of main themes, which were then classified according to the RE-AIM framework. We summarized study characteristics and methodological differences and similarities to highlight the following points: strengths and weaknesses of each implementation method, main outcomes of implementation, main resources used and their impacts, and if any trade-offs are described and their effect on the results of the study.
Results
Overview
The PRISMA diagram (Figure 1) shows the results of the search strategy and study selection process. Our search strategy identified 12,525 references. The exclusion of duplicates (n=3998) and the first selection stage (titles and abstracts) led us to retain 761 references. A further 749 references were excluded when the texts were read in full. In all, 22 studies met the selection criteria and were therefore selected.
Study Characteristics
Table 1 summarizes descriptive data from the 22 selected studies; of these, 5 (23%) are mixed-design studies, 6 (27%) are qualitative studies, and 11 (50%) are quantitative studies. These 22 studies were published between 2015 and 2022, and the majority were conducted in the United States (n=9, 41%) and Canada (n=8, 36%). The number of participants included in these studies ranged between 2334 and 8, and only 3 studies had >300 participants. The aims of the studies selected align around improving clinical outcomes for patients with chronic diseases and enhancing health care delivery through better integration of patient-reported data into clinical practice. In most studies, the average age of the population was >50 years. Gender representation varied from study to study, but in the majority (16/22, 73%), women represented >50% of participants. Additional information about the intervention are presented in .
The main chronic diseases population targeted are asthma (4/22, 18%), cardiovascular diseases (3/22, 14%), multimorbidity (5/22, 23%), mental health (3/22, 14%), diabetes (2/22, 9%), or complex cases of chronic disease (2/22, 9%). Regarding ePROMs, the digital tool most frequently implemented was a mobile app (9/22, 41%). The constructs of ePROMs varied between 3 main categories: general health (eg, quality of life and health status), symptom monitoring (eg, for sleep, pain, anxiety, and depression), and self-management (eg, self-efficacy and patient activation).
| Study | Country | Aim of the study | Age (y), mean | Women (%) | Targeted Constructs | Chronic diseases | Sample size, n | Study designa |
| Staeheli et al [] | United States | Compare screening results to data derived from chart reviews of patients seen before the deployment of the screening intervention to determine the following:
| Not specified but around 50 | 66.54 | Depression, PTSD, and problem drinking | PTSD, depression, and risky drinking | 275 | Quantitative |
| Ainsworth et al [] | United Kingdom |
| 56.6 | 53 | Quality of life and self-monitoring | Physician-diagnosed asthma | 88 | Quantitative |
| Harle et al [] | United States |
| 43.8 | 62 | Pain interference, pain behavior, fatigue, and anger | Chronic musculoskeletal | 680 | Quantitative |
| Kroenke et al [] | United States |
| 49 | 72 | SPADEe | SPADE symptom | 256 | Quantitative |
| Lear et al [] | Canada |
| 70.5 | 38.40 | Self-monitoring and self-management | ≥2 chronic conditions | 229 | Quantitative |
| Miranda et al [] | Canada |
| Not specified but they recruiter patients ≥60 years old | Not specified | Global health | ≥2 chronic conditions | 45 | Quantitative |
| Baron and Duffecy [] | United States |
| 45.8 | 50 | Sleep quality | 24 h ambulatory blood pressure | 16 | Quantitative |
| Owen-Smith et al [] | United States |
| Not specified | Not specified | Pain | Chronic pain | 632 | Quantitative |
| Ramallo-Fariña et al [] | Spain |
| 55.7 | 51.90 | Diabetes empowerment | Type 2 diabetes mellitus at least 1 y before | 2334 | Quantitative |
| Tamisier et al [] | France |
| 50.6 | 63.59 | Quality of life | OSA | 206 | Quantitative |
| Trick et al [] | United States |
| 57.0 | 58 | General health | Chronic illness | 1670 | Quantitative |
| Yanicelli et al [] | Argentina |
| 52 | 20 | Weight, blood pressure, heart rate, and symptoms | HF | 30 | Quantitative |
| Bezerra Giordan et al [] | Australia |
| 69 | 66.66 | Self-management | Confirmed diagnosis of HF | 12 | Qualitative |
| Steele Gray et al [] | Canada |
| 58 | 54.54 | General Health Scale and pain | ≥2 chronic conditions | 14 | Qualitative |
| Hans et al [] | Canada |
| 56.3 | 50 | Global health and pain interference | ≥2 chronic conditions | 18 | Qualitative |
| Harle et al [] | United States |
| Not specified | Not specified | Pain interference, pain behavior, fatigue, and physical function | Chronic pain | 12 | Qualitative |
| Irfan Khan et al [] | Canada |
| 58 | 50 | Global health, pain interference, and generalized anxiety | Multimorbid | 9 | Qualitative |
| Schoenthaler et al [] | United States |
| 62.5 | 67 | Diabetes self-management | T2D for ≥6 mo | 12 | Qualitative |
| Steele Gray et al [] | Canada |
| Not clear but around 63 | 68.75 | Goal attainment | Not specified | 16 | Mixed methods |
| Ahmed et al, 2021 [] | Canada |
| 39 | 39 | Not specified | LBP | 18 | Mixed methods |
| Steele Gray et al [] | Canada |
| 68.7 | 65.22 | Goal setting, self-management, mental health, and social health | Older adults with complex needs | 44 | Mixed methods |
| Bauer et al [] | United States |
| Not specified but around 35 | 59 | Depressive and anxiety symptoms | Depression or anxiety disorder | 17 | Mixed methods |
aQualitative: studies primarily collecting and analyzing nonnumerical data (eg, interviews and focus groups), quantitative: studies primarily collecting and analyzing numerical data (eg, randomized controlled trials and cohort and cross‐sectional studies), and mixed methods: studies integrating both quantitative and qualitative approaches. We also reference these criteria in the Data Extraction and Appraisal section for clarity.
bPTSD: posttraumatic stress disorder.
cPRO: patient-reported outcome.
dEHR: electronic health record.
eSPADE: sleep problems, pain, anxiety, depression, and low energy or fatigue.
fePRO: electronic patient-reported outcome.
gPROM: patient-reported outcome measure.
hT2DM: type 2 diabetes mellitus.
iOSA: obstructive sleep apnea.
jHF: heart failure.
kLBP: lower back pain.
Assessment of Studies’ Methodological Quality
The evaluation of the studies’ methodological quality shows that the majority (15/22, 68%) received a score of ≥80%. Specifically, 38% (8/22) of the studies scored 100%, 33% (7/22) of the studies scored between 80% and 95%, and 19% (4/22) of the studies scored between 60% and 75%. Only 2 studies scored ≤40% (Table 2). While most of the included studies (15/22, 68%) met a high standard of methodological rigor (≥80% on the MMAT), this does not entirely eliminate concerns about heterogeneity, smaller sample sizes, and diverse outcome measures. Higher-quality studies generally provided robust justifications for their choice of ePROM tools and specified clear implementation processes (eg, training sessions and technical support). In contrast, studies with lower MMAT scores often lacked detail on how they integrated ePROMs into existing workflows or offered minimal information on training protocols. These distinctions may influence both the internal validity and generalizability of the findings.
| Study | MMAT items | ||||||||||||||||||||||||||||||||||
| 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 2.1 | 2.2 | 2.3 | 2.4 | 2.5 | 3.1 | 3.2 | 3.3 | 3.4 | 3.5 | 5.1 | 5.2 | 5.3 | 5.4 | 5.5 | ||||||||||||||||
| Staeheli et al [] | —a | — | — | — | — | — | — | — | — | — | 1b | 1 | 1 | 0c | 1 | — | — | — | — | — | |||||||||||||||
| Bauer et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | |||||||||||||||
| Trick et al [] | — | — | — | — | — | — | — | — | — | — | 1 | 1 | 0 | 1 | 1 | — | — | — | — | — | |||||||||||||||
| Owen-Smith et al [] | — | — | — | — | — | — | — | — | — | — | 0 | 1 | 1 | 0 | 1 | — | — | — | — | — | |||||||||||||||
| Steele Gray et al [] | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | — | — | — | — | — | 1 | 1 | 1 | 0 | 0 | |||||||||||||||
| Bezerra Giordan et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Schoenthaler et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Ahmed et al [] | 1 | 1 | 0 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Kroenke et al [] | — | — | — | — | — | 1 | 1 | 1 | 0 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Lear et al [] | — | — | — | — | — | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Baron et al [] | — | — | — | — | — | 0 | 0 | 0 | 0 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Ramallo-Fariña et al [] | — | — | — | — | — | 1 | 1 | 1 | 0 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Tamisier et al [] | — | — | — | — | — | 1 | 0 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Yanicelli et al [] | — | — | — | — | — | 1 | 0 | 0 | 0 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Ainsworth et al [], 2019 | — | — | — | — | — | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Steele Gray et al [] | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | 1 | 1 | 1 | 0 | 1 | |||||||||||||||
| Steele Gray et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Hans et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Harle et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Harle et al [] | — | — | — | — | — | 1 | 0 | 1 | 0 | 1 | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
| Irfan Khan et al [] | 1 | 1 | 1 | 1 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |||||||||||||||
aNot applicable.
bThis criterion was met.
cThis criterion was not met.
Qualitative Results
Overview
We present the entire thematic organization and representatives quotes of the qualitative studies structured according to the RE-AIM model in Table 3.
| Study | Reach, effectiveness, adoption, implementation, and maintenance | Actor | Categories | Themes | Representative quotes | |
| Barriers | ||||||
| Steele Gray et al [] | Reach | Patient | Communication | Lack of patient-provider interaction | “However, some patients reported feeling isolated with the mobile device, and felt that the tool could become a replacement for in person consultation.” | |
| Bezerra Giordan et al [] | Reach | Patients, clinicians, and organizations | Digital literacy | Lack of digital literacy of patients and providers’ lack of training | “Patients mentioned seeing themselves as not being tech-savvy enough to use an app [mostly due to a perceived age barrier] and expected it would take them a long time and effort to learn how to use it properly.” | |
| Irfan Khan et al [] | Efficacy | Patient | Communication | Lack of feedback from providers | “Patients identified gaps in the tool’s ability to promote self-efficacy in terms of the adherence to self-regulation activities because of limited feedback on their progress from providers.” | |
| Harle et al [] | Efficacy | Providers | Treatment | Take time away from treatment | “Discussing these issues (psychological data on depression and anxiety) could harm care quality by diverting their attention away from acute problems that they judged to be more relevant at a given visit or more aligned with their clinical expertise.” | |
| Steele Gray et al [] | Efficacy | Providers | Workload | Increase workload | “Finally, some providers felt that they may be liable for monitoring patients during out-of-office hours, which would require additional time and resources.” | |
| Irfan Khan et al [] | Adoption | Patients, clinicians, and organizations | Adaptation | Reticence to adopt new practices, lack of personalization, and lack of alignment with clinical reality | “In addition, patients stressed the need for greater personalization and customizability of goals and monitoring protocols. The questions in the ePRO tool appeared to lack the depth that was considered vital to incorporating patient context into self-management activities.” | |
| Schoenthaler et al [] | Implementation | Patient | ePROMa digital solution | Confidentiality, difficult data entry, difficult use, retrieval of information, data presentation, technical issues, and no technical support | “Several patients had difficulty reading the bar graphs of PROs that were collected biweekly (eg, quality of life) and recommended changing the items to weekly measures to be consistent with other PROs.” | |
| Steele Gray et al [] | Implementation | Patients, clinicians, and organizations | Workflow | Workflow integration and data overload | “After the initial visit, provider participants reported experiencing difficulty incorporating patient data into their workflow in terms of: (1) increased charting time required to input data into the provider’s EMR and (2) being able to view data in manageable chunks.” | |
| Bauer et al [] | Maintenance | Patient | Adaptation | Lack of personalization of system features and sensitivity to change of PROMb | “Patients’ desire for more personalized features. Patients wanted to customize the app to meet their individual needs, for example, by adjusting the timing of prompts, the types of symptoms they were reporting on, or the frequency or content of health tips.” | |
| Irfan Khan et al [] | Maintenance | Patient | Relationship | Lack of patient-provider interaction | “Patient expectations around social facilitation and social support indicated that they were expecting a more active role in self-management efforts from providers. Patients felt the tool should supplement patient-provider interaction through regular feedback and encouragement as an ‘add-on’ to existing in-person appointments rather than a replacement for in-person interaction and consults with their providers.” | |
| Bezerra Giordan et al [] | Maintenance | Patients, clinicians, and organizations | Burden of treatment | Increased treatment burden | “Participants mentioned the use of an app could be associated with an increase in the burden of managing heart failure. learning how to use the app and remembering to use it could be seen as an additional responsibility in people’s already busy lives, which could be demotivating and lead to a lack of interest and decreased willingness to use the app over time.” | |
| Hans et al [] | Maintenance | Providers | Workload | Increase workload | “Providers questioned whether the app would actually improve workflow functions or simply add another task. Multiple providers expressed their concerns with incorporating the ePRO into their daily visit routine.” | |
| Facilitators | ||||||
| Steele Gray et al [] | Reach | Providers and patient | Relationship | Existing patient-provider relationship | “The meaningfulness of the ePRO tool was reliant to strong relationships between patients and providers (enabling collective action)” | |
| Irfan Khan et al [] | Reach | All participants | Digital literacy | Digital training and digital literacy | “The technology partner (QoC Health) offered providers two 1-hour hands-on training sessions (facilitated by the research team) on the mobile phone app and portal to provide a walk-through of the ePRO tool before starting the study, whereas, patients received one-on-one training through a 30-minute hands-on session with a member of the research team at the time when patients gave consent to participate in this study.” | |
| Ahmed et al [] | Reach | Providers | ePROM literacy | Score interpretation | “Participants agreed that having the knowledge and skills to interpret PROM scores is required for clinicians to be able to use PROMs in clinical care for management of LBP.” | |
| Irfan Khan et al [] | Efficacy | Patient and providers | Self-management | Improved self-management, self-efficacy, symptoms management, goal setting, and treatment quality | “Patients acknowledged the potential of the ePRO tool in building capacity to support self-management in a team-based care environment by helping to better distribute the workload across providers to meet the evolving needs of patients.” | |
| Bezerra Giordan et al [] | Efficacy | Patients, providers, and organizations | Communication | Interactivity, timely action, team problem-solving, goal setting, patient-provider communication, and patient-oriented treatment | “Participants mentioned that the ability for both patients and clinicians to monitor signs of deterioration allowed for timely action.” | |
| Schoenthaler et al [] | Efficacy | Providers | ePROM digital solution | Data presentation | “Providers want PRO data that are specific and actionable and can help them focus the clinic visit on what is most important for their T2D patients’ care.” | |
| Irfan Khan et al [] | Efficacy | Providers | Treatment | Treatment quality | “Providers emphasized the value of the ePRO tool in helping to generate insights into underlying patient context (ie, patient preferences and readiness) to offer a fulsome sense of how patients are coping, and thereby adjust goals and self-management activities as needed.” | |
| Hans et al [] | Efficacy | Providers | Workload | Reduced resource use and time saved in data retrieval during encounters | “Providers reported that the app presented an additional resource that they could leverage to quickly orient themselves to their patients’ wellbeing.” | |
| Schoenthaler et al [] | Adoption | Patient and providers | ePROM digital solution | Availability of the data collection, Interoperability, and data presentation for decision-making | “These included defining a threshold that patients’ data can fall above or below and depicting it in a way that makes it easily detectible and depicting it in a way that makes it easily detectible and actionable, using bar graphs to show directionality, including icons or coloring schemes in addition to PRO labels that enhance the readability of the report, and, including summary data in percentages or raw numbers to show the patient’s progress over time.” | |
| Bezerra Giordan et al [] | Adoption | Patient | Introduction | Introduction after an exacerbation | “However, this barrier could be overcome by introducing the app soon after an exacerbation, when they might be more willing to improve their self-management practices.” | |
| Schoenthaler et al [] | Implementation | Providers | Communication | Patient-oriented treatment | “Similar to patients, they felt that the insight messages were helpful for interpreting the data and prompting behavioral changes.” | |
| Schoenthaler et al [] | Implementation | Providers and patient | ePROM digital solution | Data presentation, collection of only relevant patient data, and comprehensible user interfaces | “Patients preferred layouts that used darker fonts and lighter background colors to help make the text easier to read. All patients viewed the color-coded schema favorably because it helped draw attention to the most important aspects of the report and made the data easy to interpret.” | |
| Schoenthaler et al [] | Implementation | Providers and patient | Psychometrics | Valid PROM and sensitivity to change | “PROs should show variability in patients’ responses over time and be actionable by both patients and providers.” | |
| Steele Gray et al [] | Maintenance | Providers | Adaptation | ePROMs process not aligned with clinical reality | “To improve effectiveness (and efficiency), providers wanted the tool to fit better with their existing workflows and programs, for example, through better alignment with creation of SMART goals for patients or allowing for monitoring protocols that aligned with goals of existing chronic disease management programs.” | |
| Steele Gray et al [] | Maintenance | Patient and providers | Communication | Peer support, feedback aligned with needs, and automated personalized milestones | “Tool-enabled feedback, particularly from peers, was also viewed to offer encouragement on progress toward goal attainment by way of a shared experience.” | |
| Steele Gray et al [] | Maintenance | Providers and patient | ePROM digital solution | Longitudinal training, patient prompting, technical support, and interoperability | “The meaningfulness of the ePRO tool was reliant to consistent positive assessments of the tool’s utility (regular reflexive monitoring).” | |
aePROM: electronic patient-reported outcome measure.
bPROM: patient-reported outcome measure.
Digital Literacy and Training
The data highlight challenges with the lack of digital literacy among patients and providers, alongside insufficient training for providers [,]. These issues affected various aspects of the RE-AIM framework, notably impacting reach, adoption, and implementation. Addressing digital literacy through targeted training sessions for both patients and providers emerges as a facilitator by enhancing the usability and acceptance of ePROMs tools [,].
Patient-Provider Communication and Relationship
The dynamics of communication and the relationship between patients and providers are identified as both barriers and facilitators [,]. The lack of patient-provider interaction and inadequate feedback are significant barriers, while strong relationships, interactivity, and timely communication facilitate implementation [].
Personalization and Integration Into Clinical Workflow
The necessity for ePROMs tools to be personalized and seamlessly integrated into clinical workflows is a common theme across barriers and facilitators. Studies showed the reluctance to adopt new practices due to personalization and integration issues [,], contrasting with facilitators that advocate for straightforward data presentation and interoperability [].
Technical Challenges and Support
Studies identified technical difficulties, including data entry, information retrieval, and a lack of technical support, as significant barriers []. Facilitators highlighted in the studies were ongoing technical support and user-friendly interfaces [].
Workload and Treatment Quality
Concerns related to the impact of ePROMs on workload and treatment quality are highlighted barriers [,]. The increased workload for providers and potential distractions from acute health issues pose major barriers, while the potential for improved treatment quality through enhanced patient contexts offers a significant advantage [].
Quantitative Results
Overview
We present an overview of all quantitative results in Table 4.
| Study | Intervention and outcome | Effect of the intervention | Results | Sample size |
| Steele Gray et al [] | Adopting the ePROa tool on:
| Null |
| 16 |
| Staeheli et al [] | Screening intervention on:
| Positive effect |
| 275 |
| Ainsworth et al [] | Digital interventions on:
| Positive effect |
| 88 |
| Steele Gray et al [] | ePRO mobile app and portal system on:
| Null |
| 44 |
| Harle et al [] | PROe data in an EHRf on:
| Null |
| 370 |
| Kroenke et al [] | PROMISg symptom scores to clinicians on:
| Null |
| 256 |
| Lear et al [] |
| Null |
| 229 |
| Miranda et al [] | ePRO tool on:
| Null |
| 45 |
| Baron and Duffecy [] | Technology-assisted sleep extension on:
| Positive effect |
| 16 |
| Bauer et al [] | Mobile health platform supporting collaborative care on:
| Positive effect |
| 17 |
| Ramallo-Fariña et al [] | Interventions of knowledge transfer and behavior modification on:
| Positive effect |
| 2334 |
| Tamisier et al [] | Telemonitoring intervention on:
| Null |
| 206 |
| Trick et al [] | ACASIq system on:
| Positive effect |
| 1670 |
| Yanicelli et al [] | Home telemonitoring system on:
| Positive effect |
| 30 |
| Owen-Smith et al [] | Automate PRO data collection on:
| Positive effect |
| 632 |
aePRO: electronic patient-reported outcome.
bQoL: quality of life.
cAQLQ: Asthma Quality of Life Questionnaire.
dAQC: acceptable quality level.
ePRO: patient-reported outcome.
fEHR: electronic health record.
gPROMIS: Patient-Reported Outcomes Measurement Information System.
hCDM: chronic disease management.
iQALY: quality-adjusted life year.
jTST: total sleep time.
kPTI: patient-therapist interaction.
lUC: usual care.
mPFI: Physical Function Index.
nCBI: cognitive behavioral intervention.
oCPAP: continuous positive airway pressure.
pTM: telemonitoring.
qACASI: audio computer-assisted self-interview.
rCG: control group.
sEHFScB: European Heart Failure Self-care Behaviour scale.
tIG: intervention group.
Clinical Outcomes and Quality of Life
A tablet-based electronic screening tool significantly improved the identification of behavioral health problems, offering an efficient approach to behavioral health screening and follow-up health care []. Overall, 3 studies showed no improvement on symptoms [,,,], while another study indicated a positive association []. The use of ePROMs in asthma management showed improvements within group in asthma-related quality of life and symptom control but were not statistically significant between groups []. The adoption of an ePROMs tool did not demonstrate statistical differences in overall or subscale scores of quality of life, patient activation, or satisfaction with chronic pain health care across both control and intervention groups [,,,]. Quality of life was used as the dependent variable by 5 studies, 3 showed no improvements [,,], 1 showed improvement [], and 1 showed decline [].
Self-Management and Health Behaviors
Some studies showed significant improvement in self-management [], skill and technique acquisition, and social integration [], as well as adherence to diet recommendations and mental health outcomes over time []. Two studies showed no improvements in patient activation [] and self-management [].
System-Level Outcomes
Two studies indicated that implementing ePROMs does not reduce rehospitalizations [,]. Moreover, compared to usual care, the implementation of ePROMs is associated with a drop in cost-effectiveness because the data collection targets all patients and not specific cases. In addition, differences in patients’ digital literacy may also explain this drop in cost-effectiveness []. The study by Staeheli et al [] showed an improvement in follow-up care.
Feasibility and Acceptability
Four studies indicated that ePROMs’ implementation is feasible and achieves a good level of acceptability [,,,], while 2 studies showed no association with satisfaction [,]. One study focusing on mental health outcomes also highlighted significant feasibility and acceptability as well as high engagement []. However, it also reported a steep attrition in long-term data collection []. One study implementing an audio computer-assisted self-interview system showed minimal time burden and high feasibility for patients regardless of age, language, and computer literacy [].
Discussion
Principal Findings
It is difficult to draw clear conclusions about the effects of ePROM implementation in PHC. However, combining insights from both qualitative and quantitative data on the implementation and effects of ePROMs yields a nuanced understanding of their potential and challenges. A lack of digital literacy and engagement seems to be a key barrier to effectiveness, and in quantitative studies, ePROMs were well received when researchers emphasized feasibility and acceptability and provided training and support. This suggests that improving digital literacy and ensuring user-friendly design and adequate support are crucial for enhancing patient engagement with ePROMs and their effectiveness. Qualitative findings highlighted that the lack of personalization and patient-provider communication were impactful. Quantitatively, interventions that allowed for personalized feedback, goal setting, and self-management support showed positive effects. This indicates that ePROMs that are personalized and facilitate or enhance communication between patients and providers can lead to better health outcomes and patient experiences.
Qualitative data provided insights into perceived benefits, such as increased confidence and motivation and improved person-centered health care, even when quantitative outcomes showed null effects regarding clinical metrics such as health status and quality of life. This suggests a complex relationship between perceived benefits and measurable health outcomes, indicating that ePROMs may impact aspects of health care and patient experience not fully captured by quantitative measures. It also underscores the nuanced effects of digital and behavioral interventions on self-management and behavioral outcomes, highlighting the importance of personalized, interactive approaches and the potential for digital platforms to support but not fully substitute for comprehensive health care strategies aimed at enhancing patient engagement and self-management capabilities [,,,]. Qualitative insights suggest that despite the enthusiasm for ePROMs and digital interventions, there are concerns about their long-term impact and cost-effectiveness, with quantitative data echoing these concerns. While qualitative data highlight benefits such as increased confidence and motivation and improved patient-centered care, these effects do not always translate into measurable clinical outcomes. This highlights the need for more comprehensive research into the long-term outcomes, cost-effectiveness, and broader societal utility of digital health interventions.
Our review found that most studies were conducted in the United States (9/22, 41%) and Canada (8/22, 36%). The relative emphasis on these regions means that the findings may be influenced by North American health care financing models and policy environments. For instance, in the United States, fragmented insurance coverage and variable reimbursement structures can influence the adoption of digital health tools. In Canada, publicly funded health care and provincial eHealth strategies might support ePROM implementation by offering centralized funding or infrastructure but can also slow adoption. Other high-income countries may have distinct funding mechanisms, regulatory requirements, or national digital health agendas that shape ePROM uptake differently. With most included studies conducted in North America, our results have limited generalizability. Health care systems worldwide vary in digital infrastructure, reimbursement models, and regulatory frameworks, influencing ePROMs adoption. Future research should explore cross-cultural differences and assess implementation in diverse settings to better inform global policy and practice.
The results of this study corroborate those of other systematic reviews. First, systematic reviews carried out in pediatric health care [], oncology [,], or breast cancer treatment [] also present divergent results about patient satisfaction, quality of health care, health outcomes, patient management, and patient health behavior. The authors of these studies conclude that differences in the context of health care and the quality of ePROM implementation make it difficult to generalize the results of the studies that have been identified. Ishaque et al [] also associates this divergence to the low statistical power of most of the studies reviewed. Second, the authors also conclude that ePROM implementations have the potential to improve patient health, provided they consider the barriers and facilitators specific to each health care setting [,,].
Some of our results also differ from those of systematic reviews that have looked at other health care contexts. Indeed, systematic reviews indicate a positive, clear, and well-supported relationship between the implementation of ePROMs and health benefits. For example, in pediatrics, the integration of ePROMs is associated with an increase in health-related quality of life and patient satisfaction [], while in oncology, the detection of unidentified problems and the monitoring of treatment response are improved by the implementation of ePROMs []. This difference in results from our study can probably be explained by the greater heterogeneity of populations and treatments in PHC, pointing at the significant challenge to widespread ePROMs implementation and uptake in this setting.
Limitations
Our study has several limitations. The heterogeneity of study designs and outcome measures precluded meta-analysis, necessitating a narrative synthesis. The quantitative studies included in our sample assessed a diverse range of health indexes using scales that were not based on a common metric, making it impossible to extract standardized results for quantification. Consequently, we interpreted their findings using the vote-counting method []. However, the vote-counting method has been criticized [], particularly because it does not account for effect sizes, limiting comparability and resulting in lower precision than meta-analysis. Our qualitative data analysis was limited to a descriptive approach. To accurately capture the net effect of ePROM implementation, future studies should aim to minimize the influence of barriers while maximizing the impact of facilitators. To advance this understanding, we plan to conduct a meta-synthesis of the identified qualitative studies, offering a comprehensive analysis of barriers, facilitators, and their interrelationships.
In terms of public decision-making, we can question the real size of the social benefits of implementing ePROMs. Indeed, only 1 of the studies in our sample carried out a cost-benefit analysis [], and none of them did so in an ecological manner. No study has compared the benefits of implementing ePROMs to those of another intervention with the same level of financial investment. Future research should conduct cost-benefit analyses in a real-world context to identify which sector would yield the greatest overall benefit from financial investment.
Success in implementing ePROMs in PHC appears to hinge on addressing digital literacy, ensuring personalization and meaningful patient-provider interactions, carefully integrating technology into clinical workflows, and conducting thorough research on their long-term impacts and cost-effectiveness. ePROM implementation research could conduct large‐scale, multisite randomized trials or pragmatic trials to compare ePROM implementation strategies; investigate long-term outcomes and sustainability (eg, cost‐utility analyses and ecological cost‐benefit comparisons); focus on diverse patient populations, especially underserved communities, to address health equity and digital literacy gaps; and use standardized outcome measures or validated ePROM instruments to facilitate cross‐study comparisons. By focusing on these areas, future efforts could better assess the benefits of digital health technologies for patients, providers, and health care systems.
Acknowledgments
The authors would like to acknowledge the insight and impact of their deceased colleague Pierre Pluye, who helped plan the mixed methods methodology, the funding proposal, and the protocol of this study, as well as inspire them as researchers. This research was funded by the Canadian Institute for Health Research.
Conflicts of Interest
None declared.
Multimedia Appendix 3
Digital tools used for electronic patient-reported outcome measures’ collection and implementation in electronic medical records.
DOCX File , 27 KBReferences
- Primary health care. World Health Organization. URL: https://www.who.int/health-topics/primary-health-care#tab=tab_1 [accessed 2024-01-01]
- Declaration of Alma-Ata. World Health Organization. Oct 8, 2019. URL: https://www.who.int/publications/i/item/WHO-EURO-1978-3938-43697-61471 [accessed 2024-01-01]
- Why Canada’s health system needs (a lot more) team-based care. Canadian Medical Association. URL: https://tinyurl.com/5xz7j5s5 [accessed 2024-01-01]
- Access to primary care. Office of Disease Prevention and Health Promotion, Office of the Assistant Secretary for Health, Office of the Secretary, U.S. Department of Health and Human Services. URL: https://odphp.health.gov/healthypeople/priority-areas/social-determinants-health/literature-summaries/access-primary-care [accessed 2024-01-01]
- About chronic diseases. Centers for Disease Control and Prevention. Oct 4, 2024. URL: https://www.cdc.gov/chronicdisease/about/index.htm [accessed 2024-01-01]
- Betancourt MT, Roberts KC, Bennett TL, Driscoll ER, Jayaraman G, Pelletier L. Monitoring chronic diseases in Canada: the Chronic Disease Indicator Framework. Chronic Dis Inj Can. Jun 2014;34(Supplement 1):1-30. [CrossRef]
- Evaluation of the Strategy for Patient-Oriented Research (SPOR). Canadian Institutes of Health Research. Jun 2023. URL: https://cihr-irsc.gc.ca/e/53635.html [accessed 2024-01-01]
- Nguyen H, Butow P, Dhillon H, Sundaresan P. A review of the barriers to using Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs) in routine cancer care. J Med Radiat Sci. Jun 19, 2021;68(2):186-195. [FREE Full text] [CrossRef] [Medline]
- Roux-Lévy PH, Poitras ME, Supper W, Attisso E, Dugas M, LeBlanc A, et al. Systèmes de mesures auto-raportées pour guider la prise de décisions poliWques. SPOR Evidence Alliance. Nov 18, 2023. URL: https://sporevidencealliance.ca/wp-content/uploads/2023/11/SPOREA_CSBE-Decision-Making-Report_Final.pdf [accessed 2024-01-01]
- Glenngård AH, Anell A. Process measures or patient reported experience measures (PREMs) for comparing performance across providers? A study of measures related to access and continuity in Swedish primary care. Prim Health Care Res Dev. Sep 15, 2017;19(01):23-32. [CrossRef]
- Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. Mar 2008;11(2):322-333. [FREE Full text] [CrossRef] [Medline]
- Meirte J, Hellemans N, Anthonissen M, Denteneer L, Maertens K, Moortgat P, et al. Benefits and disadvantages of electronic patient-reported outcome measures: systematic review. JMIR Perioper Med. Apr 03, 2020;3(1):e15588. [FREE Full text] [CrossRef] [Medline]
- Bele S, Chugh A, Mohamed B, Teela L, Haverman L, Santana MJ. Patient-reported outcome measures in routine pediatric clinical care: a systematic review. Front Pediatr. Jul 28, 2020;8:364. [FREE Full text] [CrossRef] [Medline]
- van Egdom LS, Oemrawsingh A, Verweij LM, Lingsma HF, Koppert LB, Verhoef C, et al. Implementing patient-reported outcome measures in clinical breast cancer care: a systematic review. Value Health. Oct 2019;22(10):1197-1226. [FREE Full text] [CrossRef] [Medline]
- Ainsworth B, Greenwell K, Stuart B, Raftery J, Mair F, Bruton A, et al. Feasibility trial of a digital self-management intervention 'My Breathing Matters' to improve asthma-related quality of life for UK primary care patients with asthma. BMJ Open. Nov 12, 2019;9(11):e032465. [FREE Full text] [CrossRef] [Medline]
- Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. London, UK. The Cochrane Collaboration; 2008.
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. Jun 2021;134:103-112. [CrossRef] [Medline]
- Chien PF, Khan KS, Siassakos D. Registration of systematic reviews: PROSPERO. BJOG. Jul 17, 2012;119(8):903-905. [CrossRef] [Medline]
- Sasseville M, Supper W, Gartner JB, Layani G, Amil S, Sheffield P, et al. Clinical integration of digital patient-reported outcome measures in primary health care for chronic disease management: protocol for a systematic review. JMIR Res Protoc. Aug 18, 2023;12:e48155. [FREE Full text] [CrossRef] [Medline]
- Hupe M. EndNote X9. J Electron Resour Med Libr. Nov 26, 2019;16(3-4):117-119. [CrossRef]
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [FREE Full text] [CrossRef] [Medline]
- Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. J Clin Epidemiol. Jul 2019;111:49-59.e1. [FREE Full text] [CrossRef] [Medline]
- Glasgow RE, Dickinson P, Fisher L, Christiansen S, Toobert DJ, Bender BG, et al. Use of RE-AIM to develop a multi-media facilitation tool for the patient-centered medical home. Implement Sci. Oct 21, 2011;6(1):118. [FREE Full text] [CrossRef] [Medline]
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. Sep 1999;89(9):1322-1327. [CrossRef] [Medline]
- Hong QN, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Rev. Mar 23, 2017;6(1):61. [FREE Full text] [CrossRef] [Medline]
- Laux G, Kuehlein T, Rosemann T, Szecsenyi J. Co- and multimorbidity patterns in primary care based on episodes of care: results from the German CONTENT project. BMC Health Serv Res. Jan 18, 2008;8(1):14. [FREE Full text] [CrossRef] [Medline]
- Staeheli M, Aseltine RHJ, Schilling E, Anderson D, Gould B. Using mHealth technologies to improve the identification of behavioral health problems in urban primary care settings. SAGE Open Med. Jun 08, 2017;5:2050312117712656. [FREE Full text] [CrossRef] [Medline]
- Harle CA, Marlow NM, Schmidt SO, Shuster JJ, Listhaus A, Fillingim RB, et al. The effect of EHR-integrated patient-reported outcomes on satisfaction with chronic pain care. Am J Manag Care. Dec 01, 2016;22(12):e403-e408. [FREE Full text] [Medline]
- Kroenke K, Talib TL, Stump TE, Kean J, Haggstrom DA, DeChant P, et al. Incorporating promis symptom measures into primary care practice-a randomized clinical trial. J Gen Intern Med. Aug 5, 2018;33(8):1245-1252. [FREE Full text] [CrossRef] [Medline]
- Lear SA, Norena M, Banner D, Whitehurst DG, Gill S, Burns J, et al. Assessment of an interactive digital health-based self-management program to reduce hospitalizations among patients with multiple chronic diseases: a randomized clinical trial. JAMA Netw Open. Dec 01, 2021;4(12):e2140591. [FREE Full text] [CrossRef] [Medline]
- Miranda RN, Bhuiya AR, Thraya Z, Hancock-Howard R, Chan BC, Steele Gray C, et al. An electronic patient-reported outcomes tool for older adults with complex chronic conditions: cost-utility analysis. JMIR Aging. Apr 20, 2022;5(2):e35075. [FREE Full text] [CrossRef] [Medline]
- Baron K, Duffecy J. Technology assisted behavior intervention to extend sleep among adults with short sleep duration and prehypertension/stage 1 hypertension: a randomized pilot feasibility study. Sleep Med. Dec 2019;64:S26. [CrossRef]
- Owen-Smith A, Mayhew M, Leo MC, Varga A, Benes L, Bonifay A, et al. Automating collection of pain-related patient-reported outcomes to enhance clinical care and research. J Gen Intern Med. May 9, 2018;33(Suppl 1):31-37. [FREE Full text] [CrossRef] [Medline]
- Ramallo-Fariña Y, Rivero-Santana A, García-Pérez L, García-Bello MA, Wägner AM, Gonzalez-Pacheco H, et al. Patient-reported outcome measures for knowledge transfer and behaviour modification interventions in type 2 diabetes-the INDICA study: a multiarm cluster randomised controlled trial. BMJ Open. Dec 15, 2021;11(12):e050804. [FREE Full text] [CrossRef] [Medline]
- Tamisier R, Treptow E, Joyeux-Faure M, Levy P, Sapene M, Benmerad M, et al. Impact of a multimodal telemonitoring intervention on CPAP adherence in symptomatic OSA and low cardiovascular risk: a randomized controlled trial. Chest. Nov 2020;158(5):2136-2145. [CrossRef] [Medline]
- Trick WE, Deamant C, Smith J, Garcia D, Angulo F. Implementation of an audio computer-assisted self-interview (ACASI) system in a general medicine clinic. Appl Clin Inform. Dec 19, 2017;06(01):148-162. [CrossRef]
- Yanicelli LM, Goy CB, González VD, Palacios GN, Martínez EC, Herrera MC. Non-invasive home telemonitoring system for heart failure patients: a randomized clinical trial. J Telemed Telecare. Jan 23, 2020;27(9):553-561. [CrossRef]
- Bezerra Giordan L, Ronto R, Chau J, Chow C, Laranjo L. Use of mobile apps in heart failure self-management: qualitative study exploring the patient and primary care clinician perspective. JMIR Cardio. Apr 20, 2022;6(1):e33992. [FREE Full text] [CrossRef] [Medline]
- Steele Gray C, Gill A, Khan AI, Hans PK, Kuluski K, Cott C. The electronic patient reported outcome tool: testing usability and feasibility of a mobile app and portal to support care for patients with complex chronic disease and disability in primary care settings. JMIR Mhealth Uhealth. Jun 02, 2016;4(2):e58. [FREE Full text] [CrossRef] [Medline]
- Hans PK, Gray CS, Gill A, Tiessen J. The provider perspective: investigating the effect of the Electronic Patient-Reported Outcome (ePRO) mobile application and portal on primary care provider workflow. Prim Health Care Res Dev. Sep 13, 2017;19(02):151-164. [CrossRef]
- Irfan Khan A, Gill A, Cott C, Hans PK, Steele Gray C. mHealth tools for the self-management of patients with multimorbidity in primary care settings: pilot study to explore user experience. JMIR Mhealth Uhealth. Aug 28, 2018;6(8):e171. [FREE Full text] [CrossRef] [Medline]
- Schoenthaler A, Cruz J, Payano L, Rosado M, Labbe K, Johnson C, et al. Investigation of a mobile health texting tool for embedding patient-reported data into diabetes management (i-Matter): development and usability study. JMIR Form Res. Aug 31, 2020;4(8):e18554. [FREE Full text] [CrossRef] [Medline]
- Steele Gray C, Gravesande J, Hans PK, Nie JX, Sharpe S, Loganathan M, et al. Using exploratory trials to identify relevant contexts and mechanisms in complex electronic health interventions: evaluating the electronic patient-reported outcome tool. JMIR Form Res. Feb 27, 2019;3(1):e11950. [FREE Full text] [CrossRef] [Medline]
- Ahmed S, Zidarov D, Eilayyan O, Visca R. Prospective application of implementation science theories and frameworks to inform use of PROMs in routine clinical care within an integrated pain network. Qual Life Res. Nov 02, 2021;30(11):3035-3047. [FREE Full text] [CrossRef] [Medline]
- Steele Gray C, Chau E, Tahsin F, Harvey S, Loganathan M, McKinstry B, et al. Assessing the implementation and effectiveness of the electronic patient-reported outcome tool for older adults with complex care needs: mixed methods study. J Med Internet Res. Dec 02, 2021;23(12):e29071. [FREE Full text] [CrossRef] [Medline]
- Bauer AM, Iles-Shih M, Ghomi RH, Rue T, Grover T, Kincler N, et al. Acceptability of mHealth augmentation of collaborative care: a mixed methods pilot study. Gen Hosp Psychiatry. Mar 2018;51:22-29. [FREE Full text] [CrossRef] [Medline]
- Harle CA, Listhaus A, Covarrubias CM, Schmidt SO, Mackey S, Carek PJ, et al. Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report. J Am Med Inform Assoc. Jan 2016;23(1):74-79. [FREE Full text] [CrossRef] [Medline]
- Ishaque S, Karnon J, Chen G, Nair R, Salter AB. A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Qual Life Res. Mar 3, 2019;28(3):567-592. [CrossRef] [Medline]
- Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. Jun 11, 2013;13(1):211. [FREE Full text] [CrossRef] [Medline]
- Glenwright BG, Simmich J, Cottrell M, O'Leary SP, Sullivan C, Pole JD, et al. Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review. J Patient Rep Outcomes. Feb 14, 2023;7(1):13. [FREE Full text] [CrossRef] [Medline]
- Rothman KJ, Greenland S, Lash TL. Modern Epidemiology. Philadelphia, PA. Lippincott Williams & Wilkins; 2008.
Abbreviations
| ePROM: electronic patient-reported outcome measure |
| MMAT: Mixed Methods Appraisal Tool |
| PHC: primary health care |
| PICOS: Population, Intervention, Comparison, Outcomes, and Setting |
| PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
| RE-AIM: reach, effectiveness, adoption, implementation, and maintenance |
Edited by J Sarvestan, T Leung; submitted 27.06.24; peer-reviewed by M Peeples, W-J He; comments to author 20.01.25; revised version received 09.02.25; accepted 03.03.25; published 05.05.25.
Copyright©Maxime Sasseville, Wilfried Supper, Jean-Baptiste Gartner, Géraldine Layani, Samira Amil, Peter Sheffield, Marie-Pierre Gagnon, Catherine Hudon, Sylvie Lambert, Eugène Attisso, Steven Ouellet, Mylaine Breton, Marie-Eve Poitras, Pierre-Henri Roux-Lévy, James Plaisimond, Frédéric Bergeron, Rachelle Ashcroft, Sabrina T. Wong, Antoine Groulx, Jean-Sébastien Paquette, Natasha D'Anjou, Sylviane Langlois, Annie LeBlanc. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 05.05.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.