This paper is in the following e-collection/theme issue:

Digital Health Reviews (1522) Personal Health Records, Patient-Accessible Electronic Health Records, Patient Portals (636) Electronic Health Records (1183) Patient-Reported Outcome Measures (PROMs) (325) Health Services Research in Oncology (43) Cancer Self-Management (69) Innovations and Technology in Cancer Care (463) Mobile Apps for Cancer Care and Cancer Prevention and Screening (240) Participatory Cancer Care (106)

Published on in Vol 27 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/69398, first published .
Sustainability and Time Trends in Electronic Patient-Reported Outcome Assessment in Routine Cancer Care: Systematic Scoping Review and Follow-Up Survey

Sustainability and Time Trends in Electronic Patient-Reported Outcome Assessment in Routine Cancer Care: Systematic Scoping Review and Follow-Up Survey

Sustainability and Time Trends in Electronic Patient-Reported Outcome Assessment in Routine Cancer Care: Systematic Scoping Review and Follow-Up Survey

Authors of this article:

Niclas J Hubel1 Author Orcid Image ;   Samuel M Vorbach2 Author Orcid Image ;   Kelly M de Ligt3 Author Orcid Image ;   Ines S Rathgeber1 Author Orcid Image ;   Katharina Beyer4 Author Orcid Image ;   Lisa M Wintner1 Author Orcid Image ;   Barbara Faller5 Author Orcid Image ;   Jasmin Nemec5 Author Orcid Image ;   Bernhard Holzner1 Author Orcid Image ;   Monika Sztankay1 Author Orcid Image ;   Jens Lehmann1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Review

    1University Hospital of Psychiatry II, Medical University of Innsbruck, Innsbruck, Austria

    2Department of Radiation Oncology, Medical University of Innsbruck, Innsbruck, Austria

    3Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute, Amsterdam, The Netherlands

    4Department of Urology, ​​Erasmus Medical Center Cancer Institute, University Medical Center Rotterdam, Rotterdam, The Netherlands

    5Department of Psychology, Universität Innsbruck, Innsbruck, Austria

    *these authors contributed equally

    Corresponding Author:

    Jens Lehmann, PhD

    University Hospital of Psychiatry II

    Medical University of Innsbruck

    Anichstrasse 35

    Innsbruck, 6020

    Austria

    Phone: 43 5050481551

    Email: jens.lehmann@i-med.ac.at


    Background: Routine electronic assessment of patient-reported outcomes (ePROs) can improve cancer care; yet, its implementation in routine practice and long-term sustainability remain unclear. Understanding these aspects is critical to advancing the field.

    Objective: To review and describe the past and current status, time trends, and long-term sustainability of clinical ePRO applications in routine oncology care.

    Methods: We conducted a systematic review of publications on ePRO use in oncology care up to December 31, 2023, searching PubMed and Web of Science and extracting data on clinical ePRO applications. We included peer-reviewed studies including patients with cancer using ePRO assessments in clinical practice, excluding research letters and conference abstracts. Data from the review were analyzed using descriptive statistics and univariate regression models to evaluate time trends, with year of publication as the predictor. A follow-up survey was sent to authors of published ePRO applications to assess their current use of the application or reasons for discontinuation. Responses from the follow-up survey were analyzed descriptively.

    Results: For the review, we screened 2933 references, and 303 met inclusion criteria. Results showed that Europe was the most common region (n=141, 46.5%), and study populations consisted mostly of adult patients (n=276, 91.1%) under chemotherapy treatment (n=124, 40.9%) assessed in an outpatient setting (n=261, 86.1%). The EORTC (European Organisation for Research and Treatment of Cancer; n=77, 25.4%) and PRO-CTCAE (Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events) questionnaires (n=65, 21.5%) were most frequently integrated into ePRO applications. In the univariate analysis, we found that publications increased significantly over time (2003-2023, P<.001). Trends showed a rise in mobile app use (odds ratio [OR] 1.211, P<.001), remote assessments (OR 1.094, P=.002), and feedback provided to patients (OR 1.060, P=.04). Of the 303 studies, 221 unique clinical ePRO applications were identified, merging publications at the application level. The follow-up survey had a 35.3% response rate (78/221), with 61.1% of ePRO applications still in use, lasting a median of 5 years. The most common reason for discontinuation was a lack of funding and resources (42.9%, 12/28).

    Conclusions: The field of ePRO assessment in oncology is rapidly evolving, with a shift toward remote, app-based tools and a growing emphasis on providing feedback to patients. We present, for the first time, data on the sustainability of ePRO use in routine care. While our findings offer valuable insights, they should be interpreted in light of potential response bias in the follow-up survey. Several ePRO applications remain in active use, highlighting potential for long-term integration into clinical practice. However, financial constraints, limited reimbursement models, and challenges with workflow integration continue to hinder broader and more sustainable adoption. Addressing these barriers will be essential to support the continued use of ePROs in clinical care.

    J Med Internet Res 2025;27:e69398

    doi:10.2196/69398

    Keywords

    patient-reported outcome measures; cancer, clinical routine; health-related quality of life; quality of care; mobile applications; digital technology 


    Adequate symptom monitoring is a cornerstone of clinical care for patients with cancer. During and after treatment, patients frequently experience symptoms that can impact their health-related quality of life (HRQOL), therapy compliance, or even be life-threatening [1-3]. Routine assessment and use of patient-reported outcomes (PROs) in clinical care can improve care for patients with cancer, resulting in better HRQOL, reduced hospitalization rates, and even prolonged survival [1,4-9]. Accordingly, the European Society for Medical Oncology advises the routine assessment of PROs in clinical care, emphasizing the importance of electronic PRO (ePRO) assessments [10].

    In recent years, electronic approaches to collecting PRO data—commonly referred to as ePRO assessments—have gained increasing importance. Real-time digital platforms (hereafter referred to as ePRO software) offer an efficient and scalable solution to systematically collect, store, and report PRO data in clinical settings. Compared to traditional paper-based methods, ePRO software facilitates standardized and timely assessments, can potentially reduce response burden and missing data, and enhance overall usability and patient satisfaction [7,11-14].

    Despite all these advantages, there seems to be a challenge to sustainably implement such ePRO interventions into routine workflows [15,16]. The complex nature of routine care, fragmented workflows, limited resources, and reluctance to change within health care systems may contribute to the slow progress in implementing ePRO software tools outside of research settings [16-18]. Another common difficulty lies in obtaining reliable data on the use of ePRO software in clinical practice settings [19]. While many studies are published during the initial development and implementation phase [20], there is little information on the long-term use of those clinical ePRO applications (referring to the use of ePRO software in a specific clinical context). A considerable portion of developed clinical ePRO applications may be discontinued following the completion of research initiatives, potentially failing to transition into long-term use in clinical care.

    The landscape of ePRO assessment in routine oncology is evolving rapidly, with an increasing number of publications emerging. However, since the review by Warrington et al in 2019 (which included studies published until 2017) [21], no recent comprehensive reviews have captured this trend across the field of oncology as a whole—a considerable time span for a fast-changing field. Current reviews on clinical ePRO applications often have limitations that prevent a comprehensive overview of the current landscape of clinical ePRO applications used in oncology practice, like focusing on specific patient populations or applying narrow inclusion criteria (eg, focusing on palliative care applications only or on ePRO use for specific types of treatment) [20,22].

    The objectives of this systematic scoping review and follow-up survey are to review and describe the past and current practice of published ePRO use in oncology clinical care; to assess potential trends in the clinical use and assessment of ePRO data over time; and to assess the sustainability and long-term use of the reported clinical ePRO applications.


    Review

    The review was conducted according to the methodological framework by Arksey and O’Malley [23] and reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) 2020 checklist (see Methods S1 in Multimedia Appendix 1) [24].

    We searched PubMed and Web of Science for studies published until December 31, 2023, without restrictions regarding the publication date. A detailed description of the search strategy is provided in Methods S2 in Multimedia Appendix 1. Reviews and meta-analyses identified in the initial search were used for a backward search (identifying relevant studies based on the cited references).

    We included studies that were conducted exclusively with oncology patients, reported on the electronic assessment of PROs, were peer-reviewed primary research, and were available in the English language. We excluded studies that evaluated experimental interventions other than the clinical ePRO application, research letters, editorials, and conference abstracts.

    The review software DistillerSR (DistillerSR Inc) [25] was used for the 2-level study selection process and the subsequent data extraction. References were independently screened and reviewed by 2 individual reviewers experienced with ePROs. Disagreements were resolved in discussion. If no agreement was reached, a third (senior) reviewer was consulted. The data extraction followed the same procedure.

    For general study characteristics, we extracted details on authors, address of correspondence, year, and country of origin. We categorized the respective studies into the following publication types reflecting different stages of the clinical ePRO application and its use: system development (incomplete system, still under technical evaluation), feasibility study (finalized system, still under practical evaluation), study protocol, trial (evaluating a finalized ePRO system), implementation study (integration of finalized system into clinical workflow), and routine application (describing use of PRO data from an implemented system).

    We charted the purpose of ePRO use according to the International Society for Quality of Life Research (ISOQOL) user guide for implementing PROs in clinical practice [26]. Screening tools (one-time PRO assessment with feedback to clinicians), monitoring tools (repeated assessment with feedback of results to clinicians), patient-centered care (feedback of PRO results to patients), ePRO assessment without direct feedback, decision aids (information about treatment options or similar), evaluating quality of care, and facilitating multidisciplinary team communications. Patient groups were classified into inpatients and outpatients, and the time of PRO assessment was determined according to the respective care phase.

    Review: Analysis

    We report descriptive statistics of the publication characteristics and all extracted variables. All missing data were classified as “not reported” and excluded from all analyses. To investigate time trends in ePRO data capture over time, we calculated binomial Poisson and logistic regressions with the year of publication as the predictor and the number of publications per year as the outcome. Further, we conducted univariate logistic regressions with the year of publication as the predictor and one of the following selected variables as the criterion: inpatient population, outpatient population, remote assessment, in-hospital assessment, use of website, use of app, provision of PRO feedback to patients, provision of PRO feedback to clinicians, internal PRO software development, or external PRO software development. These variables were selected as markers for overall shifts within the field and indicators of technological advancements in the last 20 years. They reflect evolving approaches to data collection and to the integration of digital tools in clinical care.

    The selected outcomes were coded as 1=[true] and 0=[false]. Model coefficients are expressed as odds ratios (ORs) for the logistic regressions and as risk ratios for the Poisson regression, each with 95% CIs and an α level of .05. Calculations were done using R (version 4.3.1; R Core Team) [27].

    Follow-Up Survey

    We sent a follow-up questionnaire to the corresponding authors of all clinical ePRO applications that were identified based on the publications in the review (ie, merging publications on the distinct use of an ePRO software in a specific clinical context). The questionnaire was distributed via email, and up to 2 reminders were sent if needed. We developed the questionnaire among the authors who are experienced with the implementation of ePROs and pilot-tested it internally before distribution.

    Follow-Up Survey: Measures

    The survey aimed to evaluate the status of clinical ePRO application use. Participants were asked whether the application was still in use, with follow-up questions tailored accordingly: reasons for discontinuation or details about current usage practices were explored in alignment with the ISOQOL user guide [26]. Nonusers were invited to share what changes or features would encourage them to consider adopting ePRO assessments or systems in the future.

    Additional questions addressed the duration of ePRO application use, the average number of participating patients per month, and satisfaction levels, which were rated on a 4-point scale. Participants also had the opportunity to provide suggestions for improvement through an open-text field. The complete survey is detailed in Methods S3 in Multimedia Appendix 1.

    Follow-Up Survey: Analysis

    We report descriptive statistics of all variables; missing data were classified as “not reported” and excluded from all analyses. To evaluate potential response bias within our data, we calculated differences in the distribution of the respondents to our survey using Chi-squared tests and univariate logistic regressions for the following variables: the year of the last publication related to the respective ePRO clinical application, the total number of published articles referring to the clinical ePRO application, the region of the cancer centers, and the publication type of the last article published based on the clinical ePRO application.

    Calculations were done using R (version 4.3.1) [27].


    Review

    The initial literature search yielded 3346 references. After the removal of 413 duplicates, the remaining 2933 records underwent title abstract screening for eligibility, of which 409 studies were included in the full-text review. A final 303 studies were included in the review. Figure 1 depicts the PRISMA-ScR flowchart. The complete list of references can be found in eReferences in Multimedia Appendix 1. The main reasons for exclusions were that the full text did not include a clinical ePRO application (n=58) and alternative interventions other than ePRO monitoring were examined (n=18). Publication dates ranged from 2003 until 2023.

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart. ePRO: electronic patient-reported outcome.

    Based on the 303 publications included in the review, we identified 221 unique clinical ePRO applications. Most clinical ePRO applications (175, 57.8%) were reported only in a single publication, while 46 clinical ePRO applications were reported in multiple publications (median 2, IQR 1-1, range 2-10).

    Study Characteristics

    The majority of included studies were conducted in Europe (46.5%) and North America (39.6%), with the United States being the most common country (34.3%; Table 1, and Table S1 in Multimedia Appendix 1). According to our classification, the most used study types were ePRO feasibility studies (33%) and studies describing the routine use of clinical ePRO applications in clinical practice (17.8%). The complete study data are shown in Multimedia Appendix 2.

    Table 1. Study characteristics (N=303).

    		Publications

    		NumberPercentage
    Regiona

    		Europe14146.5

    		North America12039.6

    		Asia227.3

    		Australia124.0

    		South America41.3

    		International10.3

    		Not reported31.0
    Publication types

    		Feasibility study10033.0

    		Routine application5417.8

    		System development4213.9

    		Trial4113.5

    		Study protocol4013.2

    		Implementation study268.6
    Patient group

    		Outpatients26186.1

    		Inpatients103.3

    		Both185.9

    		Not reported144.6
    Patient age groups

    		Adults27691.1

    		Children and adolescents (<18 years)134.3

    		Not reported144.6
    Cancer type

    		Not specified or defined11036.3

    		Application used in specific cancer types (multiple could apply)19363.7

    		Breast7840.4

    		Gastrointestinal3719.2

    		Lung3719.2

    		Prostate3417.6

    		Colorectal3116.1

    		Head and neck2613.5

    		Hematological2513.0

    		Gynecological178.8

    		Otherb157.8
    Treatment (multiple could apply)

    		Chemotherapy12440.9

    		Radiotherapy6421.1

    		Surgery5618.5

    		Hormonal therapy3110.2

    		Targeted therapy247.9

    		Immunotherapy237.6

    		No active treatment216.9

    		Otherc62.0

    		Not reported9029.7

    aSee Table S1 in Multimedia Appendix 1 for full list.

    bBladder, central nervous system, liver, neuroendocrine, renal, retinoblastoma, thyroid, and genitourinary.

    cActive surveillance, cryotherapy, and stem cell transplantation.

    The applications were most frequently designed for patients with breast (n=78, 25.7%), gastrointestinal (n=37, 12.2%), and lung (n=37, 12.2%) cancers. Most studies were conducted with adult patients (n=276, 91.1%), and among outpatients (n=261, 86.1%) treated with chemotherapy (n=124, 40.9%).

    Reported use of ePRO Assessment in Clinical Practice

    The most common use cases of ePROs, according to the ISOQOL classification, were as monitoring tools (n=219, 72.3%) and for patient-centered care (n=123, 40.6%; Table 2). Physicians (n=166, 54.8%) and nurses (n=154, 50.8%) were the main users of ePRO software, while other health care professionals were rarely mentioned (less than 3%). The user groups were not specified in 30.7% of the publications.

    The ePRO assessment was predominantly conducted remotely (n=147, 48.5%) via a website interface (n=122, 40.3%). The respective ePRO software tools were partly developed in-house at the respective cancer centers (n=106, 35%) or by external IT vendors (n=109, 36%). A total of 145 unique ePRO software tools were identified, of which 39 were mentioned in more than 1 publication (median 1 publication, IQR 1-3, range 1-16). In total, 55 (18.2%) publications did not report any distinctive tool (Table S2 in Multimedia Appendix 1 for more information).

    The questionnaires most commonly assessed via ePRO were EORTC (European Organisation for Research and Treatment of Cancer) measures (n=77, 25.4%), closely followed by the PRO-CTCAE (Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events) questionnaire (n=65, 21.5%). A full list of the 136 used questionnaires can be found in Table S3 in Multimedia Appendix 1.

    Table 2. Characteristics and use of ePROa assessments (N=303).

    		Publications

    		NumberPercentage
    Assessment setting

    		Remote14748.5

    		In clinic7926.1

    		Both in clinic and remote5518.2

    		Not reported227.3
    Clinical user group (multiple could apply)

    		Physicians16654.8

    		Nurses15450.8

    		Psychologists82.6

    		Pharmacists72.3

    		Social workers62.0

    		Dietitians31.0

    		Others (physiotherapists, case managers, etc)103.3

    		Not reported9330.7
    Technology

    		Website interface12240.3

    		App7524.8

    		Both, website interface and app4113.5

    		Not reported6521.5
    Technical development of the software

    		External IT vendors10936.0

    		In-house10635.0

    		Joint development51.7

    		Not reported8327.4
    Time of ePRO assessment (multiple could apply)

    		During active treatment21370.3

    		During early follow-up (<2 years after treatment)12541.3

    		During long-term follow-up (≥2 years after treatment)4514.9

    		In palliative care3611.9

    		Pre-treatment124.0

    		Not reported3110.2
    Purpose of ePRO use (multiple could apply)

    		Monitoring tools (data collection over time with feedback of results to clinicians)21972.3

    		Patient-centered care (data collection with feedback of PRO results to patients)12340.6

    		ePRO assessment without feedback5919.5

    		Decision aids (for shared decision-making)103.3

    		Evaluating quality of care82.6

    		Screening tools (one-time PRO assessment with feedback to clinicians)72.3

    		Facilitating multidisciplinary team communications51.7
    PROMs assessed within the ePRO software tool (multiple could apply)

    		EORTCb questionnaires (C30 and modules)7725.4

    		PRO-CTCAEc6521.5

    		EQ-5D299.6

    		FACITd questionnaires278.9

    		Own development278.9

    		Otherse30793.4

    		Not reported3411.2

    aePRO: electronic patient-reported outcome.

    bEORTC: European Organisation for Research and Treatment of Cancer.

    cPRO-CTCAE: Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events.

    dFACIT: Functional Assessment of Chronic Illness Therapy.

    eSee Table S3 in Multimedia Appendix 1 for full list.

    Time Trends in ePRO Use

    The number of publications increased significantly over time (risk ratio 1.20, 95% CI 1.17-1.22, P<.001), rising from just 2-4 publications per year in the early 2000s to a peak of 42 in 2023. Notable increases occurred from 2014 onward, with particularly sharp growth between 2015 and 2017 and again after 2019. A detailed annual overview is provided in Multimedia Appendix 3.

    The binomial logistic regression model for app use was statistically significant (OR 1.21, 95% CI 1.12-1.32, P<.001), meaning that for each additional year, the likelihood that the ePRO assessment was embedded in an app increased by 21.1%. At the same time, fewer publications per year reported the use of a website interface (OR 0.85, 95% CI 0.77-0.93, P<.001).

    Remote assessments were more frequently reported in recent years (OR 1.09, 95% CI 1.03-1.16, P=.002) with an increase of 9.4% per year. The likelihood of articles describing in-clinic assessments decreased by 8% per year (OR 0.92, 95% CI 0.87-0.97, P=.004).

    Publications over time increasingly described providing patients PRO feedback (increase of 6% per year; OR 1.060, 95% CI 1.00-1.12, P=.04). There was no significant change over time in providing PRO feedback to physicians (P=.22), but this number was consistently high throughout the years (data not shown). Similarly, no time trend was observed regarding the reported patient population for either inpatient (P=.97) or outpatient settings (P=.87). The odds of using software tools developed by an external IT vendor increased significantly over time (OR 1.08, 95% CI 1.01-1.15, P=.02), whereas the trend for in-house development was not significant (P=.06).

    Follow-Up Survey

    The follow-up survey was sent out to all corresponding authors and received a response rate of 35.3% (78/221).

    There was no link between whether someone responded and the year their last clinical ePRO publication appeared (P=.15) or the type of publication (P=.13). However, response rates varied significantly between regions (χ²6=17.0, P=.009). Specifically, response rates were lower in Asia (3/20, 15%) and North America (24/90, 26.7%), while they were higher in Australia (5/6, 83.3%), Europe (43/97, 44.3%), and South America (2/4, 50%). Finally, authors who had published more articles on their clinical ePRO application were also more likely to respond (OR 1.44, 95% CI 1.08-2.06, P=.02).

    Of the 78 responses received, 50 (64.1%) indicated that the respective application was still in use and had been operational for a median of 5 (IQR 3-10) years (Table 3).

    The most prevalent use cases, according to the ISOQOL classification, were using the clinical ePRO application as a monitoring tool (34/50, 68%), for patient-centered care (32/50, 64%), and for research purposes (36/50, 72%), with respondents able to select multiple options. Further, respondents reported 1-50 (40%) or 51-200 (30%) average patient users per month.

    Among the authors who reported that their ePRO application was no longer in use (28/78, 35.9%), a lack of funding was the most common reason for discontinuation (12/28, 42.9%). About the same number of respondents reported that they would resume the ePRO application in case the required funding or resources (eg, administrative support, dedicated personnel, or IT support) were available (13/28, 46.4%). In total, 8 (28.6%) authors reported that they were no longer using the ePRO application they had been contacted for but that they had switched to different ePRO software.

    Table 3. Follow-up survey (N=78).

    		Publications
    NumberPercentage
    Clinical ePROa application still in use5064.1
    Patients participating per month

    		1-502040.0

    		51-2001530.0

    		201-50048.0

    		>5001020.0

    		Not reported12.0
    Satisfaction with clinical ePRO application
    Not at all00.0
    A little bit714.0
    Quite a bit2346.0
    Very much2040.0
    Purpose of ePRO use (multiple could apply)
    Research3672.0
    Monitoring tools (data collection over time with feedback of results to clinicians)3468.0
    Patient-centered care (data collection with feedback of PRO results to patients)3264.0
    Screening tools (one-time PRO assessment with feedback to clinician)1734.0
    Facilitating multidisciplinary team communications1734.0
    Decision aids (for shared decision-making)1530.0
    Evaluating quality of care1530.0
    Areas for improvementb(multiple could apply)
    Functionality (eg, reports, graphs, and reminders)1326.0
    Integration with electronic medical records918.0
    Patient and clinician engagement816.0
    User-friendliness and accessibility714.0
    None reported1428.0
    Clinical ePRO application not used anymore2835.9
    Reasons why (multiple could apply)
    Lack of funding1242.9
    Logistical or organizational challenges932.1
    Technical problems828.6
    Change to different software for ePRO assessment828.6
    Underutilization310.7
    Lack of user-friendliness27.1
    Change in clinical needs13.6
    Requirements for reconsidering ePRO useb (multiple could apply)

    		Funding or resources1346.4

    		Integration with existing clinical systems517.9

    		Improved usability414.3

    		Patient and clinician engagement27.1

    		None reported932.1

    aePRO: electronic patient-reported outcome.

    bThese questions were a free text field the answers to which we categorized.


    Overview

    In this study, we systematically reviewed studies reporting on ePRO assessment in clinical practice and present the results of a follow-up survey investigating the long-term use of the identified clinical ePRO applications. The results show a significant increase in publications on clinical ePRO applications over time including time trends towards app-based ePRO software, patient-centered care (data collection with feedback of PRO results to patients), and remote assessments. Information on the current use of identified ePRO applications could be gathered for about a third, of which 64% were reportedly still in use, primarily for monitoring, feedback of PRO results to patients, and research. Lack of funding was the main reason for discontinuation of ePRO application use, with more resources being the key requirement for reconsidering reuptake.

    ePRO Use in Clinical Practice and Time Trends

    The review highlights the growing role of clinical ePRO applications in routine care, especially for monitoring symptoms and HRQOL during treatment and early aftercare. Unlike research-focused data collection, PRO monitoring and subsequent management by health care providers can offer direct patient benefits [8,9]. Increased ePRO publications over the last decade [28,29] were likely spurred by 2 landmark trials in 2017 showing survival benefits from remote ePRO monitoring [30,31]. Basch et al [30] demonstrated overall survival benefits in patients with solid tumors who were monitored via an ePRO application, where nurses received alerts when patients reported severe symptoms. Similarly, Denis et al [31] reported an improved overall survival of the ePRO arm when compared to standard of care. These results highlighted the potential of ePROs to improve outcomes, possibly driving their broader adoption. Clinical ePROs are increasingly used to provide patients with feedback on their scores, enabling them to actively manage their health and make informed care decisions [7,32].

    Our follow-up survey found discrepancies between published reports of ePRO use and their actual use in clinical practice. This may be due to varied interpretations of survey categories, unreported use cases, or the evolving role of ePROs over time. For instance, quality of care evaluation was cited more in the survey (30%) than in studies (3%). Overall, our findings suggest that clinical ePRO use is broader and more diverse than reflected in current literature.

    Our findings also show that website interfaces remain dominant; however, mobile apps are becoming increasingly more prevalent in the field of oncology, as Lu et al [33] showed in their review of 41 symptom-tracking apps. Apps can offer more accessible, user-friendly interfaces for patients and have shown notable growth in utilization. For example, a study by Pho et al [34] demonstrated a significant increase in mobile app log-ins over time, particularly in younger patient populations, leading to a higher overall frequency of ePRO use compared to users of the website interface. Looking ahead, one might expect patient apps to become even more prevalent and ePROs to become more often integrated into electronic health record systems to facilitate access for health care professionals.

    Sustainability of Clinical ePRO Applications

    Participants in the follow-up survey reported that about two-thirds of the ePRO applications were still in use, with some applications being active for over 20 years. However, it is important to note that this represents only 23% of the total number of applications reviewed. Authors with ePRO applications still in active use may have been more likely to respond to our survey, potentially leading to an overestimation of sustainable use. Hence, the true rate of sustainable use likely falls between the 64% (50/78) of applications still in use according to our survey and the more conservative estimate of 23% (50/221) according to all applications initially identified. Additionally, other research shows that mobile apps in oncology are often no longer available once their respective research project is concluded [35].

    Interestingly, some centers that discontinued their original ePRO applications reported to have transitioned to different software solutions for continued ePRO assessment. For example, in Denmark and Ontario, Canada, ePRO assessment was implemented on a national (regional) level showing the possibility of evolving and adapting ePRO applications [36,37].

    Despite these encouraging examples, the sustainability of clinical ePRO applications remains a significant challenge, particularly due to financial constraints. The high initial costs of implementing clinical ePRO applications can often come without a clear direct return on investment, making it difficult to maintain these tools over the long term. Additionally, switching to another software or vendor can double these expenses, further straining budgets. The lack of follow-up financing was cited as the primary reason for discontinuing many applications. Some countries, such as France, are exploring reimbursement models for remote patient monitoring to offset these financial challenges that can also apply to ePRO applications in clinical care [38]. Notably, reimbursement has recently been granted for a clinical ePRO application in 33 cancer centers with an additional 18 about to follow, demonstrating the policy's potential impact on broader adoption [39].

    In addition to financial barriers, integrating ePRO assessment into electronic health records and clinical workflows, the need for appropriate user support, and the demand for features like enhanced data visualization and real-time feedback to improve sustainability are frequently mentioned challenges [18,40]. Correspondingly, a review of 15 ePRO systems in nephrology identified sustained patient involvement, clinician champions and expanding existing electronic platforms to integrate ePROs as important factors for a sustainable ePRO implementation [41].

    To address these challenges and promote long-term success, we recommend using established methodologies designed to support successful clinical implementation, where sustainability is a key outcome criterion (see, for example, [42-44]).

    Strength and Limitations

    Our paper offers valuable insights into the long-term use and sustainability of clinical ePRO applications, backed by both a comprehensive literature review and follow-up survey data. Another key strength lies in the novel exploration of time trends over the past 20 years that illustrate the use of clinical ePRO applications in oncology. The key limitation of the follow-up survey and the sustainability analysis is potential response bias. Potentially, authors whose ePRO application is still in use might be more prone to reply to the survey, thereby biasing the analysis. We also identified an increased likelihood of a response from clinical ePRO application with a higher total number of published articles which can be seen as an indicator for a bias towards more active research groups. Hence, we have interpreted our sustainability analysis based on all the applications initially identified and provide a more conservative estimate based on the total number of identified applications. Furthermore, we found that specific regions were overrepresented among respondents therefore biasing our results geographically.

    Inconsistent reporting of ePRO systems in the literature poses another challenge in comprehensively reporting on the current practices of ePRO applications in clinical care. The lack of a specific reporting framework (or limited compliance with more general guidelines like [45]) makes it difficult to compare studies and draw conclusions about best practices for ePRO implementation. However, ongoing efforts to develop specific guidelines for reporting patient-reported outcome measures in routine care offer hope for addressing this issue (Patient-Reported Outcome Measures—Guideline for Reporting In clinical Practice [46]).

    Conclusions

    Our study presents data on the sustainability and growing use of clinical ePRO applications in routine cancer care, with the potential to improve symptom monitoring and patient-centered care. While many applications remain in use, financial challenges and integration barriers might limit widespread adoption. Addressing these obstacles through reimbursement models, improved workflow integration, and enhanced support features will be essential for the sustained success of clinical ePRO applications in oncology.

    Acknowledgments

    No generative artificial intelligence tools were used to create the original content for publication. However, the authors declare that they have used generative artificial intelligence (ChatGPT-4o, OpenAI, 2025) to improve the readability and language of the work. All input was thoroughly checked for accuracy by the authors to ensure the integrity of the content. We acknowledge Open Access Funding provided by the Medical University of Innsbruck.

    Data Availability

    All data from the review are available as supplementary material. Data from the follow-up survey are available in anonymized form upon reasonable request from the authors.

    Authors' Contributions

    NJH involved in the conceptualization, methodology, project administration, investigation, data curation, formal analysis, writing—original draft, and writing—review & editing; SMV involved in the investigation, and writing—review & editing; KML involved in the investigation, and writing—review & editing; IR involved in the investigation and writing—review & editing; KB involved in the investigation and writing—review & editing; LMW involved in the investigation and writing—review & editing; JN involved in the data curation and writing—review & editing; BF involved in the data curation and writing—review & editing; BH involved in the supervision and writing—review & editing; MS involved in the conceptualization, methodology, investigation, supervision, resources, and writing—review & editing; and JL involved in the conceptualization, methodology, investigation, data curation, supervision, writing—original draft, and writing—review & editing.

    Conflicts of Interest

    BH reports intellectual property rights to the Computer-based Health Evaluation Software, outside of the submitted work. JL reports consultancy for Evaluation Software Development and a research grant from Takeda, both outside of the submitted work. NJH, JN, and BF report consultancy for Evaluation Software Development outside the submitted work. The other authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

    Multimedia Appendix 1

    Additional methods and material.

    PDF File (Adobe PDF File), 871 KB
    Multimedia Appendix 2

    Overview of the data extraction results (N=303).

    XLSX File (Microsoft Excel File), 270 KB
    Multimedia Appendix 3

    Number of publications over time (annual overview).

    PNG File , 28 KB
    1. Barbera L, Sutradhar R, Seow H, Earle CC, Howell D, Mittmann N, et al. Impact of standardized edmonton symptom assessment system use on emergency department visits and hospitalization: results of a population-based retrospective matched cohort analysis. JCO Oncol Pract. 2020;16(9):e958-e965. [CrossRef] [Medline]
    2. Lizán L, Pérez-Carbonell L, Comellas M. Additional value of patient-reported symptom monitoring in cancer care: a systematic review of the literature. Cancers (Basel). 2021;13(18):4615. [FREE Full text] [CrossRef] [Medline]
    3. Reilly CM, Bruner DW, Mitchell SA, Minasian LM, Basch E, Dueck AC, et al. A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment. Support Care Cancer. 2013;21(6):1525-1550. [FREE Full text] [CrossRef] [Medline]
    4. Barbera L, Sutradhar R, Seow H, Mittmann N, Howell D, Earle CC, et al. The impact of routine edmonton symptom assessment system (ESAS) use on overall survival in cancer patients: results of a population-based retrospective matched cohort analysis. Cancer Med. 2020;9(19):7107-7115. [FREE Full text] [CrossRef] [Medline]
    5. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22(4):714-724. [CrossRef] [Medline]
    6. Kotronoulas G, Kearney N, Maguire R, Harrow A, Di Domenico D, Croy S, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol. 2014;32(14):1480-1501. [FREE Full text] [CrossRef] [Medline]
    7. Lu S, Porter I, Valderas JM, Harrison CJ, Sidey-Gibbons C. Effectiveness of routine provision of feedback from patient-reported outcome measurements for cancer care improvement: a systematic review and meta-analysis. J Patient Rep Outcomes. 2023;7(1):54. [FREE Full text] [CrossRef] [Medline]
    8. Balitsky AK, Rayner D, Britto J, Lionel AC, Ginsberg L, Cho W, et al. Patient-reported outcome measures in cancer care: an updated systematic review and meta-analysis. JAMA Netw Open. 2024;7(8):e2424793. [CrossRef] [Medline]
    9. Perry MB, Taylor S, Khatoon B, Vercell A, Faivre-Finn C, Velikova G, et al. Examining the effectiveness of electronic patient-reported outcomes in people with cancer: systematic review and meta-analysis. J Med Internet Res. 2024;26:e49089. [FREE Full text] [CrossRef] [Medline]
    10. Di Maio M, Basch E, Denis F, Fallowfield LJ, Ganz PA, Howell D, et al. ESMO Guidelines Committee. Electronic address: clinicalguidelines@esmo.org. The role of patient-reported outcome measures in the continuum of cancer clinical care: ESMO clinical practice guideline. Ann Oncol. 2022;33(9):878-892. [FREE Full text] [CrossRef] [Medline]
    11. Allenby A, Matthews J, Beresford J, McLachlan SA. The application of computer touch-screen technology in screening for psychosocial distress in an ambulatory oncology setting. Eur J Cancer Care (Engl). 2002;11(4):245-253. [CrossRef] [Medline]
    12. Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res. 2001;10(1):15-22. [CrossRef] [Medline]
    13. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211. [FREE Full text] [CrossRef] [Medline]
    14. Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of Internet versus paper-and-pencil versions. J Med Internet Res. 2007;9(3):e25. [FREE Full text] [CrossRef] [Medline]
    15. Basch E, Barbera L, Kerrigan CL, Velikova G. Implementation of patient-reported outcomes in routine medical care. Am Soc Clin Oncol Educ Book. 2018;38:122-134. [FREE Full text] [CrossRef] [Medline]
    16. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510-520. [FREE Full text] [CrossRef] [Medline]
    17. LeBlanc TW, Abernethy AP. Patient-reported outcomes in cancer care - hearing the patient voice at greater volume. Nat Rev Clin Oncol. 2017;14(12):763-772. [CrossRef] [Medline]
    18. Laitio AM, Giunti G, Halonen R. Perceived barriers and facilitators in using patient-reported outcome systems for cancer care: systematic mapping study. JMIR Cancer. 2023;9:e40875. [FREE Full text] [CrossRef] [Medline]
    19. Anatchkova M, Donelson SM, Skalicky AM, McHorney CA, Jagun D, Whiteley J. Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature. J Patient Rep Outcomes. 2018;2(1):64. [FREE Full text] [CrossRef] [Medline]
    20. Consolo L, Castellini G, Cilluffo S, Basile I, Lusignani M. Electronic patient-reported outcomes (e-PROMs) in palliative cancer care: a scoping review. J Patient Rep Outcomes. 2022;6(1):102. [FREE Full text] [CrossRef] [Medline]
    21. Warrington L, Absolom K, Conner M, Kellar I, Clayton B, Ayres M, et al. Electronic systems for patients to report and manage side effects of cancer treatment: systematic review. J Med Internet Res. 2019;21(1):e10875. [FREE Full text] [CrossRef] [Medline]
    22. Lai-Kwon J, Cohen JE, Lisy K, Rutherford C, Girgis A, Basch E, et al. The feasibility, acceptability, and effectiveness of electronic patient-reported outcome symptom monitoring for immune checkpoint inhibitor toxicities: a systematic review. JCO Clin Cancer Inform. 2023;7:e2200185. [CrossRef] [Medline]
    23. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19-32. [CrossRef]
    24. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-473. [FREE Full text] [CrossRef] [Medline]
    25. Systematic Review and Literature Review Software. DistillerSR. URL: https://www.distillersr.com/ [accessed 2025-01-10]
    26. Snyder C, Aaronson N, Choucair A, Elliott T, Greenhalgh J, Halyard M, et al. User's guide to implementing patient-reported outcomes assessment in clinical practice. Int Soc Qual Life Res. 2015;2:1-47. [CrossRef]
    27. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria. R Foundation for Statistical Computing; 2023.
    28. Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin. 2012;62(5):337-347. [FREE Full text] [CrossRef] [Medline]
    29. Jensen RE, Snyder CF, Abernethy AP, Basch E, Potosky AL, Roberts AC, et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract. 2014;10(4):e215-e222. [FREE Full text] [CrossRef] [Medline]
    30. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. J Am Med Assoc. 2017;318(2):197-198. [FREE Full text] [CrossRef] [Medline]
    31. Denis F, Lethrosne C, Pourel N, Molinier O, Pointreau Y, Domont J, et al. Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients. J Natl Cancer Inst. 2017;109(9):djx029. [CrossRef] [Medline]
    32. Kim SH, Sung JH, Yoo S, Kim S, Lee K, Oh EG, et al. Effects of digital self-management symptom interventions on symptom outcomes in adult cancer patients: a systematic review and meta-analysis. Eur J Oncol Nurs. 2023;66:102404. [CrossRef] [Medline]
    33. Lu DJ, Girgis M, David JM, Chung EM, Atkins KM, Kamrava M. Evaluation of mobile health applications to track patient-reported outcomes for oncology patients: a systematic review. Adv Radiat Oncol. 2021;6(1):100576. [FREE Full text] [CrossRef] [Medline]
    34. Pho KK, Lu R, Gates S, Cai J, Xie D, Xie Y, et al. Mobile device applications for electronic patient portals in oncology. JCO Clin Cancer Inform. 2019;3:1-8. [FREE Full text] [CrossRef] [Medline]
    35. Ana FA, Loreto MS, José LMM, Pablo SM, María Pilar MJ, Myriam SA. Mobile applications in oncology: a systematic review of health science databases. Int J Med Inform. 2020;133:104001. [CrossRef] [Medline]
    36. Barbera L, Moody L. A decade in review: cancer care Ontario's approach to symptom assessment and management. Med Care. 2019;57(5):S80-S84. [CrossRef] [Medline]
    37. Egholm CL, Jensen S, Wandel A, Hørder M. The implementation of the 2017 national policy on patient-reported outcomes in Denmark: an overview of developments after six years. Health Policy. 2023;130:104755. [FREE Full text] [CrossRef] [Medline]
    38. Vanier A, Fernandez J, Kelley S, Alter L, Semenzato P, Alberti C, et al. Rapid access to innovative medicinal products while ensuring relevant health technology assessment. Position of the French National Authority for Health. BMJ Evid Based Med. 2024;29(1):1-5. [FREE Full text] [CrossRef] [Medline]
    39. Franzoi MA, Ferreira AR, Lemaire A, Rodriguez J, Grosjean J, Ribeiro JM, Nouhaud, et al. Implementation of a remote symptom monitoring pathway in oncology care: analysis of real-world experience across 33 cancer centres in France and Belgium. Lancet Reg Health Eur. 2024;44:101005. [FREE Full text] [CrossRef] [Medline]
    40. Basch E, Rocque G, Mody G, Mullangi S, Patt D. Tenets for implementing electronic patient-reported outcomes for remote symptom monitoring during cancer treatment. JCO Clin Cancer Inform. 2023;7:e2200187. [CrossRef] [Medline]
    41. Anderson NE, Kyte D, McMullan C, Cockwell P, Aiyegbusi OL, Verdi R, et al. Global use of electronic patient-reported outcome systems in nephrology: a mixed methods study. BMJ Open. 2023;13(7):e070927. [FREE Full text] [CrossRef] [Medline]
    42. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123-157. [FREE Full text] [CrossRef] [Medline]
    43. Stover AM, Haverman L, van Oers HA, Greenhalgh J, Potter CM, ISOQOL PROMs/PREMs in Clinical Practice Implementation Science Work Group. Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings. Qual Life Res. 2021;30(11):3015-3033. [FREE Full text] [CrossRef] [Medline]
    44. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21(8):1305-1314. [CrossRef] [Medline]
    45. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348:g1687. [FREE Full text] [CrossRef] [Medline]
    46. Reporting guidelines under development for other study designs. EQUATOR Network. URL: https:/​/www.​equator-network.org/​library/​reporting-guidelines-under-development/​reporting-guidelines-under-development-for-other-study-designs/​#PROM [accessed 2024-09-24]

    ‎
    EORTC: European Organisation for Research and Treatment of Cancer
    ePRO: electronic patient-reported outcome
    HRQOL: health-related quality of life
    ISOQOL: International Society for Quality of Life Research
    OR: odds ratio
    PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews
    PRO: patient-reported outcome
    PRO-CTCAE: Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events

    Edited by J Sarvestan; submitted 28.11.24; peer-reviewed by I Kudel, A Ojaruega; comments to author 03.01.25; revised version received 24.01.25; accepted 04.04.25; published 25.04.25.

    Copyright

    ©Niclas J Hubel, Samuel M Vorbach, Kelly M de Ligt, Ines S Rathgeber, Katharina Beyer, Lisa M Wintner, Barbara Faller, Jasmin Nemec, Bernhard Holzner, Monika Sztankay, Jens Lehmann. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 25.04.2025.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.