This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289) Innovations and Technology in Cancer Care (423) Patient-Reported Outcome Measures (PROMs) (288)

Published on in Vol 21, No 1 (2019): January

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10875, first published .
Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review

Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review

Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review

Authors of this article:

Lorraine Warrington1 Author Orcid Image ;   Kate Absolom1 Author Orcid Image ;   Mark Conner2 Author Orcid Image ;   Ian Kellar2 Author Orcid Image ;   Beverly Clayton1 Author Orcid Image ;   Michael Ayres3 Author Orcid Image ;   Galina Velikova1 Author Orcid Image
Article Authors Cited by (119) Tweetations (62) Metrics

    Review

    1Section of Patient Centred Outcomes Research, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, United Kingdom

    2School of Psychology, University of Leeds, Leeds, United Kingdom

    3Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom

    Corresponding Author:

    Lorraine Warrington, BSc, MSc, PhD

    Section of Patient Centred Outcomes Research

    Leeds Institute of Cancer and Pathology

    University of Leeds

    Beckett Street

    Leeds, LS9 7TF

    United Kingdom

    Phone: 44 1132068504

    Email: l.warrington@leeds.ac.uk


    Background: There has been a dramatic increase in the development of electronic systems to support cancer patients to report and manage side effects of treatment from home. Systems vary in the features they offer to patients, which may affect how patients engage with them and how they improve patient-centered outcomes.

    Objective: This review aimed to (1) describe the features and functions of existing electronic symptom reporting systems (eg, symptom monitoring, tailored self-management advice), and (2) explore which features may be associated with patient engagement and patient-centered outcomes.

    Methods: The review was registered with the International Prospective Register of Systematic Reviews (PROSPERO) and followed guidelines from the Centre for Reviews and Dissemination (University of York, United Kingdom). Primary searches were undertaken of MEDLINE, Embase, PsycInfo, Web of Science, Cochrane Central Register of Controlled Trials, and the Health Technology Assessment databases. Secondary searches were undertaken by screening reference lists and citations. Two researchers applied broad inclusion criteria to identify and select relevant records. Data were extracted and summarized using Microsoft Excel. In order to meet the aims, the study selection, data extraction, and data synthesis comprised two stages: (1) identifying and characterizing available systems and (2) summarizing data on patient engagement and patient-centered outcomes.

    Results: We identified 77 publications relating to 41 distinct systems. In Stage 1, all publications were included (N=77). The features identified that supported clinicians and care were facility for health professionals to remotely access and monitor patient-reported data (24/41, 58%) and function to send alerts to health professionals for severe symptoms (17/41, 41%). Features that supported patients were facility for patients to monitor/review their symptom reports over time (eg, graphs) (19/41, 46%), general patient information about cancer treatment and side effects (17/41, 41%), tailored automated patient advice on symptom management (12/41, 29%), feature for patients to communicate with the health care team (6/41, 15%), and a forum for patients to communicate with one another (4/41, 10%). In Stage 2, only publications that included some data on patient engagement or patient-centered outcomes were included (N=29). A lack of consistency between studies in how engagement was defined, measured, or reported, and a wide range of methods chosen to evaluate systems meant that it was not possible to compare across studies or make conclusions on relationships with system features.

    Conclusions: Electronic systems have the potential to help patients manage side effects of cancer treatment, with some evidence to suggest a positive effect on patient-centered outcomes. However, comparison across studies is difficult due to the wide range of assessment tools used. There is a need to develop guidelines for assessing and reporting engagement with systems, and a set of core outcomes for evaluation. We hope that this review will contribute to the field by introducing a taxonomy for characterizing system features.

    Trial Registration: PROSPERO CRD42016035915; www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016035915

    J Med Internet Res 2019;21(1):e10875

    doi:10.2196/10875

    Keywords

    oncology; chemotherapy; patient reported outcomes; patient centered; medical informatics 


    Increased efficacy of cancer treatments has led to a rising global population of people living with and beyond cancer. Effective multimodal cancer treatments can slow disease progression, ease the symptoms of the disease, and in some cases cure disease altogether. However, treatments can cause a vast array of side effects such as nausea, pain, fatigue, and diarrhea, which may negatively affect a patient’s quality of life (QoL) and may even become life-threatening, with severe cases such as neutropenic infections. Many cancer treatments are delivered in an ambulatory setting and methods of follow-up and support are highly variable dependent on disease, treatment type, and local practice and resources. Information is commonly provided by the health care team on expected and possible side effects, and patients are advised to seek help if symptoms become a cause for concern. However, patients may not always be able to fully absorb this information at the time it is provided [1] or feel confident in making decisions on when additional hospital contact is necessary between routine clinical reviews [2]. Furthermore, clinicians are mainly reliant on interpreting patient retrospective reports of treatment side effects to ensure safety of care and manage supportive medications. Side effects are not often documented in medical records in a consistent and comparable way [3].

    Over the past decade, there has been a dramatic increase in the number of electronic systems developed to support patients during and after cancer treatment by using patient-reported outcome measures (PROMs) to remotely assess symptoms [4-8]. The routine use of PROMs in oncology care as a strategy to enhance symptom monitoring has demonstrated many benefits, such as improved communication between clinicians and patients, and better symptom awareness [9]. Using electronic systems to collect and manage PROMs data has the potential to overcome some of the common challenges previously associated with collating data collected on paper. More recently developed systems can be accessed from any Web-enabled device, allowing patients to report symptoms from home using their own electronic devices such as computers, tablets, or mobile phones. This can be done in real time, rather than relying on retrospective reporting and potentially allows automated documentation of patient reports in the medical record [10].

    There is considerable variation in the features offered by symptom reporting systems. Some primarily focus on making symptom data routinely available to health professionals and provide alerts when severe symptoms have been reported [5,11-15]. Others have been developed with a greater focus on patient self-management, delivering tailored and automated self-management advice when appropriate, and advising patients to contact their health care team when necessary [8,16-20]. Some systems use a combination of both approaches [4] and can also include additional features such as facilitating communication with medical teams or other patients.

    The availability or absence of certain features may affect how patients engage with systems [21,22]. The terms “engagement” and “adherence” are often used interchangeably in this context. However, adherence suggests an optimal way to use a technology and this is not always easy to define [23]. For the purposes of this review, we refer to engagement in a broad sense of levels of patient usage of the technology. Understanding the key components that can enhance patient engagement with electronic symptom reporting is potentially crucial for improving the development of future systems and encouraging their implementation into standard practice. There are many factors that are likely to have an impact, from individual differences [24], socioeconomic status and healthy literacy [25], to basic system usability [21,26]. There is relatively little currently known about the underlying processes and particularly the role that the availability of systems features might play. However, there is evidence to suggest that individuals vary in the features that they value and use most [20]. In addition, needs may change over time, as patients become more experienced with the system, but also with their disease and treatment [27].

    The presence or absence of system features is also likely to affect the level of patient benefit gained from using the system. For example, changes in behavior or disease outcome have been more often observed with interactive interventions in comparison with those that are purely educational [28]. While the use of interactive online systems is associated with greater self-efficacy, better self-management, and more participation in health care [29-32], this may be associated only with specific features such as interactive communication and progress tracking features [33], and consultation and self-management support [34].

    Systematic reviews traditionally focus on high-quality evidence for a specific research question. However, increasingly, the value of taking a broader approach to inclusion is being recognized as important to answer complex research questions, particularly in the emerging field of online health interventions [35,36]. With this in mind, the focus of this review was to take an inclusive approach to systematically review and describe the features and functions of existing systems. We also wanted to focus on understanding the level of evidence indicating whether key system features are associated with better patient system engagement and patient-centered outcomes.

    The aims of this systematic review are to (1) describe the features and functions of existing electronic symptom reporting systems developed for patients during cancer treatment, and (2) explore which features of these systems may be associated with patient engagement and outcomes. Specifically, we wanted to summarize (1) patient engagement and whether this is related to specific system features (eg, symptom monitoring, tailored self-management advice), and (2) patient-centered outcomes used to evaluate systems and whether better outcomes are associated with specific features.


    Protocol and Registration

    Details of the protocol were registered on the International Prospective Register of Systematic Reviews (PROSPERO) database [37]. There were no major deviations from the protocol. However, study selection, data extraction, and data synthesis comprised two stages: (1) identifying and characterizing available systems, and (2) summarizing data on patient engagement and patient-centered outcomes. This staged approach was not initially planned but was necessary in order to meet the aims of the review.

    Eligibility Criteria

    The review question was refined using Population, Intervention, Comparator, Outcomes, Study design (PICOS) criteria (Table 1), and eligibility criteria were developed based on this. For Stage 1, we wanted an overview of all systems available, so all relevant publications including published abstracts, protocols, and qualitative studies were included. However, discussion papers or systematic reviews were excluded. For Stage 2, in order to review evidence available on patient engagement and any patient-centered outcomes, we wanted to include feasibility studies with any evaluation data of patient use, rather than restricting criteria to randomized controlled trials (RCTs) only. Only full papers were included in this stage. Criteria were piloted by 2 researchers (LW and KA) on a subset of 10 randomly selected papers and subsequently refined and clarified before the next stage.

    Information Sources

    Studies were identified from systematic searches of Medline, Embase, PsycInfo, Web of Science, Cochrane Central Register of Controlled Trials, and the Health Technology Assessment databases in March 2016. Due to the nature of the review, results were limited to those published after 2000. No restrictions were imposed on language of publication. Searches were updated on September 12, 2017. Reference lists of relevant publications were screened to identify papers not picked up by the electronic searches. In addition, citations of selected key papers were searched.

    Search Strategy

    A detailed example of the search strategy used for Medline is outlined in Textbox 1. This search strategy was adapted for each of the databases.

    Study Selection

    For initial screening, a decision for inclusion was made based on title and where available, abstract. This was carried out by one researcher (LW) only, and for this reason, a cautious approach erring on the side of over-inclusion was used. Following this, 2 researchers (LW and KA) independently assessed all remaining papers for relevance. Disagreements were resolved by consensus after referring to the protocol. All discussions and decision making were documented. Where there was insufficient information to make a decision, authors were contacted for further information. If no response was received within 2 weeks, a final decision was made based on available information.

    Table 1. PICOS (Population, Intervention, Comparator, Outcomes, Study) criteria.
    CategoryCriteria
    PopulationMale and female adults >18, no upper age limit, worldwide with any cancer diagnosis, receiving cancer treatment OR within 3 months of completing treatment. The cancer treatment to include any treatment with significant side effects (eg, systemic therapies, radiotherapy, biological therapies).
    InterventionOnline systems for patients to report or manage symptoms and side effects during cancer treatment from home; Internet-based or -enabled systems, including mobile apps. Other forms of interactive health communication applications, eg DVDs, games were excluded. Purely educational systems not interactive in any way were excluded. Systems developed to assess and monitor purely psychosocial symptoms were excluded (eg, depression, anxiety, emotional coping or stress). Sleep and fatigue were included. Systems designed to be accessed at one time point only were excluded; access to the system had to be ongoing.
    ComparatorStage 2 only: The review included studies with any comparator (eg, randomized or nonrandomized studies), in addition to studies with no comparator (eg, feasibility studies).
    OutcomesStage 1: Dependent on the nature and number of papers found, we aimed to characterize systems. For example, we identified if studies included features such as Monitoring of symptoms by health care professionals (HCPs), Alerts for severe symptoms sent to HCPs, Monitoring of symptoms by patients (eg, graphical or tabular), Automated feedback/advice based on responses, Access to symptom information, Communication with other cancer patients, Direct communication with HCPs (distinct from symptom monitoring by HCPs).
    Stage 2: We aimed to collect where available, information on engagement with systems and information on any patient-centered outcomes, including but not restricted to any QoL measures; self-efficacy measures including patient activation, patient empowerment, mastery; and patient satisfaction.
    Study designStage 2 only: The review was not restricted to randomized controlled trials, and feasibility studies with any evaluation data were included. Patients had to be using the system over time, and there had to be at least one intended time point of use more than 3 weeks after baseline. This timeframe was selected as many standard chemotherapy treatments are administered every 3 weeks.
    Example of search strategy used (Ovid Medline).
    1. Neoplasms/
    2. oncolog*.mp.
    3. cancer patient*.mp.
    4. 1 or 2 or 3
    5. Medical Informatics/
    6. Telemedicine/
    7. Mobile Applications/
    8. Smartphone/
    9. Self Report/
    10. Self Care/
    11. Self-Assessment/
    12. (electronic adj2 (Patient report* or Patient-report* or Self report* or Self-report* or Self manage* or Self-manage* or Self monitor* or Self-monitor* or Symptom report* or Symptom-report* or Symptom manage* or Symptom-manage*)).mp.
    13. (online adj2 (Patient report* or Patient-report* or Self report* or Self-report* or Self manage* or Self-manage* or Self monitor* or Self-monitor* or Symptom report* or Symptom-report* or Symptom manage* or Symptom-manage*)).mp.
    14. (web* adj2 (Patient report* or Patient-report* or Self report* or Self-report* or Self manage* or Self-manage* or Self monitor* or Self-monitor* or Symptom report* or Symptom-report* or Symptom manage* or Symptom-manage*)).mp.
    15. (remote* adj2 (Patient report* or Patient-report* or Self report* or Self-report* or Self manage* or Self-manage* or Self monitor* or Self-monitor* or Symptom report* or Symptom-report* or Symptom manage* or Symptom-manage*)).mp.
    16. 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15
    17. 4 and 16

    Limit 17 to (humans and yr=“2000 -Current”)

    Textbox 1. Example of search strategy used (Ovid Medline).

    Data Items

    For Stage 1, basic data were extracted on authors, title, year of publication, and country of origin, in addition to the name (if any given) and type of system being described (eg, Web-based or mobile app). If the system did not already have a descriptive name, an arbitrary name was assigned (eg, System A). A preliminary list of common features was created based on existing knowledge and further developed throughout data extraction until a comprehensive list of common or important features was identified. Data were extracted from each publication on the presence of each feature. This was coded as “Yes” only if it was explicitly described in the publication, otherwise it was coded as a “#” For abstracts, if it was unclear whether or not a feature was present by information available in an abstract, this was classed as “Unable to determine.” Where information was lacking, authors were not contacted for information. However, searches were undertaken for other publications related to the same system.

    For Stage 2, data were extracted from studies with some form of system evaluation (eg, patient use of system or evaluation of efficacy). This included data on the number of patient participants, baseline demographics, disease and treatment type, duration of the evaluation, methods used to assess engagement, and actual usage or adherence. Where available, data were also extracted on any patient-centered outcomes used and results of evaluation.

    Data Extraction

    Data were extracted using the online Systematic Review Data Repository [38]. The form was piloted on 10 randomly selected papers and further refined. For Stage 1, three additional researchers (KA, BC, MA) each double-coded a number of allocated publications, totaling 36% (27/77) of the overall included publications. A high level of agreement (86%) was found. Discrepancies were resolved by referring back to the protocol and additional publications where available. For Stage 2, the same 3 researchers again each double-coded a proportion of the included publications totaling 46% (13/29) and 100% agreement was found.

    Quality Assessment

    Quality was assessed using the Downs and Black checklist for nonrandomized studies [39] and was undertaken alongside data extraction. It was deemed appropriate to assess only studies that included some feasibility/evaluation data, that is, publications included in Stage 2. Studies were given a score along a possible range of 0-26.

    Synthesis of Results

    A narrative synthesis was undertaken using the guidelines outlined by the Economic and Social Research Council [40]. Microsoft Excel was used to manage data. For Stage 1, information from multiple publications relating to the same systems was pooled to form a description of features. Where information was conflicting due to earlier and later iterations, the most recent description was used. For Stage 2, information was collected on how patient engagement was assessed for any feasibility study or trial that included these data. For trial studies, information was collected on primary and secondary study outcomes and any results recorded. We then summarized these data to explore any relationships with system features identified in Stage 1.


    Study Selection

    An overview of search and selection procedures is outlined in Figure 1. A total of 6727 publications were identified after removal of duplicate publications, including two publications identified from secondary searches (ie, citation and reference lists). All publications were in English. We assessed 279 publications for eligibility, and a total of 202 papers were excluded at this point based on predefined eligibility criteria (intervention, eg, not home-based or Web-based, n=132; population, eg, patients not on active treatment, n=41; discussion paper or systematic review, n=19; or abstract unavailable, n=10). We included 77 publications in Stage 1 of the review (ie, systems descriptions). A large proportion (23/77, 30%) of these publications were abstracts. The reasons for exclusions are outlined in Figure 1. Those 8 publications categorized under “Other” included 2 summary papers giving an overview of development and evidence for a system, a description of standard usability testing, a cost-effectiveness analysis, a content analysis of email communication within a system, a discussion of design approaches and methodology, an evaluation focusing on blood monitoring, and one publication where we were not able to access the full paper and did not receive a response from the authors when this was requested. We identified 29 publications for inclusion in Stage 2 of the review (ie, patient engagement and evaluation of systems). These were 21 feasibility studies and 8 controlled trials (7 randomized and 1 nonrandomized).

    Stage 1: Description of Systems and Features

    The 77 publications referred to 41 individual systems. Most originated from the United States (19/41, 46%) or the United Kingdom (6/41, 15%), and all publications were available in English. Systems were commonly Web-based (24/41, 56%), 27% (11/41) were mobile apps, 2 were both mobile and Web-based (2/41, 5%), and 22% (9/41) were Web-enabled mobile devices purposely designed for symptom reporting and were provided to patients for the duration of the study.

    ‎
    Figure 1. Summary of papers identified and subsequently excluded/included in this review.
    View this figure

    Seven common system features were identified. Figure 2 outlines each of the features and its prevalence in the 41 identified systems. Features could be categorized broadly as supporting patients to monitor and manage their own symptoms, and to communicate with health professionals and one another, or supporting clinicians to monitor and manage patient symptoms.

    Table 2 [4-8,11-20,41-102] provides an overview of each identified system and its associated publications, in addition to the presence or absence of each of the features identified in Figure 2.

    Stage 2: Patient Engagement and Evaluation

    Quality Assessment

    Along a possible range of 0-26, the overall median quality assessment score of studies using the Downs and Black checklist was 17.0 (mean 16.2, SD 5.3, range 2-24). For trials described in the section on patient-centered outcomes [5,6,8,49,60,79,88,100], the median score was higher at 20.0 (mean 20.4, SD 2.6, range 17-24).

    Patient Engagement

    Table 3 [5,6,8,11-15,42,43,49,60,63,65,68,73-75,79,81,82,84,87, 88,90,92,93,100,101] summarizes data on patient engagement from the 29 included studies (ie, 21 feasibility studies, 7 RCTs, and one non-RCT [88]). All 21 feasibility studies (100%) reported some data on patient engagement, although there was variation in how engagement was defined and measured. Three of the eight trials (38%) did not report any data on patient engagement [6,79,100].

    Of the 29 studies, the most common method of assessing engagement was the number of symptom report completions or number of times the system was accessed (12/29, 41%) [15,49,60,63,65,68,74,87,88,90,92]. This was given as an overall figure for the whole sample [15,49,68,90,92], as an average per patient [13,15,65,74,90], or with a breakdown of the variance [63,87]. Nine studies (9/29, 31%) assessed adherence by number of actual completions/accesses in comparison to the number of expected completions/accesses [5,13,14,73,75,81,84,93,101]. This was reported as median or mean adherence of the overall sample for the duration of the study period [2,73,75,81,93,101], or with a breakdown of adherence at different time points [14,84]. Only 2 studies studies (2/29, 7%) categorized patients as users or nonusers dependent on predefined criteria [11,12]. Four studies (4/29, 14%) combined results of patients reporting from home and in clinic [11,13-15]. Not all studies reported on actual usage, and some used evaluation questionnaires with or without semistructured interviews to assess acceptability to patients [42,43,65,82].

    Due to the variation in the methods of reporting, it was not possible to determine if there was any overall association between engagement and specific system features.

    ‎
    Figure 2. Overall summary of prevalence of identified system features.
    View this figure
    Table 2. Identified systems with description of features and associated publicationsa.
    System name (country)and typePublication type (with relevant references)Allowed health professional to remotely access and monitor patient reported dataAllowed patients to monitor their symptom reports over time (eg, graphs)Included a function to send alerts to health professional for severe symptomsProvided tailored automated patient advice on managing symptomsProvided general patient info about cancer treatment and side effectsIncluded a feature for patients to communicate with the health care teamIncluded a forum for patients to communicate with one another
    ASyMs (UK)
    Mobile device    		Randomized trial [6], Secondary analysis of RCTb [41], Feasibility studies [42,43], Abstracts [44-47], Other [48]✓x✓✓xxx
    CASSY (USA)
    Web-based    		Randomized trial [49]x✓xx✓x✓
    CHES (Austria)
    Web-based    		Abstract [50]–––––––
    COPE-CIPN (USA)
    Web-based    		Other [51]–––––––
    CORA (USA)
    Mobile app    		Development paper [52], Protocol [53]x✓x✓✓xx
    eRAPID (UK)
    Web-based    		Protocol [4], Abstracts [54-59]✓✓✓✓✓xx
    eSMART (UK)
    Mobile device    		Protocol [7]✓✓✓✓✓xx
    ESRA-C (USA)
    Web-based    		Randomized trial [60], Secondary analysis of RCT [61], Qualitative paper [62]x✓x✓xxx
    Healthweaver (USA)
    Web-based & mobile app    		Feasibility study [63], Development paper [64]x✓xx✓xx
    HSM (UK)
    Mobile device    		Feasibility study [65]✓x✓✓✓xx
    ICT-FP7 (France)
    Mobile device    		Abstract [66]✓––––––
    INTERAKTOR (Sweden)
    Web-based & Mobile app    		Protocol [67]✓✓✓✓✓xx
    KAIKU (Finland)
    Web-based    		Feasibility study [68]✓xxxx✓x
    MADELINE (USA)
    Mobile app    		Abstract [69]––––––✓>
    MSKCC WebCore (USA)
    Web-based    		Abstract [70]–––––––
    Onco-TREC (Italy)
    Mobile app    		Development paper [71], Protocol [72]✓✓✓✓x✓x
    PatientViewpoint (USA)
    Web-based    		Feasibility study [73]✓✓✓xxxx
    PaTOS (USA)
    Web-based    		Feasibility study [74]✓xxxxxx
    Pit-a-pit (Korea)
    Mobile app    		Feasibility study [75]✓xxxxxx
    PRISMS (Australia)
    Mobile device    		Protocol [76], Abstract [77]✓✓✓✓✓xx
    PROCDIM (USA)
    Web-based    		Abstract [78]✓✓–––––
    QoC Health Inc (Canada)
    Mobile app    		Randomized trial [79], Other [80]✓x✓xxxx
    RemeCoach (Belgium)
    Mobile device    		Feasibility study [81]xx✓xxxx
    SCMS (Singapore)
    Web-based    		Feasibility study [82], Other [83]✓xxx✓✓x
    STAR (USA)
    Web-based    		Randomized trial [5], Feasibility studies [11-15,84]x✓✓✓xxx
    The Health Buddy (R) (USA)
    Mobile device    		Development paper [85]✓x✓✓xxx
    WebChoice (Norway)
    Web-based    		Randomized trial [8], Secondary analysis of RCT [17,18], Qualitative paper [20], Other [16,19]x✓x✓✓✓✓
    WRITE (USA)
    Web-based    		Abstract [86]✓––✓–––
    System A (USA)
    Web-based    		Feasibility study [87]xx✓xxxx
    System B (The Netherlands)
    Web-based    		Nonrandomized trial [88], Development paper [89], Feasibility study [90]✓✓✓x✓✓✓
    System C (USA)
    Web-based    		Other [91]–––––––
    System D (Sweden)
    Mobile app    		Feasibility study [92]✓✓✓✓✓xx
    System E (UK)
    Mobile device    		Feasibility study [93]✓✓✓✓xxx
    System F (Canada)
    Web-based    		Abstract [94,95]–✓–✓–✓–
    System G (Denmark)
    Web-based    		Abstract [96]–✓–✓–––
    System H (UK)
    Mobile device    		Other [97]✓x✓xxxx
    System I (USA)
    Web-based    		Abstract [98]–––––––
    System J (USA)
    Web-based    		Abstract [99]✓––––––
    System K (Switzerland)
    Mobile app    		Randomized trial [100]✓✓xxxxx
    System L (USA)
    Mobile app    		Feasibility study [101]✓xxxxxx
    System M (USA)
    Mobile app    		Abstract [102]–––––––

    a“✓” denotes feature is present, “x” denotes feature is not present, and “–“ denotes that it was not possible to determine whether feature was present or not.

    bRCT: randomized controlled trial.

    Patient-Centered Outcomes

    All the trials used some measure of patient-centered outcome to evaluate system efficacy, most commonly validated QoL and symptom and psychosocial outcome measures. Table 4 outlines each trial [5,6,8,49,60,79,88,100], the intervention and comparator groups, outcomes reported, and a summary of the results.

    Global Quality of Life

    CASSY [49] and STAR [5] interventions both demonstrated improvements in overall QoL. However, in addition to the online component, CASSY included access to a collaborative care coordinator with experience in cognitive behavioral therapy and psycho-oncology, which is likely to have contributed to the efficacy. In the STAR study, patients were allocated to computer-experienced and inexperienced groups prior to randomization and only the computer-experienced group had access to the system from home. Results are pooled, making it difficult to assess efficacy for our purposes. No significant impact on QoL was found for WebChoice [8].

    Physical Symptoms

    An overall reduction of symptom distress was found in the studies assessing Electronic Self-Report Assessment-Cancer (ESRA-C) [60] and WebChoice [8]. However, in addition to the online intervention, ESRA-C also included a communication coaching component to improve symptom disclosure to physicians. System B [88] was found to have significant positive impact on the general physical complaints subscale compared to the control group.

    Advanced Symptom Management System (ASyMs) [6] and Comprehensive Electronic Cancer Support System for the Treatment of Cancer Related Symptoms (CaSSY) [49] both demonstrated positive impact on levels of fatigue while System K [100] demonstrated a lesser decline in functional activity in contrast to the control group, but this was not significant. Both ASyMs and System K were evaluated using the same measure as used to assess symptoms in the intervention, which may have affected results.

    Self-Efficacy

    WebChoice [8] and System B [88] both demonstrated a positive impact on self-efficacy. However, for System B, this was assessed only as a subscale of a main measure. System K [100] reported an improvement in patient empowerment; however, this was assessed using a single item regarding using the Internet for information seeking, which is unlikely to be a reliable measure.

    Other Psychosocial Outcomes

    CASSY [49] and WebChoice [8] demonstrated significant reductions in depression in intervention compared to control groups. System B [88] demonstrated no difference on the depression subscale of a QoL measure but a significant impact on state anxiety and fear related to specific head and neck problems. WebChoice demonstrated no impact on social support [8]. QoC Health Inc [79] was primarily assessed on number of hospital contacts but also included patient scores of convenience and satisfaction using a simple 5-point Likert scale and found an impact for convenience, but not for patient satisfaction.

    Due to the considerable variation in outcomes used and study design, it was not possible to assess any relationships between outcomes and system features.

    Table 3. Overview of patient engagement data.
    System name, patient group (patients, N), treatment type and study duration, quality assessment score (QAS)Method of evaluation/ patient engagementBrief summary of findings
    Feasibility studies (n=21)

    		ASyMS-R [42], Lung (N=16)
    During and 1 month after thoracic radiotherapy
    QAS=19    		Evaluation questionnaire and semistructured interviewsActual usage not reported
    Patients perceived it to positively impact on care and promote timely reporting and management of symptoms

    		ASyMS [43], Colorectal or lung (N=18)
    During 2 cycles of chemo
    QAS=15    		Evaluation questionnaireActual usage not reported
    Patients reported it helped monitor symptoms, promote self-care, and improve symptom management

    		HealthWeaver [63], Breast (N=9)
    Undergoing active treatment, 4 weeks
    QAS=8    		# of completions/ accessesAll patients used website at least 3x/week, 7 patients used it almost daily
    Phone component used almost daily by 5 patients, 3x/week by 1 patient, and 1-2x/week by 3 patients

    		HSM [65], Lung or colorectal (N=18)
    During 2 cycles of chemo
    QAS=10    		# of completions/ accesses and evaluation questionnairesAll patients completed 1-34 symptom reports, average 14 overall (SD 10.2)
    High variation in use of self-management advice
    Patients found system easier to use and more useful than expected

    		Kaiku [68], Head & neck (N=5)
    Radiotherapy, during and 1 month after
    QAS=12    		# of completions/ accesses514 symptoms reported (including zero grades)
    23 questionnaires completed
    38 messages sent

    		PatientViewpoint [73], Breast or prostate (N=47)
    Medical oncology treatment
    UTD - 3 onsite visits (not specified)
    QAS=15    		# of accesses/ expected accesses190/224 symptom reports completed (85%)
    Median expected questionnaires completed by individual patients was 71%
    Majority of questionnaires completed offsite (n=160; 87%)

    		PaTOS [74], Any disease site (N=30)
    Chemo, 10 weeks
    QAS=6    		# of completions/ accesses28/30 patients observed for 10 weeks
    Total 231 accesses, 193 fully completed
    Total of 1870 symptoms observations (average 69 per patient, 1.5 per day)

    		Pit-a-pit [75], Breast (N=30)
    Neo-adjuvant chemo, 90 days
    QAS=14    		# of accesses/ expected accesses1215/2700 responses (compliance=45.0 %)
    Median patient-level reporting rate was 41.1% (range 6.7-95.6%)

    		RemeCoach [81], Advanced solid tumors, eg, colorectal, gastric-esophageal, and pancreatic adenocarcinoma (N=11)
    Duration of Teysuno treatment
    QAS=18    		# of accesses/ expected accessesAverage daily compliance 91.2%
    Could not determine longitudinal compliance because of the low patient number using the coach for an acceptable duration of time

    		SCMS [82], Breast, lung, or colorectal (N=4)
    During 4 cycles of chemo
    QAS=10    		Evaluation questionnaireAll patients completed at least 1 symptom report
    Questionnaire revealed patients found system useful and easy to use

    		STAR [84], Gynecologic malignancy (N=49)
    Laparotomy, 6 weeks
    QAS=20    		# of accesses/ expected accessesCompliance of patients gradually decreased
    92% of patients completed preoperative session, and 74% completed Week 6 session
    Majority of patients (82%) completed at least 4/7 total sessions in STAR

    		STAR [11], Gynecologic malignancy (N=80)
    Chemo, 8 weeks
    QAS=16    		Users/nonusers (logged in/did not log in)Patients could access from home or in clinic
    25% used only in clinic waiting area, remainder logged in from home and clinic
    Most patients with home computers (83%) logged in from home without reminders

    		STAR [12], Not specified (N=180)
    Chemo, 8 weeks
    QAS=8    		Users/nonusers (logged in/did not log in)Patients could access from home or clinic
    2/3 voluntarily logged in from home computers without prompting

    		STAR [13], Thoracic malignancies (N=107)
    Chemo, 16 months
    QAS=20    		# of accesses/ expected accessesPatients could access from home or clinic
    16 patients (15%) accessed system from home
    Home users accessed system more frequently than those using in clinic (avg=23 sessions, range 3-144) vs (avg=9, range 1-36) respectively

    		STAR [15], Lung, gynecologic, breast, genitourinary (N=286)
    Duration of chemo
    QAS=19    		# of completions/ accessesPatients could access from home or in clinic
    Total 8690 logins (median 17 logins per patient), avg 0.9 logins per patient per week
    71% from home and 29% from clinic

    		STAR [14], Gynecologic malignancy (N=96)
    Laparotomy, preoperatively & weekly 6-wks postlaparotomy
    QAS=17    		# of accesses/ expected accesses74% (n=71) completed at least 4/7 surveys and were considered responders
    63% (n=69) completed preoperative session. Remaining completed subsequent surveys.
    9 (9%) patients completed only 1 survey

    		System A [87], Hepatobiliary and GI (N=20)
    Preoperatively and 2 weeks after discharge for curative resection
    QAS=17    		# of completions/ accesses65% (13/20) completed 8 symptom assessments
    75% (15/20) completed 4 QoL assessments
    Mean 7 minutes to complete MD Anderson Symptom Inventory and mean 4 minutes to complete EuroQoL-5D-5L

    		System B [90], Head and neck cancer (N=36)
    Surgery, 6 weeks
    QAS=17    		# of completions/ accessesAll patients used system (total sessions=982)
    Avg no of sessions was 27.3 (SD 18.4, range 4-69)
    Avg session 12 minutes, longest session 1 hour 38 minutes

    		System D [92], Prostate (N=9)
    Radiation, 2 weeks
    QAS=13    		# of completions/ accessesPatients reported for mean of 10 days
    Estimated time for report 5 minutes
    Self-care advice accessed by 85%, who logged 20 views at 34 symptoms
    59 alerts: 55 yellow and 4 red

    		System E [93], Colon (N=6)
    Complete resection, during 2 cycles of chemo
    QAS=11    		# of accesses / expected accessesData entry compliance was excellent (98% of the twice-daily input was complete) from all 6 patients with the exception of one question

    		System L [101], Head and neck (N=22)
    Duration of radiation (approx. 5-7 weeks)
    QAS=16    		# of accesses/ expected accessesMedian compliance 71% (interquartile range [IQR], 45%-80%)
    6 patients (27%) compliance ≥80%, 2 patients (9%) 100% compliant
    Median reports submitted 34 (IQR 21-53)
    Randomized controlled trials (RCTs; n=7; n refers to # of patients expected to use the system [ie, intervention arm])

    		ASyMS [6], Breast, lung, or colorectal (N=56)
    4 cycles of chemo
    QAS=22    		Not reportedNot reported

    		CASSY [49], Any cancer diagnosis (N=144)
    Chemo, radiation, or surgery, 6 months
    QAS=19    		# of completions/ accessesTotal number of page views=1491
    Total duration in minutes=1813.9
    Total views and duration given for individual patients

    		ESRA-C [60], Diagnosis of cancer (N=374)
    Any therapeutic regimen, UTD, over 4 visits
    QAS=24    		# of completions/ accessesMedian access rate of 4 (range 2-4) at study time points
    Median access rates of 1 (range 0-8) at voluntary times

    		QoC Health Inc [79], Breast (N=32)
    Reconstructive surgery, 30 days
    QAS=23    		Not reportedNot reported

    		STAR [5], Metastatic breast, genitourinary, gynecologic, or lung (N=286)
    Duration of chemo
    QAS=20    		# of accesses/ expected accessesComputer experienced (home access) and inexperienced (clinic access) figures combined
    Avg 73% completed a self-report at any given clinic visit (includes clinic completions)

    		WebChoice [8], Breast or prostate (N=162)
    Surgery plus radiation, chemo, hormone therapy, or a combination, 1 year
    QAS=23    		# of completions/ accesses77% logged on at least once
    23% never logged on
    Of 103 (64%) who logged on more than once, avg logons=60 times (range 2-892)

    		System K [100], Breast cancer (N=95)
    Adjuvant or neo-adjuvant chemo, 6 weeks
    QAS=18    		Not reportedNot reported
    Non-RCT (n=1) (n refers to # of patients expected to use the system [ie, intervention arm])

    		System B [88], Head and neck cancer (N=39)
    Surgery, 6 weeks
    QAS=17    		# of completions/ accessesAvg # of sessions=27, avg length of session=12 mins
    Avg # of completions=12.6
    Avg # of messages=4.5
    Table 4. Overview of patient-centered outcomes data.
    Study, population (N); study designIntervention and comparator groupsOutcomes reportedSummary of results
    ASyMs [6], Breast, lung, or colorectal (N=112); 2-arm randomized controlled trial (RCT), 4 cycles of chemoIntervention (N=56):
    Asked to complete a symptom questionnaire integrating Common Toxicity Criteria Adverse Events (CTCAE) grading system and Chemotherapy Symptom Assessment Scale
    Symptom information sent in real time to the study server
    Patients receive severity dependent tailored self-care advice on mobile phone interface
    Evidence-based risk assessment tool alerts clinicians via a dedicated 24-h pager system of any severe symptoms
    Comparator (N=56):
    Standard care following local guidelines and procedures related to monitoring and reporting of chemo-related toxicity including written and verbal information from nurses administering chemo    		Primary outcomes: Paper version of online questionnaire; Comparison between groups on mean scores from 4 paper-based completions at baseline and before each chemo cycleHigher reports of fatigue (P=.04) and lower reports of hand-foot syndrome (P=.03) in control group compared with intervention group
    No difference on nausea, vomiting, diarrhea, or sore mouth/throat
    CASSY [49], Any diagnosis of cancer
    Chemo, radiation, or surgery (N=261)
    2-arm RCT, 6 months    		Intervention (N=144):
    Access to psycho-educational website where patients could record and monitor symptoms via graphs and journal
    Access chat room to communicate with other study patients
    Audiovisual and resource library including relaxation techniques and educational videos
    Phone contact (approx. every 2 weeks) with a collaborative care coordinator with training and experience with cognitive-behavioral therapy and psycho-oncology
    Comparator (N=117):
    Usual care provided by medical team plus assessment of symptoms and blood draws at the same time as intervention patients to evaluate efficacy of intervention    		Primary outcomes: Depression (Centre for Epidemiologic Studies-Depression≥16), Pain Brief Pain Inventory, Anemia (Functional Assessment of Cancer Therapy [FACT]-Anemia), Hepatobiliary (FACT-Hep)Reductions of fatigue at 6 months (P=.09)
    Statistically and clinically significant changes in overall QoL (P=.05)
    Reductions in pain and depression
    Secondary outcomes: Serum cytokines levels and Natural Killer Cell (NK), Comparison at 6 months follow-upMedium effect size for NK cell number (Phi=0.491) at 6 months (chi-square=3.62, P=.057)
    ESRA-C [60], Diagnosis of cancer
    Any therapeutic regimen (N=779)
    2-arm RCT, UTD, over 4 visits    		Intervention (N=374):
    Participants completed cancer symptoms and QoL (SxQoL) assessments at each study time point and ad lib between visits
    Summary reports delivered to clinicians
    Self-management advice given for 3 symptoms
    Coaching to verbalize issues to health care team
    Alert to contact health care team for severe symptoms
    Patients could monitor symptoms via graphs and journal
    Self-care strategies and coaching available at any time
    Comparator (N=378):
    Participants completed assessments at each study time point
    Summary reports delivered to clinicians
    Research staff verbally notified health care team of any severe symptoms reported at clinic visit
    Both groups were provided the same patient education typically available in each clinic    		Primary outcomes: Symptom Distress Scale (SDS) plus 2 items (impact on sexual activity and interest, fever/chills) to form SDS-15, End point was change in SDS-15 total score from baseline to the end-of-study time pointIntervention had lower symptom distress; mean change in SDS-15 score was 1.27 ([SD], 6.7) in control (higher distress) and -0.04 (SD 5.8) in intervention (lower distress)
    SDS-15 score reduced by estimated 1.21 (95% CI 0.23-2.20; P=.02) in intervention vs control group
    QoC Health Inc [79], Breast cancer
    Surgery (N=65)
    2-arm RCT, 30 days    		Intervention (N=32):
    Follow-up visits at 1 and 4 weeks replaced with examination of surgical site via photos submitted through mobile app, plus completion of pain visual analog scale and quality of recovery 9-item questionnaire
    Reporting began after discharge from recovery room
    Email reminder if submission not received
    Surgeon used wireless interface to access data and monitor patient’s condition
    Severe scores flagged in the database for quick viewing. Red flags prompted in-person follow-up
    Physicians summarized data from mobile app using prototypical subjective, objective, assessment, and plan note at 1 or more time points during 30-day monitoring period
    Comparator (N=33):
    Patients in conventional follow-up group had planned clinic follow-up at approx. 1 week and 4 weeks after operation    		Primary outcomes: Total number of follow-up visits (including specialists, family physician, and emergency department), Total number of phone calls and emails to health care team, Satisfaction and convenience scores using 5-point Likert scale, Postop complicationsControl group more likely to attend in-person follow-up care first 30 days after surgery (95% CI 0.24-0.66; P<.001)
    Intervention group sent more emails than control group (IRR 4.13; 95% CI 1.55-10.99; P=.005)
    Intervention group reported higher convenience scores (IRR 1.39; 95% CI 1.09-1.77; P=.008)
    STAR [5], Metastatic breast, genitourinary, gynecologic, or lung cancers (N=766)
    Before randomization, participants assigned to subgroups (computer-experienced and computer-inexperienced) Only computer-experienced intervention used system from home
    Duration of chemo    		Intervention (N=286):
    Remote access to Web-based interface including questions adapted for patient use from CTCAE
    Triggered email alerts to nurses when patient-reported symptom worsened by 2 points or reached an absolute grade
    Report tracking participant’s symptoms printed at each clinic visit for both nurse and treating oncologist
    No specific guidance provided to clinicians on actions to take in response to alerts or printed symptom profiles
    Comparators:
    Intervention – Computer-inexperienced (N=155):
    Similar to main intervention group but accessed system in clinic only and did not have remote access
    Computer-experienced – Usual care (N=253)
    Computer-inexperienced – Usual care (N=72):
    Usual care for the computer-experienced and computer-inexperienced subgroups consisted of standard procedure for monitoring and documenting symptoms
    Symptoms discussed and documented in the medical record during clinical encounters between patients and oncologists
    Patients encouraged to initiate phone contact between visits for concerning symptoms    		Primary outcomes: EuroQol EQ-5D Index given via paper at clinic visits every 12 ± 4 weeks throughout studyCombined results for computer-experienced (home system) & computer-inexperienced (clinic only) intervention
    Secondary outcomes: Survival at 1 year, Time to first emergency room visit and time to first hospitalization, Time receiving active cancer treatment, Number of nursing calls to patientsGreater improvement in Health-Related QoL scores in intervention vs usual care arm (34% vs 18%) and worsened among fewer (38% vs 53%; P<.001)
    Greater survival in intervention arm (69% vs 75%, P=.05)
    Fewer emergency room visits in intervention (34% vs 41%, P=.02)
    Intervention received chemo for longer (8.2 vs 6.3 months, P=.002)
    No difference in number of nursing calls to patients
    WebChoice [8], Breast or prostate cancer
    Surgery plus additional treatment of either radiation, chemo, hormone therapy, or a combination of those) (N=325)
    2-arm RCT, 1 year    		Intervention (N=162):
    Assessment component to monitor and report symptoms, problems, and priorities for support along physical, functional, and psychosocial dimensions
    Patients receive automated tailored self-management advice based on responses
    Patients receive advice to contact health care team when appropriate
    Info can be used to create a self-care plan
    Info section with access to other reliable, relevant Web resources
    Communication section including (1) unrestricted support forum for group discussion, allowing patients to post messages anonymously, (2) question-and-answer area where patients can privately ask questions of expert nurses in cancer care
    Access to diary to keep personal notes
    Comparator (N=163):
    In addition to a letter giving their group assignment, participants receive info sheet with suggestions for publicly available, cancer-relevant websites    		Primary outcomes: Memorial Symptom Assessment Scale Short FormBetween-group differences significant for the Global Distress Index only (t=4.42; P=.037)
    Secondary outcomes: Center for Epidemiological Studies Depression scale, Cancer Behaviour Inventory, 15D Health-related QoL, Medical Outcome Study Social Support SurveyNo significant differences on the other subscales or total score or any secondary outcomes
    Experimental group showed significant improvements in depression (t=-2.71; P=.007)
    Control group had worsened self-efficacy (t=-2.82; P=.005) and Health-related QoL scores significantly (t=-2.77; P=.006),
    System B, Van den Brink [88]
    Head and neck cancer
    Surgery (N=163)
    Nonrandomized trial, 6 weeks    		Intervention (N=39):
    Provided with a laptop
    Patients could be monitored at home (by means of electronic questionnaires)
    Could communicate (send messages) to team
    Access to information
    Communicate with fellow sufferers (via a forum)
    Comparator (N=128):
    Routine follow-up apps at 2 and 6 weeks after discharge
    Patients could contact their care providers, both in- and outside hospital, if considered necessary    		Primary outcomes: QoL measure assessed state anxiety, object anxiety, feelings of depression, uncertainty, feelings of insecurity, loss of control, self-efficacy, loneliness, and complaintsIntervention had significantly better change from baseline at 6 wks for state anxiety (P=.01), fear related to specific head and neck problems (P=.02), physical self-efficacy (P=.03), perceived abilities in swallowing and food intake (P=.04), general physical complaints (P=.02)
    System K, Egbring [100], Breast cancer
    Adjuvant or neo-adjuvant chemo (N=139)
    3 arm RCT, 6 weeks    		Intervention (N=49):
    App and physician: Patients used mobile app and reviewed reported data with treating physician at scheduled visits
    Patients could report daily functional activity or symptoms with indication of severity
    Patients could edit a quick list of their preselected symptoms or select any of the 48 symptoms made available from the CTCAE listing
    Treating physician enabled access to review and discuss electronically reported symptoms during scheduled visits
    Comparators:
    Attention-control group (N=46)
    App only: Patients instructed to use the mobile app without physician review
    Control group (N=44):
    Received regular physician support    		Primary outcomes: Daily functional activity measured by ECOGControl groups showed greater decline in functional activity versus intervention but not significant
    Secondary outcomes: Symptom reporting (intervention group and attention control group only), Patient-physician communication (measure not specified), Patient Empowerment (measure not specified)At last visit, intervention & attention control patients reported fewer concentration issues than control group (P=.002)
    At third visit, significantly more intervention & attention control patients confirmed use of Internet for disease information compared vs control

    Principal Findings

    The main aim of this review was to systematically describe and assess the features and functions of current systems available for patients to report and manage side effects of cancer treatment. We also wanted to focus on understanding the level of evidence indicating whether key system features are associated with better patient system engagement and patient outcomes.

    In Stage 1 of the review, we identified a total of 41 individual systems. There was significant variation between systems, though published descriptions of systems were often limited. We developed a taxonomy of features that were then classified into those supporting clinicians to deliver patient care in an innovative way and those aimed to support patients to better self-manage their condition and identify when medical input may be needed. This was successfully applied to describe the presence or absence of common system features.

    The review of features highlighted some interesting findings. It was surprising to note that while over half (58%) of systems had the facility for health care providers to monitor patient data over time, fewer than half (46%) included the facility for patients to monitor and review their own data. Given the available evidence suggesting that self-monitoring is generally beneficial to support patients’ self-management [28,33,103], this feature could be very important to improve efficacy of systems and in most cases, may be relatively easy to implement. Similarly, less than half of the systems (41%) included a feature for delivering advice to support patients to self-manage symptoms and less than a third provided patients with access to general educational information. The two least common features were facilities to support communication between patients and health care providers (15%) and communication between patients themselves, respectively (10%). Previous research has indicated that these features are highly valued and utilized by patients [20,22,29,33]. It is likely that these features are less common due to complexities in their implementation and maintenance. For example, it may be difficult to engage busy clinicians to respond to patient communication in this way, and there are ethical considerations around the need to moderate patient forums that are endorsed by a health care facility.

    In Stage 2 of the review, we found little agreement on how patient engagement with systems was defined, measured, or reported, which meant it was not possible to compare levels of engagement across studies or make any conclusions on relationships with system features. Our review also indicated heterogeneity in terms of outcomes used to evaluate systems. Even of those that focused on symptoms or global QoL, the variation in methods and measures used made meaningful comparison impossible.

    Due to the heterogeneous nature of reporting engagement and outcomes, we were unable to explore any relationships with system features. Our findings are similar to other reviews undertaken in this area, which have also found that poor assessment and reporting of patient engagement with systems makes comparison between studies difficult. Brower et al made quantifiable and comparable reports of engagement as part of their inclusion criteria for their review, and results indicated that facility for communication with other patients may be a very influential factor in patient engagement and needs careful consideration during system design [22]. However, other oncology specific reviews have found that methods of assessing and reporting patient engagement were too heterogeneous to make meaningful conclusions [104,105]. We identified only 8 trials (7 randomized and 1 nonrandomized) that evaluated systems, none of which reported any analysis on relationships between engagement and outcomes, and 3 of which did not report any data on patient engagement at all. This does not seem to be unique to oncology. Donkin et al [106] set out to review the impact of patient engagement with e-therapies across a range of disease groups and similarly found that this is not a link that is routinely explored.

    Robust evidence supporting the value of systems for patient-centered outcomes was limited, with a large proportion of feasibility studies identified and even fewer RCTs. While all trials used some measure of patient-centered outcome to evaluate systems, a wide range of assessment tools were used, again making comparison difficult. In addition, 2 studies used the same measure for symptom assessment as part of the intervention, as for the outcome measure. Only 3 trials reported any measure of self-efficacy or patient empowerment, one of which used a study-specific nonvalidated measure [79], and another that was assessed using a subscale of a global QoL measure [88]. There is an array of evidence to suggest that online interventions can have a positive impact on self-efficacy and patient activation levels [30,32,33,107]. Growing evidence suggests that self-efficacy and patient activation play a significant role in symptom management and quality of life throughout cancer treatment [108,109] and are associated with an array of improved health behaviors and health outcomes [110-112] and lower use of hospital resources [113]. The reviewed systems generally demonstrate positive outcomes for patients as has been found in other reviews [31].

    To our knowledge, this is the first systematic review in this field to identify and characterize all available systems for patients to report and manage side effects of cancer treatment, in addition to evidence on patient engagement and patient-centered outcomes.

    Limitations

    In order to meet the aims of the review, we included many publications that provided limited information about the system evaluated and some of which were of poor quality. However, we felt that this was necessary in order to meet the aims of the review and evaluate all evidence. Due to limitations on available resources, the initial stage of study selection (ie, assessment of titles and abstracts) was undertaken by a single reviewer. This is a limitation of our methodology and may have resulted in some bias of inclusion. To address this, a cautious approach erring on the side of over-inclusion was adopted, in order for records to be fully assessed by 2 researchers in the next stage of the review.

    Due to the heterogeneous nature of study designs and methods of reporting engagement and outcomes, we were unable to explore any relationships with system features. This is a field of research that is still in its infancy, and the large number of feasibility studies and abstracts identified are likely to be indicative of this. The search was last updated September 2017. Due to the fast-moving nature of this field of research, it is likely that additional publications will be available by the time of publication. This is a common limitation of systematic reviews that is particularly pertinent with reviews of technology [114]. We did identify a number of protocols for planned quality trials that may contribute to a more in-depth understanding of associations between system features, adherence, and outcomes in the future [4,7,53,67,72,76]. In addition, we have not explored how issues with implementing systems into clinical practice may have affected the efficacy of systems. A discussion of these issues is outside the scope of this review but has been well-documented elsewhere [115].

    Conclusions

    There is a real need for evidence-based guidance on developing, evaluating, and reporting systems. Based on this systematic review, we propose a taxonomy for characterizing system features to guide future development, improvement, and implementation of such systems. More work is needed to develop guidance for standardized reporting of patient engagement both in feasibility studies, and in evaluation trials. This is a complex and multifaceted issue, and it is important that barriers and facilitators to engagement are shared to help the evolution of more sustainable and valuable systems. Similarly, the development of guidance for the evaluation of systems is necessary. Variation in approaches to design and implementation will rightly affect outcomes chosen to evaluate efficacy [104,105]. However, there is enough commonality between systems to call for a set of recommended core outcomes to be developed [116]. More work is needed to develop this, and this is something we will work towards in the future. However, based on this review we recommend that all system evaluations include (1) a description of the system using our taxonomy of system features, (2) measures of feasibility and engagement, (3) patient-centered outcomes focusing on QoL and symptom improvement, in addition to those focusing on self-efficacy and patient activation, and (4) a measure of health economics. This will facilitate synthesis of evidence in order to improve the design of systems and make them practically useful for both patients and clinicians.

    Conflicts of Interest

    None declared.

    1. Furstenberg CT, Carter JA, Henderson JV, Ahles TA. Formative evaluation of a multimedia program for patients about the side effects of cancer treatment. Patient Educ Couns 2002 May;47(1):57-62. [Medline]
    2. Warrington L, Holch P, Kenyon L, Hector C, Kozlowska K, Kenny AM, et al. An audit of acute oncology services: patient experiences of admission procedures and staff utilisation of a new telephone triage system. Support Care Cancer 2016 Dec;24(12):5041-5048. [CrossRef] [Medline]
    3. National Confidential Enquiry into Patient Outcome and Death. 2008. For better, for worse? A review of the care of patients who died within 30 days of receiving systemic anti-cancer therapy   URL: https://www.ncepod.org.uk/2008report3/Downloads/SACT_report.pdf [accessed 2018-11-28] [WebCite Cache]
    4. Absolom K, Holch P, Warrington L, Samy F, Hulme C, Hewison J, eRAPID Systemic Treatment Work Group. Electronic patient self-Reporting of Adverse-events: Patient Information and aDvice (eRAPID): a randomised controlled trial in systemic cancer treatment. BMC Cancer 2017 May 08;17(1):318 [FREE Full text] [CrossRef] [Medline]
    5. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol 2016 Feb 20;34(6):557-565 [FREE Full text] [CrossRef] [Medline]
    6. Kearney N, McCann L, Norrie J, Taylor L, Gray P, McGee-Lennon M, et al. Evaluation of a mobile phone-based, advanced symptom management system (ASyMS) in the management of chemotherapy-related toxicity. Support Care Cancer 2009 Apr;17(4):437-444. [CrossRef] [Medline]
    7. Maguire R, Fox P, McCann L, Miaskowski C, Kotronoulas G, Miller M, et al. The eSMART study protocol: a randomised controlled trial to evaluate electronic symptom management using the advanced symptom management system (ASyMS) remote technology for patients with cancer. BMJ Open 2017 Dec 06;7(5):e015016 [FREE Full text] [CrossRef] [Medline]
    8. Ruland CM, Andersen T, Jeneson A, Moore S, Grimsbø GH, Børøsund E, et al. Effects of an internet support system to assist cancer patients in reducing symptom distress: a randomized controlled trial. Cancer Nurs 2013;36(1):6-17. [CrossRef] [Medline]
    9. Yang LY, Manhas DS, Howard AF, Olson RA. Patient-reported outcome use in oncology: a systematic review of the impact on patient-clinician communication. Support Care Cancer 2018 Dec;26(1):41-60. [CrossRef] [Medline]
    10. Basch E. Patient-Reported Outcomes - Harnessing Patients' Voices to Improve Clinical Care. N Engl J Med 2017 Jan 12;376(2):105-108. [CrossRef] [Medline]
    11. Basch E, Artz D, Dulko D, Scher K, Sabbatini P, Hensley M, et al. Patient online self-reporting of toxicity symptoms during chemotherapy. J Clin Oncol 2005 May 20;23(15):3552-3561. [CrossRef] [Medline]
    12. Basch E, Artz D, Iasonos A, Speakman J, Shannon K, Lin K, et al. Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. J Am Med Inform Assoc 2007;14(3):264-268 [FREE Full text] [CrossRef] [Medline]
    13. Basch E, Iasonos A, Barz A, Culkin A, Kris MG, Artz D, et al. Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. J Clin Oncol 2007 Dec 01;25(34):5374-5380. [CrossRef] [Medline]
    14. Cowan R, Suidan R, Andikyan V, Rezk Y, Einstein H, Chang K, et al. Electronic patient-reported outcomes from home in patients recovering from major gynecologic cancer surgery: A prospective study measuring symptoms and health-related quality of life. Gynecol Oncol 2016 Nov;143(2):362-366 [FREE Full text] [CrossRef] [Medline]
    15. Judson TJ, Bennett AV, Rogak LJ, Sit L, Barz A, Kris MG, et al. Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. J Clin Oncol 2013 Jul 10;31(20):2580-2585 [FREE Full text] [CrossRef] [Medline]
    16. Andersen T, Ruland CM. Cancer patients' questions and concerns expressed in an online nurse-delivered mail service: preliminary results. Stud Health Technol Inform 2009;146:149-153. [Medline]
    17. Børøsund E, Cvancarova M, Ekstedt M, Moore SM, Ruland CM. How user characteristics affect use patterns in web-based illness management support for patients with breast and prostate cancer. J Med Internet Res 2013 Mar 01;15(3):e34 [FREE Full text] [CrossRef] [Medline]
    18. Børøsund E, Cvancarova M, Moore SM, Ekstedt M, Ruland CM. Comparing effects in regular practice of e-communication and Web-based self-management support among breast cancer patients: preliminary results from a randomized controlled trial. J Med Internet Res 2014 Dec 18;16(12):e295 [FREE Full text] [CrossRef] [Medline]
    19. Ruland CM, Jeneson A, Andersen T, Andersen R, Slaughter L, Bente-Schjødt O, et al. Designing tailored Internet support to assist cancer patients in illness management. AMIA Annu Symp Proc 2007 Oct 11:635-639 [FREE Full text] [Medline]
    20. Ruland CM, Maffei RM, Børøsund E, Krahn A, Andersen T, Grimsbø GH. Evaluation of different features of an eHealth application for personalized illness management support: cancer patients' use and appraisal of usefulness. Int J Med Inform 2013 Jul;82(7):593-603. [CrossRef] [Medline]
    21. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JEWC. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012 Nov 14;14(6):e152 [FREE Full text] [CrossRef] [Medline]
    22. Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011 Jan 06;13(1):e2 [FREE Full text] [CrossRef] [Medline]
    23. Sieverink F, Kelders SM, van Gemert-Pijnen JEWC. Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence. J Med Internet Res 2017 Dec 06;19(12):e402 [FREE Full text] [CrossRef] [Medline]
    24. Glasgow RE, Christiansen SM, Kurz D, King DK, Woolley T, Faber AJ, et al. Engagement in a diabetes self-management website: usage patterns and generalizability of program use. J Med Internet Res 2011 Jan 25;13(1):e9 [FREE Full text] [CrossRef] [Medline]
    25. Irizarry T, DeVito Dabbs A, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res 2015 Jun 23;17(6):e148 [FREE Full text] [CrossRef] [Medline]
    26. Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The Effectiveness of Prompts to Promote Engagement With Digital Interventions: A Systematic Review. J Med Internet Res 2016 Jan 08;18(1):e6 [FREE Full text] [CrossRef] [Medline]
    27. Han JY, Wise M, Kim E, Pingree R, Hawkins RP, Pingree S, et al. Factors Associated with Use of Interactive Cancer Communication System: An Application of the Comprehensive Model of Information Seeking. J Comput Mediat Commun 2010 Apr;15(3):367-388 [FREE Full text] [CrossRef] [Medline]
    28. Cushing CC, Steele RG. A meta-analytic review of eHealth interventions for pediatric health promoting and maintaining behaviors. J Pediatr Psychol 2010 Oct;35(9):937-949. [CrossRef] [Medline]
    29. de Jong CC, Ros WJ, Schrijvers G. The effects on health behavior and health outcomes of Internet-based asynchronous communication between health providers and patients with a chronic condition: a systematic review. J Med Internet Res 2014 Jan 16;16(1):e19 [FREE Full text] [CrossRef] [Medline]
    30. Lu H, Shaw BR, Gustafson DH. Online health consultation: examining uses of an interactive cancer communication tool by low-income women with breast cancer. Int J Med Inform 2011 Jul;80(7):518-528 [FREE Full text] [CrossRef] [Medline]
    31. Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive Health Communication Applications for people with chronic disease. Cochrane Database Syst Rev 2005 Oct 19(4):CD004274. [CrossRef] [Medline]
    32. Samoocha D, Bruinvels DJ, Elbers NA, Anema JR, van der Beek AJ. Effectiveness of web-based interventions on patient empowerment: a systematic review and meta-analysis. J Med Internet Res 2010 Jun 24;12(2):e23 [FREE Full text] [CrossRef] [Medline]
    33. Stellefson M, Chaney B, Barry AE, Chavarria E, Tennant B, Walsh-Childers K, et al. Web 2.0 chronic disease self-management for older adults: a systematic review. J Med Internet Res 2013 Feb 14;15(2):e35 [FREE Full text] [CrossRef] [Medline]
    34. Johansen MA, Henriksen E, Horsch A, Schuster T, Berntsen GKR. Electronic symptom reporting between patient and provider for improved health care service quality: a systematic review of randomized controlled trials. part 1: state of the art. J Med Internet Res 2012 Oct 03;14(5):e118 [FREE Full text] [CrossRef] [Medline]
    35. Corbett T, Singh K, Payne L, Bradbury K, Foster C, Watson E, et al. Understanding acceptability of and engagement with Web-based interventions aiming to improve quality of life in cancer survivors: A synthesis of current research. Psychooncology 2018 Jan;27(1):22-33. [CrossRef] [Medline]
    36. Petticrew M. Time to rethink the systematic review catechism? Moving from 'what works' to 'what happens'. Syst Rev 2015 Mar 28;4:36 [FREE Full text] [CrossRef] [Medline]
    37. Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev 2012 Feb 09;1:2 [FREE Full text] [CrossRef] [Medline]
    38. Agency for Healthcare Research and Quality. Systematic Review Data Repository. 2013.   URL: https://www.ahrq.gov/cpi/about/otherwebsites/srdr.ahrq.gov/index.html [accessed 2018-11-28] [WebCite Cache]
    39. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of Epidemiology & Community Health 1998 Jun 01;52(6):377-384. [CrossRef]
    40. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M. Guidance on the conduct of narrative synthesis in systematic reviews. ESRC methods programme 2006;15(1):047-071 [FREE Full text] [CrossRef]
    41. McCann L, Maguire R, Miller M, Kearney N. Patients' perceptions and experiences of using a mobile phone-based advanced symptom management system (ASyMS) to monitor and manage chemotherapy related toxicity. Eur J Cancer Care (Engl) 2009 Mar;18(2):156-164. [CrossRef] [Medline]
    42. Maguire R, Ream E, Richardson A, Connaghan J, Johnston B, Kotronoulas G, et al. Development of a novel remote patient monitoring system: the advanced symptom management system for radiotherapy to improve the symptom experience of patients with lung cancer receiving radiotherapy. Cancer Nurs 2015;38(2):E37-E47. [CrossRef] [Medline]
    43. Kearney N, Kidd L, Miller M, Sage M, Khorrami J, McGee M, et al. Utilising handheld computers to monitor and support patients receiving chemotherapy: results of a UK-based feasibility study. Support Care Cancer 2006 Jul;14(7):742-752. [CrossRef] [Medline]
    44. Johnston B, Maguire R, Kearney N. Using Mobile Phone Technology to Assess Symptoms In Patients Receiving Palliative Care - The Advanced Symptom Management System (ASyMS (c)-P). In: European Journal of Cancer. 2011 Presented at: 2011 European Multidisciplinary Cancer Congress; Sept. 24, 2011; Stockholm, Sweden.
    45. Maguire R, Kearney N, Pedersen V, Ream E, Richardson A. Improving the Symptom Experience of Patients With Lung Cancer Receiving Radiotherapy: Advanced Symptom Management System for Radiotherapy (ASyMS-R). In: European Journal of Cancer. 2011 Presented at: 2011 European Multidisciplinary Cancer Congress; Sept 24, 2011; Stockholm.
    46. McCann L, Maguire R, Kearney N, Miller M, Taylor L, Sage M, et al. 8083 ORAL The use of a mobile phone based advanced symptom management system in the home monitoring and symptom management of chemotherapy related toxicities in patients with breast, lung and colorectal cancer: Patients' and clinicans' perceptions for supportive self care. European Journal of Cancer Supplements 2007 Sep;5(4):434. [CrossRef]
    47. Breen S, Aranda S, Ritchie D, Kofoed S, Dryden T, Maguire R. 'You cannot manage what you cannot measure': Development of a prototype remote monitoring system for haematological cancer patients undergoing chemotherapy. In: Supportive Care in Cancer. 2012 Jun Presented at: International MASCC/ISOO Symposium; June 28-30, 2012; New York p. S69.
    48. Cowie J, McCann L, Maguire R, Kearney N, Connaghan J, Paterson C, et al. Real-time management of chemotherapy toxicity using the Advanced Symptom Management System (ASyMS). Journal of Decision Systems 2013 Jan 1;22(1):43-52. [CrossRef]
    49. Steel JL, Geller DA, Kim KH, Butterfield LH, Spring M, Grady J, et al. Web-based collaborative care intervention to manage cancer-related symptoms in the palliative care setting. Cancer 2016 Apr 15;122(8):1270-1282 [FREE Full text] [CrossRef] [Medline]
    50. Holzner B, Zabernigg A, Sztankay M, Giesinger J, Kemmler G, Wintner L. Web-based QOL-monitoring for more accurate assessment of symptom burden in cancer patients undergoing chemotherapy. Onkologie 2013 Oct;36:110-111. [Medline]
    51. Tofthagen C, Kip KE, Passmore D, Loy I, Berry DL. Usability and Acceptability of a Web-Based Program for Chemotherapy-Induced Peripheral Neuropathy. Comput Inform Nurs 2016 Jul;34(7):322-329. [CrossRef] [Medline]
    52. Fishbein JN, Nisotel LE, MacDonald JJ, Amoyal Pensak N, Jacobs JM, Flanagan C, et al. Mobile Application to Promote Adherence to Oral Chemotherapy and Symptom Management: A Protocol for Design and Development. JMIR Res Protoc 2017 Apr 20;6(4):e62 [FREE Full text] [CrossRef] [Medline]
    53. Agboola S, Flanagan C, Searl M, Elfiky A, Kvedar J, Jethwani K. Improving outcomes in cancer patients on oral anti-cancer medications using a novel mobile phone-based intervention: study design of a randomized controlled trial. JMIR Res Protoc 2014 Dec 23;3(4):e79 [FREE Full text] [CrossRef] [Medline]
    54. Holch P, Gibson A, Holmes M, Rodgers Z, Clayton B, Henry A. Usability testing of an online symptom report and management system in radical prostate radiotherapy (RT) patients: Preliminary findings of the eRAPID RT programme. In: Psycho-Oncology. 2016 Presented at: British Psychosocial Oncology Society annual conference; Mar. 3-4, 2016; Cambridge, UK p. 25.
    55. Holch P, Davidson S, Routledge J, Henry A, Franks K, Gilbert A. eRAPID: Electronic self-report and management of adverse-events for radical prostate radiotherapy (RT) patients. In: Radiotherapy and Oncology. 2015 Presented at: ESTRO; April 24-28, 2015; Barcelona, Spain p. 115.
    56. Velikova G, Holch P, Bamforth L, Warrington L, Carter R, Absolom K. Development of an integrated online toxicity reporting and management system for oncology: eRAPID (Electronic patient self-Reporting of Adverse-events: Patient Information and aDvice). In: Quality of Life Research. 2014 Oct Presented at: 21st Annual Conference of the International Society for Quality of Life Research; October 15-18, 2014; Berlin, Germany.
    57. Holch P, Warrington L, Potrata B, Ziegler L, Absolom K, Hector C. Acceptability of an online system for reporting and managing symptomatic adverse events (eRAPID): Patients views eRAPID is funded by a National Institute for Health Research (NIHR) programme development grant RP-DG-1209-10031; ISCTRN trial number CCT-NAPN-21338. In: Psycho-Oncology. 2013 Presented at: British Psychosocial Oncology Society; January 17-18, 2013; Southampton, UK p. 9-10.
    58. Velikova G, Ziegler L, Holch P, Harley C, Keding A, Bamforth L. Development of an electronic platform for patient self-reporting of adverse-events during cancer treatment. In: Asia-Pacific Journal of Clinical Oncology. 2012 Presented at: COSA IPOS Psycho-Oncology 2012; November 13-15, 2012; Brisbane, Australia p. 8.
    59. Ziegler L, Harley C, Holch P, Keding A, Bamforth L, Warrington L. Towards safer delivery and monitoring of cancer treatments. Electronic patient self-reporting of adverse-events: Patient information and a advice (eRAPID). In: Psycho-Oncology. 2012 Presented at: British Psychosocial Oncology Society; January 19-20, 2012; Leeds, UK p. 21.
    60. Berry DL, Hong F, Halpenny B, Partridge AH, Fann JR, Wolpin S, et al. Electronic self-report assessment for cancer and self-care support: results of a multicenter randomized trial. J Clin Oncol 2014 Jan 20;32(3):199-205 [FREE Full text] [CrossRef] [Medline]
    61. Berry DL, Blonquist TM, Patel RA, Halpenny B, McReynolds J. Exposure to a patient-centered, Web-based intervention for managing cancer symptom and quality of life issues: impact on symptom distress. J Med Internet Res 2015 Jun 03;17(6):e136 [FREE Full text] [CrossRef] [Medline]
    62. Berry DL, Hong F, Halpenny B, Partridge A, Fox E, Fann JR, et al. The electronic self report assessment and intervention for cancer: promoting patient verbal reporting of symptom and quality of life issues in a randomized controlled trial. BMC Cancer 2014 Jul 12;14:513 [FREE Full text] [CrossRef] [Medline]
    63. Klasnja P, Hartzler A, Powell C, Pratt W. Supporting cancer patients' unanchored health information management with mobile technology. AMIA Annu Symp Proc 2011;2011:732-741 [FREE Full text] [Medline]
    64. Klasnja P, Hartzler A, Powell C, Phan G, Pratt W. Health Weaver Mobile: Designing a Mobile Tool for Managing Personal Health Information during Cancer Care. AMIA Annu Symp Proc 2010 Nov 13;2010:392-396 [FREE Full text] [Medline]
    65. McGee M, Gray P. A handheld chemotherapy symptom management system: results from a preliminary outpatient field trial. Health Informatics J 2016 Jul 25;11(4):243-258. [CrossRef]
    66. Levi F, Arbaud A, Dispersyn G, Simon J, Innominato P, Bossevot R. The European InCASA telecare-telehealth electronic platform (ICT-FP7) for the daily assessment of symptoms, weight, and activity in cancer patients on chronotherapy at home. In: Journal of Clinical Oncology Conference (15 SUPPL. 1). 2012 Presented at: ASCO Annual Meeting; 2012; Chicago, IL.
    67. Langius-Eklöf A, Crafoord M, Christiansen M, Fjell M, Sundberg K. Effects of an interactive mHealth innovation for early detection of patient-reported symptom distress with focus on participatory care: protocol for a study based on prospective, randomised, controlled trials in patients with prostate and breast cancer. BMC Cancer 2017 Jul 04;17(1):466 [FREE Full text] [CrossRef] [Medline]
    68. Peltola M, Lehikoinen J, Sippola L, Saarilahti K, Mäkitie AA. A Novel Digital Patient-Reported Outcome Platform for Head and Neck Oncology Patients-A Pilot Study. Clin Med Insights Ear Nose Throat 2016;9:1-6 [FREE Full text] [CrossRef] [Medline]
    69. McRoy L, Mitra D, Hollis K, Kaye J, Zelnak A, Cheyl J. Abstract OT3-03-01: MADELINE: A prospective observational study of mobile app-based patient reported outcomes in advanced breast cancer. Cancer Res 2017 Feb 14;77(4 Supplement):-. [CrossRef]
    70. Pusic A, Klassen A, Scott A, Cano S, Shouery M, Basch E. Feasibility and acceptability of patient-reported outcomes data collection for clinical care following breast reconstruction. In: Journal of Clinical Oncology Conference (15 SUPPL. 1). 2012 Presented at: ASCO Annual Meeting; 2012; Chicago, IL.
    71. Galligioni E, Piras EM, Galvagni M, Eccher C, Caramatti S, Zanolli D, et al. Integrating mHealth in Oncology: Experience in the Province of Trento. J Med Internet Res 2015 May 13;17(5):e114 [FREE Full text] [CrossRef] [Medline]
    72. Passardi A, Rizzo M, Maines F, Tondini C, Zambelli A, Vespignani R, et al. Optimisation and validation of a remote monitoring system (Onco-TreC) for home-based management of oral anticancer therapies: an Italian multicentre feasibility study. BMJ Open 2017 Dec 29;7(5):e014617 [FREE Full text] [CrossRef] [Medline]
    73. Snyder CF, Blackford AL, Wolff AC, Carducci MA, Herman JM, Wu AW, PatientViewpoint Scientific Advisory Board. Feasibility and value of PatientViewpoint: a web system for patient-reported outcomes assessment in clinical practice. Psychooncology 2013 Apr;22(4):895-901 [FREE Full text] [CrossRef] [Medline]
    74. Della Mea V, De Momi I, Aprile G, Puglisi F, Menis J, Casetta A, et al. Feasibility study of a web application for self-report of anticancer treatment toxicities. Stud Health Technol Inform 2009;150:562-566. [Medline]
    75. Min YH, Lee JW, Shin Y, Jo M, Sohn G, Lee J, et al. Daily collection of self-reporting sleep disturbance data via a smartphone app in breast cancer patients receiving chemotherapy: a feasibility study. J Med Internet Res 2014 May 23;16(5):e135 [FREE Full text] [CrossRef] [Medline]
    76. Breen S, Ritchie D, Schofield P, Hsueh Y, Gough K, Santamaria N, et al. The Patient Remote Intervention and Symptom Management System (PRISMS) - a Telehealth- mediated intervention enabling real-time monitoring of chemotherapy side-effects in patients with haematological malignancies: study protocol for a randomised controlled trial. Trials 2015 Oct 19;16:472 [FREE Full text] [CrossRef] [Medline]
    77. Breen S, Aranda S, Ritchie D, Schofield P, Kamateros R, Maguire R. Improving the Management of Chemotherapy Toxicities in Haematological Cancer Patients: A Phase II Randomised Controlled Trial of the Patient Remote Intervention and Symptom Management System (PRISMS). In: Asia-Pacific Journal of Clinical Oncology. 2012 Presented at: ASCO Annual Meeting; 2012; Chicago, IL.
    78. Aggarwal S, Topaloglu H. Novel electronic patient reported outcomes tool for prostate cancer patients. In: Value in Health. 2013 May Presented at: ISPOR 18th Annual International Meeting; May 18-22, 2013; New Orleans, LA p. A147.
    79. Armstrong KA, Coyte PC, Brown M, Beber B, Semple JL. Effect of Home Monitoring via Mobile App on the Number of In-Person Visits Following Ambulatory Surgery: A Randomized Clinical Trial. JAMA Surg 2017 Jul 01;152(7):622-627 [FREE Full text] [CrossRef] [Medline]
    80. Armstrong KA, Semple JL, Coyte PC. Replacing ambulatory surgical follow-up visits with mobile app home monitoring: modeling cost-effective scenarios. J Med Internet Res 2014 Sep 22;16(9):e213 [FREE Full text] [CrossRef] [Medline]
    81. Rasschaert M, Helsen S, Rolfo C, Van Brussel I, Ravelingien J, Peeters M. Feasibility of an interactive electronic self-report tool for oral cancer therapy in an outpatient setting. Support Care Cancer 2016 Dec;24(8):3567-3571. [CrossRef] [Medline]
    82. Chan M, Ang E, Duong MC, Chow YL. An online Symptom Care and Management System to monitor and support patients receiving chemotherapy: a pilot study. Int J Nurs Pract 2013 Feb;19 Suppl 1:14-18. [CrossRef] [Medline]
    83. Chan MF, Ang NKE, Cho AA, Chow YL, Taylor B. Online chemotherapy symptom care and patient management system: an evaluative study. Comput Inform Nurs 2014 Feb;32(2):75-83. [CrossRef] [Medline]
    84. Andikyan V, Rezk Y, Einstein MH, Gualtiere G, Leitao MM, Sonoda Y, et al. A prospective study of the feasibility and acceptability of a Web-based, electronic patient-reported outcome system in assessing patient recovery after major gynecologic cancer surgery. Gynecol Oncol 2012 Nov;127(2):273-277 [FREE Full text] [CrossRef] [Medline]
    85. Head B, Studts J, Bumpous J, Gregg J, Wilson L, Keeney C, et al. Development of a telehealth intervention for head and neck cancer patients. Telemed J E Health 2009 Jan;15(1):44-52 [FREE Full text] [CrossRef] [Medline]
    86. Donovan LM, Rueschman M, Weng J, Basu N, Dudley KA, Bakker JP, et al. The effectiveness of an obstructive sleep apnea screening and treatment program in patients with type 2 diabetes. Diabetes Res Clin Pract 2017 Dec;134:145-152. [CrossRef] [Medline]
    87. Sun V, Dumitra S, Ruel N, Lee B, Melstrom L, Melstrom K, et al. Wireless Monitoring Program of Patient-Centered Outcomes and Recovery Before and After Major Abdominal Cancer Surgery. JAMA Surg 2017 Sep 01;152(9):852-859 [FREE Full text] [CrossRef] [Medline]
    88. van den Brink JL, Moorman PW, de Boer MF, Hop WCJ, Pruyn JFA, Verwoerd CDA, et al. Impact on quality of life of a telemedicine system supporting head and neck cancer patients: a controlled trial during the postoperative period at home. J Am Med Inform Assoc 2007;14(2):198-205 [FREE Full text] [CrossRef] [Medline]
    89. van den Brink JL, Moorman PW, de Boer MF, van Bemmel JH, Pruyn JFA, Verwoerd CDA. An information system to support the care for head and neck cancer patients. Support Care Cancer 2003 Jul;11(7):452-459. [CrossRef] [Medline]
    90. van den Brink JL, Moorman PW, de Boer MF, Pruyn JFA, Verwoerd CDA, van Bemmel JH. Involving the patient: a prospective study on use, appreciation and effectiveness of an information system in head and neck cancer care. Int J Med Inform 2005 Oct;74(10):839-849. [CrossRef] [Medline]
    91. Newlon C, Hu C, Stratton R, McDaniel A. Design of a Web-Based Symptom Management Intervention for Cancer Patients. 2009 Presented at: Human Centered Design; July 19-24, 2009; San Diego, CA p. 775-784.
    92. Sundberg K, Eklöf AL, Blomberg K, Isaksson A, Wengström Y. Feasibility of an interactive ICT-platform for early assessment and management of patient-reported symptoms during radiotherapy for prostate cancer. Eur J Oncol Nurs 2015 Oct;19(5):523-528. [CrossRef] [Medline]
    93. Weaver A, Young AM, Rowntree J, Townsend N, Pearson S, Smith J, et al. Application of mobile phone technology for managing chemotherapy-associated side-effects. Ann Oncol 2007 Nov;18(11):1887-1892. [CrossRef] [Medline]
    94. Prince R, Parente L, Yee A, Powis M, Enright K, Gandhi S. Building “bridges”: Use of participatory design to create an electronic tool to improve management of chemotherapy toxicities. In: Journal of Clinical Oncology Conference. 2016 Presented at: ASCO; June 3-7, 2016; Chicago, IL.
    95. Prince RM, Parente L, Soung Yee A, Powis M, Enright K, Gandhi S, et al. Defining user needs for an electronic tool to improve chemotherapy-related toxicity management. JCO 2016 Mar;34(7_suppl):157-157. [CrossRef]
    96. Due J, Christensen H, Vestlev P. Treatment adverse effect registration. In: A PC based electronic patient - hospital communication portal that allows registration of adverse effects of chemotherapy and returns advice and recommandations for action. Cancer Research Conferenceth Annual CTRC AACR San Antonio Breast Cancer Symposium San Antonio, TX United States Conference Start. (9 SUPPL. 1). 2015 Presented at: Cancer Research Conference: 37th Annual CTRC AACR San Antonio Breast Cancer Symposium; 2015; San Antonio, TX p. 37.
    97. Nimako K, Lu SK, Ayite B, Priest K, Winkley A, Gunapala R, et al. A pilot study of a novel home telemonitoring system for oncology patients receiving chemotherapy. J Telemed Telecare 2013 Apr;19(3):148-152. [CrossRef] [Medline]
    98. Fann JR, Hong F, Halpenny B, Blonquist TM, Berry DL. Psychosocial outcomes of an electronic self-report assessment and self-care intervention for patients with cancer: a randomized controlled trial. Psychooncology 2017 Nov;26(11):1866-1871 [FREE Full text] [CrossRef] [Medline]
    99. Bullard E, Coffman E, Bryant A, Bradley J, Tan X, Wood W. Using Electronic Patient Symptom Reporting to Reduce Symptom Burden During Hospitalization for Preparatory Chemotherapy Prior to Hematopoietic Stem Cell Transplant: The Role of the Oncology Nurse. In: Oncology Nursing Forum. 2017 Mar Presented at: Oncology Nursing Society 42nd Annual Congress; May 4-7, 2017; Denver, CO p. 000401160800611.
    100. Egbring M, Far E, Roos M, Dietrich M, Brauchbar M, Kullak-Ublick GA, et al. A Mobile App to Stabilize Daily Functional Activity of Breast Cancer Patients in Collaboration With the Physician: A Randomized Controlled Clinical Trial. J Med Internet Res 2016 Dec 06;18(9):e238 [FREE Full text] [CrossRef] [Medline]
    101. Falchook A, Tracton G, Stravers L, Fleming M, Reeve B, Basch E. Use of mobile device technology to collect patient-reported symptoms during radiotherapy for head and neck cancer: A prospective feasibility study. In: Journal of Clinical Oncology Conference (15 SUPPL. 1). 2015 Presented at: ASCO; May 29-June 2, 2015; Chicago, IL.
    102. Yang M, Thea J, An Y, Yu JB. Digital health application for real-time patient-reported outcomes during prostate radiotherapy. JCO 2016 Jan 10;34(2_suppl):157-157. [CrossRef]
    103. McCorkle R, Ercolano E, Lazenby M, Schulman-Green D, Schilling LS, Lorig K, et al. Self-management: Enabling and empowering patients living with cancer as a chronic illness. CA Cancer J Clin 2011;61(1):50-62 [FREE Full text] [CrossRef] [Medline]
    104. Kofoed S, Breen S, Gough K, Aranda S. Benefits of remote real-time side-effect monitoring systems for patients receiving cancer treatment. Oncol Rev 2012 Mar 05;6(1):e7 [FREE Full text] [CrossRef] [Medline]
    105. McAlpine H, Joubert L, Martin-Sanchez F, Merolli M, Drummond K. A systematic review of types and efficacy of online interventions for cancer patients. Patient Educ Couns 2015 Mar;98(3):283-295. [CrossRef] [Medline]
    106. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug 05;13(3):e52 [FREE Full text] [CrossRef] [Medline]
    107. Solomon M, Wagner SL, Goes J. Effects of a Web-based intervention for adults with chronic conditions on patient activation: online randomized controlled trial. J Med Internet Res 2012 Feb 21;14(1):e32 [FREE Full text] [CrossRef] [Medline]
    108. Hibbard J, Mahoney E, Sonet E. Does patient activation level affect the cancer patient journey? Patient Educ Couns 2017 Jul;100(7):1276-1279. [CrossRef] [Medline]
    109. Papadopoulou C, Kotronoulas G, Schneider A, Miller MI, McBride J, Polly Z, et al. Patient-Reported Self-Efficacy, Anxiety, and Health-Related Quality of Life During Chemotherapy: Results From a Longitudinal Study. Oncol Nurs Forum 2017 Dec 01;44(1):127-136. [CrossRef] [Medline]
    110. Hibbard JH, Mahoney ER, Stock R, Tusler M. Do increases in patient activation result in improved self-management behaviors? Health Serv Res 2007 Aug;42(4):1443-1463 [FREE Full text] [CrossRef] [Medline]
    111. Marshall R, Beach MC, Saha S, Mori T, Loveless MO, Hibbard JH, et al. Patient activation and improved outcomes in HIV-infected patients. J Gen Intern Med 2013 May;28(5):668-674 [FREE Full text] [CrossRef] [Medline]
    112. Rask KJ, Ziemer DC, Kohler SA, Hawley JN, Arinde FJ, Barnes CS. Patient activation is associated with healthy behaviors and ease in managing diabetes in an indigent population. Diabetes Educ 2009;35(4):622-630. [CrossRef] [Medline]
    113. Begum N, Donald M, Ozolins IZ, Dower J. Hospital admissions, emergency department utilisation and patient activation for self-management among people with diabetes. Diabetes Res Clin Pract 2011 Aug;93(2):260-267. [CrossRef] [Medline]
    114. Beller EM, Chen JK, Wang UL, Glasziou PP. Are systematic reviews up-to-date at the time of publication? Syst Rev 2013 May 28;2:36 [FREE Full text] [CrossRef] [Medline]
    115. Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 2012;62(5):337-347 [FREE Full text] [CrossRef] [Medline]
    116. Boger E, Ellis J, Latter S, Foster C, Kennedy A, Jones F, et al. Self-Management and Self-Management Support Outcomes: A Systematic Review and Mixed Research Synthesis of Stakeholder Views. PLoS One 2015;10(7):e0130990 [FREE Full text] [CrossRef] [Medline]

    ‎
    ASyMs: Advanced Symptom Management System
    CASSY: Comprehensive Electronic Cancer Support System for the Treatment of Cancer Related Symptoms
    CTCAE: Common Toxicity Criteria Adverse Events
    ESRA-C: Electronic Self-Report Assessment-Cancer
    HCP: health care professional
    PICOS: Population, Intervention, Comparator, Outcomes, Study design
    PROM: patient-reported outcome measure
    PROSPERO: International Prospective Register of Systematic Reviews
    QAS: Quality Assessment Score
    QoL: quality of life
    RCT: randomized controlled trial

    Edited by G Eysenbach; submitted 25.04.18; peer-reviewed by L Hess, M Krzyzanowska, V Kukreti, R Collado-Borrell; comments to author 25.08.18; revised version received 20.09.18; accepted 10.10.18; published 24.01.19

    Copyright

    ©Lorraine Warrington, Kate Absolom, Mark Conner, Ian Kellar, Beverly Clayton, Michael Ayres, Galina Velikova. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.01.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.