This paper is in the following e-collection/theme issue:

Patient-Reported Outcome Measures (PROMs) (288) Research Instruments, Questionnaires, and Tools (576) Breast Cancer (206)

Published on in Vol 21, No 1 (2019): January

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10004, first published .
An Electronic Patient-Reported Outcome Tool for the FACT-B (Functional Assessment of Cancer Therapy-Breast) Questionnaire for Measuring the Health-Related Quality of Life in Patients With Breast Cancer: Reliability Study

An Electronic Patient-Reported Outcome Tool for the FACT-B (Functional Assessment of Cancer Therapy-Breast) Questionnaire for Measuring the Health-Related Quality of Life in Patients With Breast Cancer: Reliability Study

An Electronic Patient-Reported Outcome Tool for the FACT-B (Functional Assessment of Cancer Therapy-Breast) Questionnaire for Measuring the Health-Related Quality of Life in Patients With Breast Cancer: Reliability Study

Authors of this article:

Lina Maria Matthies1 Author Orcid Image ;   Florin-Andrei Taran2 Author Orcid Image ;   Lucia Keilmann1 Author Orcid Image ;   Andreas Schneeweiss1 Author Orcid Image ;   Elisabeth Simoes3 Author Orcid Image ;   Andreas D Hartkopf2 Author Orcid Image ;   Alexander N Sokolov3, 4 Author Orcid Image ;   Christina B Walter2 Author Orcid Image ;   Nina Sickenberger1 Author Orcid Image ;   Stephanie Wallwiener1 Author Orcid Image ;   Manuel Feisst5 Author Orcid Image ;   Paul Gass6 Author Orcid Image ;   Michael P Lux6 Author Orcid Image ;   Florian Schuetz1 Author Orcid Image ;   Peter A Fasching6 Author Orcid Image ;   Christof Sohn1 Author Orcid Image ;   Sara Y Brucker3 Author Orcid Image ;   Joachim Graf3, 4 Author Orcid Image ;   Markus Wallwiener1 Author Orcid Image
Article Authors Cited by (23) Tweetations (2) Metrics

    Original Paper

    1Hospital for General Obstetrics and Gynecology, Gynecologic Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany

    2Department of Women’s Health, University Hospital Tübingen, Tübingen, Germany

    3Research Institute for Women’s Health, Department of Women’s Health, University Hospital Tübingen, Tübingen, Germany

    4Section of Midwifery Science, Institute for Health Sciences, University Hospital Tübingen, Tübingen, Germany

    5Institute for Medical Biometry and Informatics, University Hospital Heidelberg, Heidelberg, Germany

    6Department of Gynecology and Obstetrics, University Breast Center Franconia, Comprehensive Cancer Center Erlangen-EMN, University Hospital Erlangen, Erlangen, Germany

    *these authors contributed equally

    Corresponding Author:

    Markus Wallwiener

    Hospital for General Obstetrics and Gynecology

    Gynecologic Oncology, National Center for Tumor Diseases

    University Hospital Heidelberg

    Im Neuenheimer Feld 440

    Heidelberg, 69120

    Germany

    Phone: 49 (0)6221 5636956

    Email: markus.wallwiener@gmail.com


    Background: The most frequent malignant disease in women is breast cancer. In the metastatic setting, quality of life is the primary therapeutic goal, and systematic treatment has only a limited effect on survival rates; therefore, the concept of the health-related quality of life (HRQoL) and measurement of patient-reported outcomes (PROs) are gaining more and more importance in the therapy setting of diseases such as breast cancer. One of the frequently used questionnaires for measuring the HRQoL in patients with breast cancer is the Functional Assessment of Cancer Therapy-Breast (FACT-B). Currently, paper-based surveys still predominate, as only a few reliable and validated electronic-based questionnaires are available. ePRO tools for the FACT-B questionnaire with proven reliability are missing so far.

    Objective: The aim of this study was to analyze the reliability of tablet-based measurement of FACT-B in the German language in adjuvant (curative) and metastatic breast cancer patients.

    Methods: Paper- and tablet-based questionnaires were completed by a total of 106 female adjuvant and metastatic breast cancer patients. All patients were required to complete the electronically based (ePRO) and paper-based version of the FACT-B. A frequency analysis was performed to determine descriptive sociodemographic characteristics. Both dimensions of reliability (parallel forms reliability using Wilcoxon test and test of internal consistency using Spearman ρ) and agreement rates for single items, Kendall tau for each subscale, and total score were analyzed.

    Results: High correlations were shown for both dimensions of reliability (parallel forms reliability and internal consistency) in the patients’ response behavior between paper-based and electronically based questionnaires. Regarding the reliability test of parallel forms, no significant differences were found in 35 of 37 single items, while significant correlations in the test for consistency were found in all 37 single items, in all 5 sum individual item subscale scores, as well as in total FACT-B score.

    Conclusions: The ePRO version of the FACT-B questionnaire is reliable for patients with breast cancer in both adjuvant and metastatic settings, showing highly significant correlations with the paper-based version in almost all questions all subscales and the total score.

    J Med Internet Res 2019;21(1):e10004

    doi:10.2196/10004

    Keywords

    breast cancer; ePRO measurement; FACT-B; HRQoL; patient-reported outcomes; reliability of ePRO 


    Breast Cancer: Epidemiological Relevance

    The most frequent malignant disease in women is breast cancer; indeed, about 70,000 new cases of breast cancer are diagnosed in Germany every year. Therapeutic options have been improved, resulting in an overall 5-year survival rate of patients with early-stage disease of >90% [1-3]. In contrast, the prognosis of metastatic breast cancer is significantly poorer, since palliative treatment often remains the only option due to the low probability of cure in patients with metastatic disease [4,5].

    Patient-Reported Outcomes and Health-Related Quality of Life in Patients With Breast Cancer

    Since systematic palliative treatment has only a limited effect on survival rates, the concept of the health-related quality of life (HRQoL) and measurement of patient-reported outcomes (PROs) are gaining more and more importance in the therapy setting of progressive diseases, such as breast cancer, especially in the adjuvant or metastatic setting [6-10]. Patients with cancer often suffer from symptoms and adverse events during their treatment, which are sometimes underestimated in clinical routine [11-13]. Although clinicians’ perception of symptoms can predict unfavorable clinical events more precisely, patients´ reports can reflect the daily health status more adequately [14]. PROs comprise various aspects of the subjectively perceived state of health from the patients’ point of view such as HRQoL, satisfaction with care, and drug adherence [9,15,16]. With regard to the therapy setting, monitoring HRQoL and the occurrence of symptoms appears to be of particular relevance, primarily during therapy, but also as a long-term follow-up for improving and supporting patients´ well-being [17-20]. The importance of measuring PRO in patients with breast cancer is also stated in the German S3-guideline [21]. As (metastatic) breast cancer often remains an incurable disease with only palliative treatment options, monitoring the HRQoL is highly relevant in these patients [4,22,23]. Different questionnaires highlight different aspects of symptoms and HRQoL [24]. One of the frequently used questionnaires for measuring the HRQoL in patients with breast cancer is the Functional Assessment of Cancer Therapy-Breast (FACT-B), a validated, multidimensional questionnaire with 37 items that build 5 dimensions (subscales) when using a 5-point Likert scale [25-27].

    Electronic Measurement of Patient-Reported Outcomes

    Collecting and analyzing pencil and paper-based data are difficult tasks without possibilities for direct response or interaction [15]; therefore, the electronic-based measurement of PRO (ePRO) is gaining more and more importance. There are several advantages of ePRO such as rapid access to data, a probable avoidance of errors during data entry, fewer missing data in comparison with paper-based surveys, the capacity to trigger alerts or notifications for answers to special circumstances, and an improvement in patients’ willingness to report sensitive information [23,28]. Although paper-based surveys of PRO still predominate because there are only a few reliable and validated ePRO questionnaires, numerous projects have evaluated the feasibility and acceptance of HRQoL in the ePRO measurement in the last few years [29-34]. Nevertheless, knowledge regarding patients’ acceptance, feasibility, and barriers remains limited [35], especially because hurdles might exist in relation to health status, technical skills, and socioeconomic aspects, which could influence both patients´ willingness to use ePRO and their response behavior [10,36,37]. Although studies have already demonstrated a potential equivalence between some paper-based PRO and ePRO, the reliability of ePRO questionnaires should be verified so as not to endanger the validity of ePRO surveys [10,36-40]. Indeed, ePRO tools for the FACT-B questionnaire with proven reliability are missing so far.

    Aims and Objectives

    The aim of the study was to analyze the reliability of a tablet-based ePRO app for FACT-B in German for measuring the HRQoL in adjuvant and metastatic breast cancer patients in comparison with the validated paper-based version of FACT-B. It was planned to determine whether differences exist in response behavior between the validated paper-based PRO version of FACT-B and a new ePRO version, whether the answers between paper-based and ePRO questionnaire differ in a relevant way, and whether the patients’ response behavior is influenced by the mode of answering (paper- or tablet-based). In order to achieve these aims, patients were asked to complete both the paper- and tablet-based version of the FACT-B questionnaire.


    Sample and Study Design

    From July 2015 to May 2016, paper-based and tablet-based PRO questionnaires were completed by a total of 106 female adjuvant and metastatic breast cancer patients treated consecutively at the Department of Women’s Health (Tübingen, Germany) and the National Cancer Center (Heidelberg, Germany). Patients were recruited as a part of the electronic Patient-Reported Outcomes and Compliance Analysis (ePROCOM) and Patient Engagement Pilotstudie Mammakarzinom-individualisierte und Ressourcen-effiziente Patient Reported Outcomes Erfassung durch Digitale Therapieunterstützungssysteme (PEPPER) studies. The aims of ePROCOM were to evaluate the general patient acceptance and practicability of a Web-based app for a PRO questionnaire for patients with adjuvant or metastatic breast cancer. Patients were asked to participate to compare their response behavior in paper-based and Web-based questionnaires and analyze the reliability of the ePRO versions of the questionnaires European Organisation for Research and Treatment of Cancer (EORTC) quality of life questionnaire (QLQ)-C30, as published previously [40], and FACT-B (reported in this paper). The PEPPER study intends to evaluate the impact of Web-based and paper-based PRO for health care services by patients from the Prospective Academic Translational Research Network for the Optimization of the Oncological Health Care Quality in the Adjuvant and Advanced/Metastatic Setting: Health Care Research, Pharmacogenomics, Biomarkers, Health Economics study. The methods are described in detail in the EORTC paper [40]. The inclusion criteria of the ePROCOM and PEPPER studies were female gender, full legal age, the proven diagnosis of breast cancer in adjuvant or metastatic setting, sufficient language skills in German, and signed declaration of consent. The exclusion criterion was participation in other studies to minimize the burden of questionnaires. Patients were asked to complete the questionnaire during an outpatient visit at the hospital under the supervision of an attending physician. The study was designed as a 2-center (Tübingen and Heidelberg), 2-armed, prospective randomized trial. All patients were required to complete both the electronically based (ePRO) and paper-based version of the FACT-B HRQoL questionnaire. Patients in arm A were assigned to start with the tablet version followed by the paper version in the same session. Patients in arm B filled out the paper-based version followed by the tablet-based questionnaire. The randomization procedure was based on permuted-block randomization, which strives to generate equally large treatment groups [41,42]. The postexposure acceptance for using the ePRO tool was high (92%) as patients were asked whether they could potentially imagine using tablet-based tools before using ePRO [37]. Patients were informed about the aims of this study and were asked for their consent ex ante. The study was approved by the Ethics Committee at the University of Tübingen (project number 089/2015B02) [40].

    Procedure

    The data collection was performed in 4 parts. The first part focused on patientsʼ socioeconomic variables. The second part comprised the FACT-B questionnaire, consisting of 37 questions with responses required on a 5-point Likert scale (from 0, not at all through 4, very much) that constitute 5 dimensions [25-27,43]. The response options labels (not at all through very much; Figure 1) were the same in both the standard German paper-based and the tablet-based versions. While in the third part of the assessment, patients were asked about preexisting technical skills, their willingness to use ePRO, and potential barriers in relation to their health status [37], the fourth part concerned with the patients’ evaluation of the ePRO tool (manuscript in preparation). All patients completed the second part of the assessment both on paper and using a tablet, while they answered the questions in the other parts only in paper-based form. In this study, we report the results of the second part of the assessment (reliability analysis of the ePRO tool of FACT-B) [40].

    Specifics of the Electronic Patient-Reported Outcomes Tool

    For measuring ePRO, we used the “Patient-informiert- interaktiv-Arzt (PiiA),” that is “patient interactively informs doctor” Web-based app, which allows patients to answer the relevant questions on a tablet. The PiiA portal is a Web-based solution for capturing PROs, which our working group has developed. Patients receive anonymized user credentials and are asked to complete the FACT-B questionnaire. Figure 1 shows the user interface of the first set of questions of the German FACT-B. After completing the questionnaires, patients log out and their pseudoanonymized data are backed up on a local storage device and securely locked [40].

    Statistical Analyses

    All statistical analyses were conducted using SPSS Statistics (IBM, version 24). First, a frequency analysis was performed to determine the descriptive sociodemographic characteristics of patients. After that, we analyzed both dimensions of reliability (parallel forms reliability and test of internal consistency) and examined the disparity of responses and the rate of consistency between paper- and tablet-based responses. Both types of reliability were calculated for the 37 single items as well as for scores of the 5 dimensions, including the subscales for Physical Well-Being (PWB), Social/Family Well-Being (SWB), Emotional Well-Being (EWB), Functional Well-Being (FWB), and Breast Cancer Subscale (BCS), and the FACT-B total score in accordance with the FACT-B guidelines [25-27]. According to the Shapiro-Wilks test, the paired samples were not normally distributed; therefore, we used the Wilcoxon test to identify possible statistically significant differences in the test of parallel forms reliability both between the single items and the scores. Initially, the mean values of the paper- and tablet-based measures were calculated according to the official FACT-B guidelines [25-27]. Second, the consistency analyses were performed by calculation of Spearman rank correlation coefficient (Spearman ρ) and agreement rates for every FACT-B item along with rank correlation (Kendall tau) for each scale. Prior to that, we performed chi-square tests and Shapiro-Wilks test comparing metastatic and adjuvant breast cancer patients to identify possible statistically significant differences in relation to HRQoL. In all analyses, P<.05 (2-tailed) was considered indicative of statistically significant differences (alpha=.05).

    ‎
    Figure 1. Screenshot of the Patient-informiert-interaktiv-Arzt app’s FACT-B (Functional Assessment of Cancer Therapy-Breast) questionnaire for the dimension “physical well-being” (German). Source: Authors work, licensed under fair use.
    View this figure

    As such an analysis is considered an explorative study, all reported P values can be taken as purely descriptive. Missing values (which arose when patients did not answer individual questions) were appropriately taken into account in the calculation of the scores that were ignored in the statistical calculation [40]. Both figures (boxplot and correlation diagram) were generated in SPSS 24.


    Patient Enrollment

    Overall, 106 eligible female patients with breast cancer were recruited who completed questions from FACT-B both in a paper-based format and electronically using a tablet. Originally, 153 patients were assessed for eligibility, of which 47 were excluded during recruiting, allocation, and data analysis, as shown in the Consolidated Standards of Reporting Trials flow diagram (Figure 2).

    In all, 53 patients were assigned to tablet-based filling followed by paper-based filling in the same session (arm A), while the same number of patients completed the paper-based version followed by the tablet-based questionnaire (arm B). Both the paper- and tablet-based questionnaires were completed by patients consecutively during the same ambulance visit. Patients who had not completed more than half of the FACT-B questions in either paper- or tablet-based format were excluded (arm A, 1 patient; arm B, 2 patients). We did not find any significant differences between the 2 arms in the response behavior, sociodemographic status, or therapy setting; therefore, the 2 arms were considered together. Beforehand, all single items of the 2 arms were compared. Ten patients (arm A) and 16 patients (arm B), respectively, produced missing data in some questions (more often in the tablet-based questionnaire) [40].

    ‎
    Figure 2. Consolidated Standards of Reporting Trials flow diagram.
    View this figure

    Sociodemographic Variables

    Tables 1 and 2 shows the sociodemographic characteristics of the study group, with 71.6 (76/106) of patients in adjuvant therapy and 28.3% (30/106) in a metastatic situation. We did not find any marked intergroup differences between metastatic and adjuvant breast cancer patients in either ePRO or paper-based PRO. Although adjuvant and metastatic patients differed in age and HRQoL, as metastatic patients were older and reported a poorer HRQoL, we found no differences in the ePRO response behavior between the 2 groups. In addition, there were no differences in the reliability in any of the single items and scales between metastatic and adjuvant patients; hence, the complete patient collective was considered together. The mean age of the whole collective was 51.0 years; nearly one-third of the patients had a higher level of education (high school diploma) [40].

    Table 1. Sociodemographic characteristics of the patients (n=106).
    Sociodemographic variablesDescriptive analyses
    Age

    		Mean (SD)51 (11.31)

    		Median (range, minimum-maximum)52 (54, 30-84)
    Level of educationa

    		Median3

    		Interquartile range (25%-75% quantiles)2 (3-5)

    aLevel of education: 1=lowest; 5=highest.

    Table 2. Education level and therapy setting of the patients (n=106).
    VariableFrequency, n (%)95% CI
    Level of education

    		No qualification1 (0.9)0-6

    		Main or secondary school graduation43 (40.5)32-50

    		Advanced technical graduation19 (17.9)10-26

    		High school diplomaa33 (31.1)22-40

    		Not specified10 (9.4)2-15
    Therapy setting

    		Metastatic30 (28.3)19-35

    		Adjuvant setting76 (71.6)61-83

    aHigh school diploma indicates “Abitur.”

    Parallel Forms Reliability

    Table 3 presents the results of the Wilcoxon test for analyzing parallel forms reliability in the single items of FACT-B. The ePRO tool seems to demonstrate acceptable parallel forms reliability as only 2 significant differences (out of 37 in total) could be found in the single-item comparison. A weak statistically significant difference could only be identified in questions GP 4 (I have pain) and GS 2 (I get emotional support from my family). The pain reports were slightly higher in the ePRO questionnaire (P=.012), while emotional support was evaluated somewhat higher in paper-based PRO (P=.036). The differences were only slight, though, as the medians of the 2 items could not be distinguished from each other. In 35 of 37 questions in the FACT-B, no statistically significant differences were observed between patients’ answers in the paper-based questionnaire and ePRO.

    In addition, slight differences were noted between paper-based PRO and ePRO in the individual item scores of the 5 dimensions and the total FACT-B score (Table 4). The differences were significant in 2 dimensions (EWB and BCS) but not in the total scores because the scores included several questions (PWB, SWB, and FWB: 7 questions; EWB: 6 questions; BCS: 10 questions), whereby differences in patients’ responses to single questions could be multiplied in the scores and missing values caused fluctuations between the paper- and tablet-based PRO. The statistical differences in 2 scores, therefore, appear to be of methodological origin rather than attributable to differences in the response behavior. As only 5 response options (0-4) are available for each question in the FACT-B questionnaire, the ranking procedure often results in a large number of ties in the Wilcoxon test, which produces larger P values.

    The total score is slightly higher in ePRO (mean difference: 1.73; median difference: 0.63), but without statistically significant differences. Figure 3 shows the distribution of the paper-based and ePRO total score for FACT-B in a boxplot. It is obvious that the whisker but not the interquartile range is broader in the paper-based version.

    Table 3. Parallel forms reliability (Wilcoxon test) in single items.
    Single itemsPaper-based patient-reported outcomesElectronic patient-reported outcomesP value
    Mean (SD)Median (Interquartile range)Mean (SD)Median (Interquartile range)
    Physical Well-Being

    		GP11.68 (1.22)2 (2)1.59 (1.21)1 (1).22

    		GP20.54 (0.86)0 (1)0.60 (0.88)0 (1).26

    		GP31.34 (1.21)1 (2)1.29 (1.18)1 (2).72

    		GP40.98 (1.01)1 (1.25)1.03 (0.99)1 (2).01a

    		GP51.50 (1.17)1 (1)1.58 (1.09)1 (1).08

    		GP61.22 (1.12)1 (2)1.23 (1.05)1 (2).24

    		GP70.77 (1.09)0 (1)0.73 (1.03)0 (1)>.99
    Social/Family Well-Being

    		GS13.20 (0.94)3 (1)3.16 (1.04)3 (1).82

    		GS23.61 (0.73)4 (1)3.5 (0.86)4 (1).04a

    		GS33.18 (1.09)4 (1)3.14 (1.1)3 (1)>.99

    		GS43.33 (0.75)3 (1)3.28 (0.81)3 (1).39

    		GS53.37 (0.82)4 (1)3.33 (0.77)3 (1).11

    		GS63.65 (0.77)4 (.0)3.63 (0.82)4 (1).83

    		GS71.98 (1.66)2 (2.5)2.07 (1.11)2 (2).72
    Emotional Well-Being

    		GE11.32 (1.09)1 (1)1.22 (1.01)1 (1).40

    		GE22.68 (1.16)3 (2)2.91 (0.98)3 (2).05

    		GE30.60 (1.17)0 (1)0.44 (0.77)0 (1).11

    		GE41.19 (1.03)1 (2)1.12 (0.99)1 (2).83

    		GE51.22 (1.14)1 (2)1.18 (1.05)1 (1.5).70

    		GE61.42 (1.26)1 (1)1.32 (1.06)1 (1).59
    Functional Well-Being

    		GF12.12 (1.22)1 (1)2.21 (1.20)1 (1).23

    		GF22.30 (1.20)2 (2)2.32 (1.12)2 (2).34

    		GF32.50 (1.10)3 (1)2.51 (1.11)3 (1).81

    		GF42.58 (1.04)3 (1)2.55 (1.01)3 (1).39

    		GF52.40 (1.18)3 (1)2.41 (1.15)3 (1).25

    		GF62.53 (1.15)3 (1)2.65 (1.05)3 (1).20

    		GF72.19 (1.15)2 (1)2.17 (1.07)2 (1.5).81
    Breast Cancer Subscale

    		B10.75 (0.93)0.25 (1)0.72 (0.90)0.5 (1).49

    		B20.58 (1.10)0 (1)0.52 (0.99)0 (1).79

    		B30.68 (1.03)0 (1)0.63 (0.91)0 (1)>.99

    		B41.73 (1.04)2 (1)1.69 (1.01)2 (1).81

    		B51.47 (1.41)1 (2)1.47 (1.38)1 (2).49

    		B61.88 (1.41)2 (2)1.82 (1.45)1.5 (2).06

    		B72.02 (1.38)2 (2)2.14 (1.33)2 (2).33

    		B81.23 (1.34)1 (2)1.13 (1.37)1 (2).11

    		B92.55 (1.18)3 (1)2.62 (1.17)3 (2).58

    		P21.28 (1.11)1 (2)1.13 (1.05)1 (1.75).18

    aStatistically significant difference.

    Table 4. Parallel forms reliability (Wilcoxon test) for scoring values of 5 Functional Assessment of Cancer Therapy-Breast (FACT-B) dimensions (subscales).
    FACT-Ba dimensionsFACT-B questionsPaper-based patient-reported outcomesElectronic patient-reported outcomesP value
    Mean (SD)Median (Interquartile range)Mean (SD)Median (Interquartile range)
     
    Physical Well-Being Sum individual item scoresGP1-GP719.97 (6.11)21.0 (9.0)19.89 (5.88)20.0 (8.25)0.05
    Social/Family Well-Being Sum individual item scoresGS1-GS722.88 (3.93)24.0 (5.0)22.34 (4.60)23.33 (5.0)0.25
    Emotinal Well-Being Sum individual item scoresGE1-GE616.89 (4.84)18.0 (6.0)17.73 (4.68)18.0 (6.0).01b
    Functional Well-Being Sum individual item scoresGF1-GF716.73 (6.29)18.0 (9.0)16.75 (6.10)18.0 (9.45)0.69
    Breast Cancer Subscale Sum individual item scoresB1-B9; P226.35 (6.28)28.0 (9.0)28.56 (7.11)30.0 (11.33)<.001b
    FACT-B total score 102.66 (22.0)106.33 (28.81)104.39 (22.47)107 (30.31)0.05

    aFACT-B: Functional Assessment of Cancer Therapy-Breast

    aStatistically significant difference.

    ‎
    Figure 3. Distribution of total scores (Boxplot). PRO: patient-reported outcome.
    View this figure

    Test of Internal Consistency

    Table 5 shows the Spearman ρ correlation values and agreement rates, which were obtained for every question of the FACT-B questionnaire. In 27 questions, a high correlation (>.8) was found between paper-based PRO and ePRO, while the correlation was moderate (>.5) in the other 10 questions. In 20 questions, the correlation levels amounted to >.85. In all 37 correlated questions, agreement rates fluctuated between 65.7% and 91.8%.

    Table 6 shows the results of internal consistency testing for the sum individual item subscale scores and the total FACT-B score between paper-based PRO and ePRO. In all functional (sub)scales, the rank correlation was moderate to high as Kendall tau coefficient ranged between.64 and.80 in the sum individual item subscale scores and amounted to.80 in the total FACT-B score. The analysis of internal consistency tests showed the highest correlation in the sum individual item subscale scores EWB and FWB. All results of internal consistency tests were statistically highly significant. For illustrative purposes, Figure 4 represents a strong positive correlation between the ePRO and paper-based total FACT-B scores. Each data point reflects individual scores of patients.

    Table 5. Test of internal consistency in single items: results of correlation (Spearman ρ) and agreement analyses.
    DimensionsSpearman ρP value of Spearman ρaAgreement (%)
    Physical Well-Being

    		GP10.869<.00172.0

    		GP20.836<.00186.9

    		GP30.866<.00170.1

    		GP40.836<.00175.4

    		GP50.837<.00168.3

    		GP60.842<.00171.5

    		GP70.889<.00186.4
    Social/Family Well-Being

    		GS10.782<.00177.8

    		GS20.880<.00185.4

    		GS30.908<.00187.8

    		GS40.782<.00176.3

    		GS50.829<.00181.3

    		GS60.931<.00191.8

    		GS70.747<.00177.0
    Emotional Well-Being

    		GE10.753<.00171.0

    		GE20.525<.00166.6

    		GE30.796<.00182.3

    		GE40.868<.00178.0

    		GE50.931<.00185.8

    		GE60.733<.00165.7
    Functional Well-Being

    		GF10.881<.00174.0

    		GF20.770<.00169.7

    		GF30.889<.00181.0

    		GF40.821<.00171.5

    		GF50.934<.00187.2

    		GF60.910<.00182.4

    		GF70.897<.00175.2
    Breast Cancer Subscale

    		B10.858<.00186.4

    		B20.925<.00187.0

    		B30.904<.00187.9

    		B40.777<.00180.7

    		B50.900<.00180.6

    		B60.949<.00183.6

    		B70.843<.00169.5

    		B80.945<.00186.0

    		B90.684<.00170.1

    		P20.850<.00173.8

    aStatistically highly significant correlations.

    Table 6. Test of internal consistency in the individual subscale item scores and the total score: Kendall tau analysis.
    DimensionsKendall tau (95% CI)P value of Kendall taua
    Physical Well-Being Sum individual item scores0.784 (0.723-0.835)<.001
    Social/Family Well-Being Sum individual item scores0.648 (0.545-0.749)<.001
    Emotional Well-Being Sum individual item scores0.737 (0.638-0.820)<.001
    Functional Well-Being Sum individual item scores0.797 (0.731-0.858)<.001
    Breast Cancer Subscale Sum individual item scores0.638 (0.536-0.724)<.001
    Total score0.801 (0.741-0.852)<.001

    aStatistically significant correlations.

    ‎
    Figure 4. Correlation between electronic patient-reported outcome (ePRO) and paper-based total Functional Assessment of Cancer Therapy-Breast scores.
    View this figure

    Principal Results

    In both dimensions of reliability (parallel forms reliability and internal consistency), we found high correlations when comparing single items in the patients’ response behavior between paper-based PRO and ePRO in the FACT-B questionnaire. In the test of parallel forms reliability, we only found statistically significant differences in all but 2 questions. In the test of consistency, moderate to high correlations were found in all 37 single items and all sum individual item subscale scores. Based on the empirical results, the PiiA tool’s ePRO version of FACT-B seems to be reliable for measuring the HRQoL in patients with metastatic and adjuvant breast cancer. According to these results, we would not expect that future use of the PiiA tool in the same patient group will show marked differences between the paper-based PRO and ePRO version of FACT-B or that patients’ response behavior is significantly influenced by the survey tool after transfer to electronically based patient surveys. Therefore, the tool is suitable for determining the HRQoL in patients with metastatic or adjuvant breast cancer.

    Comparison With Prior Work

    Although ePRO apps are being used increasingly frequently, paper-based surveys of PRO still predominate in clinical research because reliable, electronically validated questionnaires are lacking. One of the most commonly used questionnaires for measuring the HRQoL, especially in patients with breast cancer, is the FACT-B, for which there is a reliable, paper-based format in many languages but no reliable, electronically based version exists in German. Using the electronically based version of FACT-B and other PRO without verifying the reliability could endanger the significance of ePRO surveys. Thus, a corresponding analysis in relation to differences and correlations between the paper-based version of FACT-B and newly developed Web-based tools are of great importance. It can be assumed that many aspects (ie, sociodemographic aspects, technical skills, health condition, and, perhaps, design specifics of the ePRO tool) may influence both patients’ willingness to use the tool and their response behavior, which underlines the need for reliability analyses [10,24,35-37]. However, almost no scientific studies have dealt with the reliability of ePRO questionnaires. Now, our working group has verified the reliability of an ePRO version of the FACT-B questionnaire in this paper, as well as the reliability of a tool for the ePRO measurement of the HRQoL in the EORTC questionnaire [40].

    Limitations

    Despite positive results, some limitations of the study design and methodological implementation should be mentioned, which could reduce the representativeness of the data. In 2 of 37 questions, we found statistically significant differences by the Wilcoxon test (parallel forms reliability). This can be a random observation based on numerous tests performed. A further explanation could be that patients were less concentrated owing to time pressure. Patients (both in arms A and B) were required to complete both paper-based and ePRO during an outpatient hospital visit. Patients were surveyed while they were receiving chemotherapy and were not permitted to take the questionnaire home to complete it; this also explains increasing missing values, especially in the last quarter of both questionnaire versions; possibly the length of the survey (paper-based and ePRO FACT-B and EORTC QLQ C-30, socioeconomic data, and evaluation of the tool) was too extensive for some patients. Possibly, the burden of disease and the therapy were potentially affecting the ability of some patients to complete both the paper- and tablet-based version of the questionnaire during an outpatient visit. Hence, a possible limiting factor was an inadequate screening as to whether all patients were able to cope with the psychological burden of participating in the study, as it is known that psycho-oncological distress is a commonly associated burden that could potentially influence the willingness to use ePRO and as a result ePRO’s reliability [37,44]. Nonetheless, the influence of psycho-oncological stress was likely low in this study, as the test of internal consistency showed no abnormalities. Both the individual questions and the sum individual item subscale scores showed consistently statistically significant correlations. It also needs to be noted that there was possibly a selection bias, as we did not examine whether the HRQoL was lower and the psychological distress was higher in those patients who could not be motivated to participate in the study. Therefore, it cannot be claimed with certainty that the tool per se is reliable for all patients with metastatic and adjuvant breast cancer; hence, further studies are needed that focus on the willingness to use ePRO depending on the state of health [10].

    Strengths of the Study

    Although ePRO is being used more and more often, questionnaires with proven reliability and validity are lacking. FACT-B is one of the most commonly used questionnaires worldwide for measuring the HRQoL in patients with breast cancer, but a reliable ePRO version is also missing here. One of the strengths of this study is that the reliability of a new tool for the ePRO measurement in patients with breast cancer was analyzed, while other studies often assign paper-based versions to a tablet-based format without verifying the reliability. The reliability of the PiiA tool could be ascertained for the questionnaires FACT-B (this paper) and EORTC QLQ C-30 [40] for measuring the HRQoL in metastatic and adjuvant breast cancer patients. The second strength is the methodological approach of the study ePROCOM and the statistical evaluation as all patients completed both the paper-based and ePRO questionnaire, and the reliability was ascertained in a multidimensional fashion (parallel forms reliability as well as internal consistency). Finally, the third strength points to the fact that ePRO tools are reliable and suitable in both the adjuvant and metastatic settings, although hurdles can be expected in these patient groups, in particular, depending on their health status, HRQoL, sociodemographic specifics, and technical ability and experience [10,37]. The results of this study can improve the quality of ePRO measurements as they seem to be generalizable, and the PiiA app of FACT-B can be used for reliable e-based measurement of the HRQoL in other studies and clinical routine.

    Conclusions

    Electronically based PRO is constantly being adopted in clinical research and clinical routine, which underlines the need for reliable questionnaires. The evaluated electronic version of the FACT-B is reliable for patients with breast cancer in an adjuvant or metastatic setting because high correlations were found in almost all questions, all subscales, and the total score. Thus, this study concludes that the validated paper-based questionnaire of FACT-B and the ePRO tool are equal.

    Acknowledgments

    This study was supported by the Klaus Tschira Stiftung, gemeinnützige GmbH.

    Conflicts of Interest

    PG has received honoraria from Novartis and financial support for symposia from Novartis, Roche, and PharmaMar.

    ‎

    Multimedia Appendix 1

    CONSORT-EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 161KB
    1. Robert Koch Institute. Berlin. 2016. Bericht zum Krebsgeschehen in Deutschland 2016   URL: http:/​/www.​krebsdaten.de/​Krebs/​DE/​Content/​Publikationen/​Krebsgeschehen/​Krebsgeschehen_download.​pdf?__blob=publicationFile [WebCite Cache]
    2. Verma S, Miles D, Gianni L, Krop IE, Welslau M, Baselga J, EMILIA Study Group. Trastuzumab emtansine for HER2-positive advanced breast cancer. N Engl J Med 2012 Nov 08;367(19):1783-1791 [FREE Full text] [CrossRef] [Medline]
    3. Heil J, Gondos A, Rauch G, Marmé F, Rom J, Golatta M, et al. Outcome analysis of patients with primary breast cancer initially treated at a certified academic breast unit. Breast 2012 Jun;21(3):303-308. [CrossRef] [Medline]
    4. Dufresne A, Pivot X, Tournigand C, Facchini T, Altweegg T, Chaigneau L, et al. Impact of chemotherapy beyond the first line in patients with metastatic breast cancer. Breast Cancer Res Treat 2008 Jan;107(2):275-279. [CrossRef] [Medline]
    5. Kassam F, Enright K, Dent R, Dranitsaris G, Myers J, Flynn C, et al. Survival outcomes for patients with metastatic triple-negative breast cancer: implications for clinical practice and trial design. Clin Breast Cancer 2009 Feb;9(1):29-33. [CrossRef] [Medline]
    6. Cortés J, Baselga J, Im Y, Im S, Pivot X, Ross G, et al. Health-related quality-of-life assessment in CLEOPATRA, a phase III study combining pertuzumab with trastuzumab and docetaxel in metastatic breast cancer. Ann Oncol 2013 Oct;24(10):2630-2635. [CrossRef] [Medline]
    7. Wallwiener M, Heindl F, Brucker SY, Taran F, Hartkopf A, Overkamp F, et al. Implementation and Feasibility of Electronic Patient-Reported Outcome (ePRO) Data Entry in the PRAEGNANT Real-Time Advanced and Metastatic Breast Cancer Registry. Geburtshilfe Frauenheilkd 2017 Aug;77(8):870-878 [FREE Full text] [CrossRef] [Medline]
    8. Karamouzis MV, Ioannidis G, Rigatos G. Quality of life in metastatic breast cancer patients under chemotherapy or supportive care: a single-institution comparative study. Eur J Cancer Care (Engl) 2007 Sep;16(5):433-438. [CrossRef] [Medline]
    9. Bitton A, Onega T, Tosteson ANA, Haas JS. Toward a better understanding of patient-reported outcomes in clinical practice. Am J Manag Care 2014 Apr;20(4):281-283 [FREE Full text] [Medline]
    10. Graf J, Simoes E, Wißlicen K, Rava L, Walter CB, Hartkopf A, et al. Willingness of Patients with Breast Cancer in the Adjuvant and Metastatic Setting to Use Electronic Surveys (ePRO) Depends on Sociodemographic Factors, Health-related Quality of Life, Disease Status and Computer Skills. Geburtshilfe Frauenheilkd 2016 May;76(5):535-541 [FREE Full text] [CrossRef] [Medline]
    11. Reilly CM, Bruner DW, Mitchell SA, Minasian LM, Basch E, Dueck AC, et al. A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment. Support Care Cancer 2013 Jun;21(6):1525-1550 [FREE Full text] [CrossRef] [Medline]
    12. Fromme EK, Eilers KM, Mori M, Hsieh Y, Beer TM. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. J Clin Oncol 2004 Sep 01;22(17):3485-3490. [CrossRef] [Medline]
    13. Pakhomov SV, Jacobsen SJ, Chute CG, Roger VL. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 2008 Aug;14(8):530-539 [FREE Full text] [Medline]
    14. Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst 2009 Dec 02;101(23):1624-1632 [FREE Full text] [CrossRef] [Medline]
    15. Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care 2013 Aug;51(8 Suppl 3):S73-S79 [FREE Full text] [CrossRef] [Medline]
    16. Valderas JM, Alonso J. Patient reported outcome measures: a model-based classification system for research and clinical practice. Qual Life Res 2008 Nov;17(9):1125-1135. [CrossRef] [Medline]
    17. Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med 2014;65:307-317. [CrossRef] [Medline]
    18. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol 2016 Feb 20;34(6):557-565 [FREE Full text] [CrossRef] [Medline]
    19. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol 2004 Feb 15;22(4):714-724. [CrossRef] [Medline]
    20. Wittmann V, Látos M, Horváth Z, Simonka Z, Paszt A, Lázár G, et al. What contributes to long-term quality of life in breast cancer patients who are undergoing surgery? Results of a multidimensional study. Qual Life Res 2017 Dec;26(8):2189-2199. [CrossRef] [Medline]
    21. Kreienberg R, Albert U, Follmann M, Kopp I, Kühn T, Wöckel A, et al. Leitlinienprogramm Onkologie. 2012. Interdisziplinäre S3-Leitlinie für die Diagnostik, Therapie und Nachsorge des Mammakarzinoms   URL: https:/​/www.​leitlinienprogramm-onkologie.de/​fileadmin/​user_upload/​LL_Mammakarzinom_Langversion_4.​0.​pdf [accessed 2018-11-15] [WebCite Cache]
    22. Jones SE. Metastatic breast cancer: the treatment challenge. Clin Breast Cancer 2008 Jun;8(3):224-233. [CrossRef] [Medline]
    23. Stefanovic S, Wallwiener M, Karic U, Domschke C, Katic L, Taran F, et al. Patient-reported outcomes (PRO) focused on adverse events (PRO-AEs) in adjuvant and metastatic breast cancer: clinical and translational implications. Support Care Cancer 2017 Dec;25(2):549-558. [CrossRef] [Medline]
    24. Wallwiener M, Simoes E, Sokolov AN, Brucker SY, Fasching PA, Graf J. Health-related Quality of Life in Metastatic and Adjuvant Breast Cancer Patients. Geburtshilfe Frauenheilkd 2016 Oct;76(10):1065-1073 [FREE Full text] [CrossRef] [Medline]
    25. Hahn E, Segawa E, Kaiser K, Cella D, Smith B. Validation of the Functional Assessment of Cancer Therapy-Breast (FACT-B-B) quality of life instrument. Journal of Clinical Oncology 2015;33(suppl e1):7753. [CrossRef]
    26. Brady MJ, Cella DF, Mo F, Bonomi AE, Tulsky DS, Lloyd SR, et al. Reliability and validity of the Functional Assessment of Cancer Therapy-Breast quality-of-life instrument. J Clin Oncol 1997 Mar;15(3):974-986. [CrossRef] [Medline]
    27. FACIT Measurement System. FACT-B scoring template. In: FACIT Functional Assessment of Chronic Illness Therapy. Elmhurst: Selfpublishing; 2013.
    28. Basch E, Goldfarb S. Electronic patient-reported outcomes for collecting sensitive information from patients. J Support Oncol 2009;7(3):98-99. [Medline]
    29. Abernethy AP, Herndon JE, Wheeler JL, Day JM, Hood L, Patwardhan M, et al. Feasibility and acceptability to patients of a longitudinal system for evaluating cancer-related symptoms and quality of life: pilot study of an e/Tablet data-collection system in academic oncology. J Pain Symptom Manage 2009 Jun;37(6):1027-1038. [CrossRef] [Medline]
    30. Jones JB, Snyder CF, Wu AW. Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res 2007 Oct;16(8):1407-1417. [CrossRef] [Medline]
    31. Basch E, Artz D, Iasonos A, Speakman J, Shannon K, Lin K, et al. Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. J Am Med Inform Assoc 2007;14(3):264-268 [FREE Full text] [CrossRef] [Medline]
    32. Basch E, Iasonos A, Barz A, Culkin A, Kris MG, Artz D, et al. Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. J Clin Oncol 2007 Dec 01;25(34):5374-5380. [CrossRef] [Medline]
    33. Snyder CF, Blackford AL, Wolff AC, Carducci MA, Herman JM, Wu AW, PatientViewpoint Scientific Advisory Board. Feasibility and value of PatientViewpoint: a web system for patient-reported outcomes assessment in clinical practice. Psychooncology 2013 Apr;22(4):895-901 [FREE Full text] [CrossRef] [Medline]
    34. Trautmann F, Hentschel L, Hornemann B, Rentsch A, Baumann M, Ehninger G, et al. Electronic real-time assessment of patient-reported outcomes in routine care-first findings and experiences from the implementation in a comprehensive cancer center. Support Care Cancer 2016 Dec;24(7):3047-3056. [CrossRef] [Medline]
    35. Graf J, Moreno B, Wallwiener M, Menzel K, Brucker SY, Simoes E. [Practicability and Efficiency of E-health Applications in Patient-Reported Outcomes: State of and Need for Research]. Gesundheitswesen 2018 Dec 27. [CrossRef] [Medline]
    36. Barentsz MW, Wessels H, van Diest PJ, Pijnappel RM, Haaring C, van der Pol CC, et al. Tablet, web-based, or paper questionnaires for measuring anxiety in patients suspected of breast cancer: patients' preferences and quality of collected data. J Med Internet Res 2014 Oct 31;16(10):e239 [FREE Full text] [CrossRef] [Medline]
    37. Hartkopf AD, Graf J, Simoes E, Keilmann L, Sickenberger N, Gass P, et al. Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients. JMIR Cancer 2017 Aug 07;3(2):e11 [FREE Full text] [CrossRef] [Medline]
    38. Campbell N, Ali F, Finlay AY, Salek SS. Equivalence of electronic and paper-based patient-reported outcome measures. Qual Life Res 2015 Aug;24(8):1949-1961. [CrossRef] [Medline]
    39. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, ISPOR ePRO Task Force. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health 2009 Jun;12(4):419-429 [FREE Full text] [CrossRef] [Medline]
    40. Wallwiener M, Matthies L, Simoes E, Keilmann L, Hartkopf AD, Sokolov AN, et al. Reliability of an e-PRO Tool of EORTC QLQ-C30 for Measurement of Health-Related Quality of Life in Patients With Breast Cancer: Prospective Randomized Trial. J Med Internet Res 2017 Sep 14;19(9):e322 [FREE Full text] [CrossRef] [Medline]
    41. Lachin JM. Statistical properties of randomization in clinical trials. Control Clin Trials 1988 Dec;9(4):289-311. [Medline]
    42. Schulz KF. Randomized controlled trials. Clin Obstet Gynecol 1998 Jun;41(2):245-256. [Medline]
    43. Kemmler G, Holzner B, Kopp M, Dünser M, Margreiter R, Greil R, et al. Comparison of two quality-of-life instruments for cancer patients: the functional assessment of cancer therapy-general and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-C30. J Clin Oncol 1999 Sep;17(9):2932-2940. [CrossRef] [Medline]
    44. Schaeffeler N, Pfeiffer K, Ringwald J, Brucker S, Wallwiener M, Zipfel S, et al. Assessing the need for psychooncological support: screening instruments in combination with patients' subjective evaluation may define psychooncological pathways. Psychooncology 2015 Dec;24(12):1784-1791. [CrossRef] [Medline]

    ‎
    BCS: Breast Cancer Subscale
    ePROCOM: electronic Patient-Reported Outcomes and Compliance Analysis
    EORTC: European Organisation for Research and Treatment of Cancer
    EWB: Emotional Well-Being
    FACT-B: Functional Assessment of Cancer Therapy-Breast
    FWB: Functional Well-Being
    HRQoL: health-related quality of life
    PEPPER: Patient Engagement Pilotstudie Mammakarzinom - individualisierte und Ressourcen-effiziente Patient Reported Outcomes Erfassung durch Digitale Therapieunterstützungssysteme
    PiiA: Patient-informiert-interaktiv-Arzt
    PRO: patient-reported outcomes
    PWB: Physical Well-Being
    QLQ: quality of life questionnaire
    SWB: Social/Family Well-Being

    Edited by G Eysenbach; submitted 31.01.18; peer-reviewed by A Birukov, Z Zhou; comments to author 25.08.18; revised version received 13.10.18; accepted 13.10.18; published 22.01.19

    Copyright

    ©Lina Maria Matthies, Florin-Andrei Taran, Lucia Keilmann, Andreas Schneeweiss, Elisabeth Simoes, Andreas D Hartkopf, Alexander N Sokolov, Christina B Walter, Nina Sickenberger, Stephanie Wallwiener, Manuel Feisst, Paul Gass, Michael P Lux, Florian Schuetz, Peter A Fasching, Christof Sohn, Sara Y Brucker, Joachim Graf, Markus Wallwiener. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 22.01.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.