This paper is in the following e-collection/theme issue:

Mobile Health (mhealth) (2748) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (457) mHealth for Data Collection and Research (907) Usability of Apps and User Perceptions of mHealth (892)

Published on in Vol 21, No 8 (2019): August

Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People With Multiple Sclerosis: Nonrandomized Interventional Feasibility Study

Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People With Multiple Sclerosis: Nonrandomized Interventional Feasibility Study

Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People With Multiple Sclerosis: Nonrandomized Interventional Feasibility Study

Authors of this article:

Luciana Midaglia1, 2 Author Orcid Image ;   Patricia Mulero1 Author Orcid Image ;   Xavier Montalban1, 3 Author Orcid Image ;   Jennifer Graves4 Author Orcid Image ;   Stephen L Hauser5 Author Orcid Image ;   Laura Julian6 Author Orcid Image ;   Michael Baker7 Author Orcid Image ;   Jan Schadrack7 Author Orcid Image ;   Christian Gossens7 Author Orcid Image ;   Alf Scotland7 Author Orcid Image ;   Florian Lipsmeier7 Author Orcid Image ;   Johan van Beek7 Author Orcid Image ;   Corrado Bernasconi7 Author Orcid Image ;   Shibeshih Belachew7 Author Orcid Image ;   Michael Lindemann7, 8 Author Orcid Image
Article Authors Cited by (73) Tweetations (20) Metrics

    Original Paper

    1Department of Neurology-Neuroimmunology, Multiple Sclerosis Centre of Catalonia, Vall d’Hebron University Hospital, Barcelona, Spain

    2Department of Medicine, Autonomous University of Barcelona, Barcelona, Spain

    3Division of Neurology, University of Toronto, Toronto, ON, Canada

    4Department of Neurology, University of California, San Diego, San Diego, CA, United States

    5Department of Neurology, University of California, San Francisco, San Francisco, CA, United States

    6Genentech Inc, South San Francisco, CA, United States

    7F Hoffmann–La Roche Ltd, Basel, Switzerland

    8Department of Economics, Baden-Wuerttemberg Cooperative State University, Loerrach, Germany

    Corresponding Author:

    Christian Gossens, MBA, PhD

    F Hoffmann–La Roche Ltd

    124 Grenzacherstrasse

    Basel,

    Switzerland

    Phone: 41 61 687 5113

    Fax:41 61 691 9391

    Email: christian.gossens@roche.com


    Related ArticleThis is a corrected version. See correction statement in: https://www.jmir.org/2019/10/e16287

    Background: Current clinical assessments of people with multiple sclerosis are episodic and may miss critical features of functional fluctuations between visits.

    Objective: The goal of the research was to assess the feasibility of remote active testing and passive monitoring using smartphones and smartwatch technology in people with multiple sclerosis with respect to adherence and satisfaction with the FLOODLIGHT test battery.

    Methods: People with multiple sclerosis (aged 20 to 57 years; Expanded Disability Status Scale 0-5.5; n=76) and healthy controls (n=25) performed the FLOODLIGHT test battery, comprising active tests (daily, weekly, every two weeks, or on demand) and passive monitoring (sensor-based gait and mobility) for 24 weeks using a smartphone and smartwatch. The primary analysis assessed adherence (proportion of weeks with at least 3 days of completed testing and 4 hours per day passive monitoring) and questionnaire-based satisfaction. In-clinic assessments (clinical and magnetic resonance imaging) were performed.

    Results: People with multiple sclerosis showed 70% (16.68/24 weeks) adherence to active tests and 79% (18.89/24 weeks) to passive monitoring; satisfaction score was on average 73.7 out of 100. Neither adherence nor satisfaction was associated with specific population characteristics. Test-battery assessments had an at least acceptable impact on daily activities in over 80% (61/72) of people with multiple sclerosis.

    Conclusions: People with multiple sclerosis were engaged and satisfied with the FLOODLIGHT test battery. FLOODLIGHT sensor-based measures may enable continuous assessment of multiple sclerosis disease in clinical trials and real-world settings.

    Trial Registration: ClinicalTrials.gov: NCT02952911; https://clinicaltrials.gov/ct2/show/NCT02952911

    J Med Internet Res 2019;21(8):e14863

    doi:10.2196/14863

    Keywords

    multiple sclerosis; patient adherence; patient satisfaction; smartphone; wearable electronic devices; mobile phone 


    Disease progression throughout the clinical course of multiple sclerosis (MS) is measured using clinician-reported outcomes, most commonly the Expanded Disability Status Scale (EDSS) [1], magnetic resonance imaging (MRI), and patient-reported outcomes (PROs). Conventional assessment of the clinical course of MS relies on relapse-associated and periodic in-clinic visits. However, the current intermittently conducted clinic-based outcome measures in MS have limitations in studying the insidiously subtle progression in MS and may fail to comprehensively capture transient symptomatic and performance fluctuations that affect people with multiple sclerosis.

    The ability of consumer wearable technology to measure functional impairment associated with various neurological disease symptoms through smartphone-based assessments is an important area of research [2-5]. Smartphones as vehicles for sensor-based technologies offer the potential for enhanced active and passive real-time data capture that may fundamentally shift traditional paradigms of clinical monitoring [6-9]. A recent large-scale study demonstrated that more than 95% of people with multiple sclerosis have access to a mobile device and most use it routinely [10]. Recently published studies have described the use of technologies in developing tools to assess people with multiple sclerosis [7,9,11-13]. In a recent study, it was reported that both healthy participants and people with multiple sclerosis were capable of completing daily tasks on a smartphone for 1 year [7]. Collection of data on a variety of cognitive and motor tests via the smartphone may represent a feasible way to gather highly granular data to accurately describe the MS disease course outside of the clinic [7].

    The FLOODLIGHT study [NCT02952911] was a prospective pilot study to assess the feasibility of remote measurements using smartphones and smartwatches in people with multiple sclerosis and healthy controls (HCs). The smartwatch and smartphone contained apps that prompted the user to perform various assessments and protocols, referred to as active tests. The app also passively recorded sensor data during daily life, referred to as passive monitoring. The novel smartphone- and smartwatch-based FLOODLIGHT active tests were developed to be self-administered by people with multiple sclerosis to capture MS symptoms, including hand motor function, gait and posture, mood, and cognitive impairment.

    The primary objectives of the FLOODLIGHT study, which began in November 2016, were to evaluate participant adherence to smartphone- and smartwatch-based assessments and collect feedback from people with multiple sclerosis and HCs on the smartphone and smartwatch schedule of assessments and its impact on their daily activities using a patient satisfaction questionnaire. Other objectives of the study, which will be addressed in subsequent publications, are to determine the association between exploratory sensor-based outcomes derived from the respective components of the FLOODLIGHT test battery and conventional MS clinical outcomes and explore whether the FLOODLIGHT test battery can differentiate between participants with and without MS.


    Trial Design and Participants

    After providing written informed consent, people with multiple sclerosis and HCs, preferentially partners or cohabitants, were evaluated for eligibility for enrollment in the FLOODLIGHT study. Eligibility criteria for people with multiple sclerosis included the ability to comply with the study protocol, age 18 to 55 years, diagnosis of MS (2010 revised McDonald criteria, treated or untreated) [14], EDSS score 0 to 5.5 (inclusive), and weight from 99 to 243 lbs (45 to 110 kg). An EDSS score of 5.5 as a maximum limit was meant to ensure any patient with relapsing or progressive MS would not have any significant difficulty in participating in the proposed testing as per study protocol. Further details on eligibility criteria are provided in Multimedia Appendix 1.

    This study was conducted at two sites in two countries with a total of 76 people with multiple sclerosis and 25 HCs; 60 people with multiple sclerosis and 20 HCs were recruited from the Multiple Sclerosis Centre of Catalonia, Vall d’Hebron University Hospital, Barcelona, Spain, and 16 people with multiple sclerosis and 5 HCs were recruited from the University of California, San Francisco, California.

    The protocol, informed consent forms, any information given to the participants, and relevant supporting information were reviewed and approved by the institutional review board/ethics committee before the study was initiated. Confidentiality was maintained by assigning each participant enrolled in the study a unique identification number.

    Study Design

    The FLOODLIGHT study combines continuous sensor data capture with smartphones and smartwatches and standard clinical outcome measures. Eligible people with multiple sclerosis and HCs were enrolled in the study and assessed clinically at the enrollment visit, week 12, and termination visit (week 24). In addition, participants were asked to perform a set of daily active tests and contribute sensor data via passive monitoring with smartphone and smartwatches over a period of 24 weeks (Figure 1).

    ‎
    Figure 1. FLOODLIGHT study design. PRO: patient-reported outcome. "a" indicates that active tests were administered weekly or every two weeks (see next figure for schedule). "b" indicates that brain magnetic resonance imaging was performed in people with multiple sclerosis.
    View this figure

    In-Clinic Assessments

    At each scheduled in-clinic visit (enrollment, week 12, and week 24), the following reference clinical tests were performed for all participants: 9-Hole Peg Test (9HPT), oral version of Symbol Digit Modalities Test (SDMT) [15-17], Timed 25-Foot Walk (T25FW) test, Berg Balance Scale (BBS) [18], Fatigue Scale for Motor and Cognitive Functions (FSMC) [19], and Patient Health Questionnaire–9 (PHQ-9) [20]. For people with multiple sclerosis only, disability was measured by EDSS [1], Patient Determined Disease Steps (PDDS) [21], and Multiple Sclerosis Impact Scale–29 (MSIS-29; version 2) [22-24]. While performing some of the in-clinic tests, people with multiple sclerosis and HCs were asked to carry or wear the smartphone and smartwatch to collect sensor data alongside the in-clinic measures. On the scheduled in-clinic visits, the smartphone and smartwatch FLOODLIGHT active tests were performed under investigator supervision. The satisfaction questionnaire (Multimedia Appendix 2) assessed people with multiple sclerosis’ and HCs’ experience regarding smartphone and smartwatch use and its impact on their daily activities at the week 12 visit and at the study termination/early discontinuation visit. Brain MRI was performed in people with multiple sclerosis at the enrollment visit and at week 24.

    Smartphone and Smartwatch Testing

    At the enrollment visit, people with multiple sclerosis and HCs were provided with the FLOODLIGHT solution that included a smartphone and smartwatch preconfigured so participants could only run the FLOODLIGHT software. A belt bag was also provided for participants to carry their smartphone in an anterior medial position. The smartphone and smartwatch pair contained preinstalled apps that prompted the user to perform various assessments, referred to as active tests. The apps also passively recorded sensor data, referred to as passive monitoring. At the enrollment visit, participants received training on the use of the smartphone and smartwatch and were provided with supporting content to help them complete the tests successfully. Participants were instructed to complete the active tests at approximately the same time each day and carry the smartphone and smartwatch throughout the day, recharging the devices overnight. Data transfer from the smartphone and smartwatch is described in Multimedia Appendix 1.

    FLOODLIGHT Active Tests

    People with multiple sclerosis and HCs were asked to perform various active tests (daily, weekly, every two weeks, or on demand) via the smartphone (Figure 2 and Table 1). These novel active tests were developed to be self-administered by people with multiple sclerosis to capture MS symptoms. A range of clinical and sensor-based assessments were chosen to capture the most prominent symptoms of MS from a broad spectrum of symptoms. People with multiple sclerosis and HCs were required to wear the smartwatch throughout the active tests.

    ‎
    Figure 2. FLOODLIGHT active tests and their schedule frequency. DMQ: Daily Mood Question; MSIS-29: Multiple Sclerosis Impact Scale–29; SBT: Static Balance Test; SDMT: Symbol Digit Modalities Test; ST: Symptom Tracker; 2MWT: Two-Minute Walk Test; 5UTT: 5 U-Turn Test.
    View this figure
    Table 1. FLOODLIGHT active tests.
    Domain and testShort description
    Daily hand motor function testsa

    		Draw a Shape (DaS) TestThe aim of the DaS Test is to assess fine finger/manual dexterity while the participants are instructed to hold the mobile device in the untested hand and draw on the smartphone touchscreen six prewritten alternating shapes of increasing complexity (linear, rectangular, circular, sinusoidal, and spiral) with the second finger of the tested hand as fast and as accurately as possible within a maximum time (30 seconds for each of the two attempts per shape).
     Pinching TestThe aim of the Pinching Test is to assess fine pinching/grasping dexterity while the participants are instructed to hold the mobile device in the untested hand and touch the screen with two fingers from the tested hand (thumb + second or thumb + third finger preferred) to squeeze/pinch as many round shapes (ie, tomatoes) as they can during 30 seconds.
    Daily gait testsb

    		Two-Minute Walk Test (2MWT)Participants are instructed to walk as fast and as long as they can for 2 minutes but walk safely. The 2MWT is a simple test that is required to be performed on an even ground in a place where participants have identified they could walk straight for as far as ≥200 meters without U-turns. Participants are allowed to wear regular footwear and an assistive device and/or orthotic as needed.
     5 U-Turn Test (5UTT)The aim of this test is to assess difficulties or unusual patterns in performing U-turns while walking on a short distance at comfortable pace. The 5UTT can be performed indoors or outdoors, on an even ground where participants are instructed to walk safely and perform five successive U-turns going back and forward between two points a few meters apart for 1 minute. Participants are allowed to wear regular footwear and an assistive device and/or orthotic as needed.
     Static Balance Test (SBT)Participants are asked to stand still unsupported for 30 seconds with relaxed arms straight alongside the body if possible.
    Weekly cognitive test

    		Electronic version of the Symbol Digit Modalities Test (SDMT) [15-17]The aim of SDMT testing is to detect impairment of key neurocognitive functions that underlie many substitution tasks.
    Patient-reported outcomes (PROs)

    		Daily Mood Question (DMQ)This test represents an assessment of participants’ perceived overall state by responding daily to the question “How do you feel now?” on a 5-item Likert scale, ranging from excellent to horrible.
     Electronic version of the Multiple Sclerosis Impact Scale–29, version 2 (MSIS-29) [22-24]; people with multiple sclerosis onlyThis questionnaire measures the physical and psychological impact of multiple sclerosis.
     Multiple Sclerosis Symptom Tracker (MSST); people with multiple sclerosis onlyPatients are asked if they experienced new or significantly worsening symptoms during the last 2 weeks. If yes, onset of the symptoms and the patients’ perception to whether they think they experienced a relapse (yes, no, or unsure) are recorded.

    aTests alternatingly performed with right and left hand; users are instructed on daily alternation.

    bRecommended position of smartphone in an anterior medial position in the belt bag.

    FLOODLIGHT Passive Monitoring

    Passive monitoring collected metrics on gait and mobility throughout the day in a continuous and unobtrusive manner. Participants were instructed to carry their smartphone preferably in an anterior medial position in a belt bag or, alternatively, in their pocket and wear the smartwatch all day as they went about their daily routine until the devices ran out of charge.

    Statistical Analyses

    The analyses of the primary objectives of this study were descriptive. Statistical tests were exploratory and conducted at the two-sided 5% significance level without adjustment for multiple comparisons. The analyses were based on all enrolled patients (full analysis set [FAS]). Patients who prematurely withdrew from the study for any reason were still included in the FAS. Supportive analyses of selected variables were carried out in the per-protocol population, which included people with multiple sclerosis who completed at least 1 week in the study and did not discontinue due to an adverse event or a reason unrelated to the use of the FLOODLIGHT solution (Multimedia Appendix 3).

    Adherence was evaluated for the following tests and test groups: all FLOODLIGHT active tests, Two-Minute Walk Test (2MWT), all active tests except 2MWT, smartphone use, smartwatch use, and for the per-protocol population.

    Adherence to active tests was measured as the proportion of study weeks with at least 3 days of completed testing (study co-primary endpoint). Adherence to sensor-based passive monitoring was measured as the proportion of study weeks with at least 3 days of passive monitoring for at least 4 hours per day while the devices were worn by the participant [25] (study co-primary endpoint). Descriptive statistics of calculated adherence were reported for all active tests, all active tests except 2MWT, 2MWT, smartphone use, and smartwatch use. Categorical and numeric variables with fewer than five values were tested for association with adherence using the Kruskal-Wallis test. The association between continuous variables was assessed using the Spearman rank correlation.

    Participant complete abandoning of active testing and passive monitoring was also investigated in a time-to-event survival analysis based on the Kaplan-Meier method along the FLOODLIGHT study. The abandoning event was defined as the last week in which the participant was adherent according to the definitions above for active tests and passive monitoring. Active tests performed on days of in-clinic visits were not considered in the adherence calculation to focus the abandoning analysis on the remote use. Participants leaving the study before the terminal visit were considered as censored. The impact of different characteristics on adherence was assessed using Cox regression.

    A satisfaction score was developed from the satisfaction questionnaire (Multimedia Appendix 2) that sums the individual answers to questions 1-7 and 10-12 rescaled to 0-100 from their original Likert scale. An interquestion correlation analysis was performed to ensure questions are equally correlated and can be combined. Descriptive statistics of satisfaction score and items are reported, along with covariate analyses of demographics and disease state. Analysis of the change in satisfaction score between week 12 and week 24 are reported using the Wilcoxon signed-rank test.

    Patient baseline characteristics incorporated as covariates in the analysis of correlation with FLOODLIGHT adherence and satisfaction outcomes were age, gender, body mass index, time since first MS symptom onset, EDSS, T25FW time, 9HPT time, and the oral SDMT correct responses. A descriptive analysis of safety variables, including adverse events and serious adverse events, was carried out in the FAS.


    Baseline Demographics and Characteristics

    Participants’ baseline demographics for the FAS are described in Table 2. There was an expected greater proportion of females among the people with multiple sclerosis compared with HCs. The majority (69/76, 91%) of people with multiple sclerosis had relapsing-remitting MS (RRMS), with a mild EDSS score (mean 2.4) and presumably “normal” hand/arm function based on an upper limit of normal range defined as the average 9HPT time for HCs plus two standard deviations [26] (Table 2).

    Overall, 92% (70/76) of people with multiple sclerosis and 64% (16/25) of HCs who enrolled in the FLOODLIGHT study reached the week 24 visit. Reasons for discontinuation from the study are described in Multimedia Appendix 3.

    Table 2. Demographics and characteristics of people with multiple sclerosis and healthy controls (HCs) at baseline.
    ParameterPeople with multiple sclerosis (n=76)HCs (n=25)
    Age (years), mean (SD)39.5 (7.9)34.9 (9.3)
    Female, n (%)53 (70)7 (28)
    Multiple sclerosis (MS)diagnosis, n (%)


    		Primary progressive multiple sclerosis3 (4)—a

    		Secondary progressive multiple sclerosis4 (5)—

    		Relapsing-remitting multiple sclerosis69 (91)—
    Time since MS symptom onset (years), mean (SD)11.3 (7.0)b—
    Proportion of people with multiple sclerosis with ≥1 relapse in the past year, n (%)18 (24)—
    Expanded Disability Status Scale, mean (SD)2.4 (1.4)—
    Proportion of people with multiple sclerosis with ≥1 T1 Gdc-enhancing lesion, n (%)2 (3)d—
    Total FLAIRe lesion volume (mL), mean (SD)6.3 (7.5)f—
    9-Hole Peg Test(seconds), mean (SD)


    		Dominant hand22.1 (4.6)g18.9 (2.1)

    		Nondominant hand22.8 (4.9)h19.5 (2.0)
    Timed 25-Foot Walk (seconds), mean (SD)6.0 (2.1)b5.0 (1.0)
    Symbol Digit Modalities Test (correct responses), mean (SD)53.8 (11.8)b63.8 (10.0)
    Berg Balance Scale, mean (SD)52.5 (5.7)i56.0 (0)j
    Patient Determined Disease Steps, mean (SD)1.5 (1.6)—
    Fatigue Scale for Motor and Cognitive Functions (total score), mean (SD)59.1 (22.7)g25.5 (6.0)
    Patient Health Questionnaire–9, mean (SD)8.3 (6.1)k2.4 (2.9)l
    Participants with any previous medications, n (%)46 (61)6 (24)
    Previous disease-modifying treatmentm, n (%)


    		Daclizumab (Zinbryta)0 (0)—

    		Glatiramer acetate (Copaxone)12 (16)—

    		Glatiramer acetate (Glatopa)1 (1)—

    		IFNn β-1a IMo (Avonex)4 (5)—

    		IFN β-1a SCp (Rebif)5 (7)—

    		IFN β-1b SC (Betaseron/Betaferon)6 (8)—

    		IFN β-1b SC (Extavia)1 (1)—

    		Pegylated IFN β-1a (Plegridy)2 (3)—

    		Dimethyl fumarate (Tecfidera)9 (12)—

    		Fingolimod (Gilenya)9 (12)—

    		Teriflunomide (Aubagio)3 (4)—

    		Alemtuzumab (Lemtrada)2 (3)—

    		Mitoxantrone (Novantrone)1 (1)—

    		Natalizumab (Tysabri)19 (25)—

    		Otherq5 (7)—

    aNot applicable.

    bn=75.

    cGd: gadolinium.

    dn=68.

    eFLAIR: fluid-attenuated inversion recovery.

    fn=70.

    gn=73.

    hn=74.

    in=71.

    jn=22.

    kn=60.

    ln=20.

    mTotal baseline disease-modifying treatment history.

    nIFN: interferon.

    oIM: intramuscular.

    pSC: subcutaneous.

    qHidroferol; Radiance study (RPC1063 versus IFN β-1a); Rituximab (Rituxan).

    Adherence

    Over a period of 18 months (November 2016-April 2018), more than 6 terabytes of raw data were collected from 76 people with multiple sclerosis and 25 HCs. Participants performed 67,544 active tests, of which 9787 were the 2MWT, and recorded 200,171 hours of passive monitoring, of which 113,165 hours were captured with the smartwatch. Over 24 weeks, most participants performed 5 to 7 active tests per week, including the 2MWT (Figure 3). Adherence of people with multiple sclerosis to completing active tests and passive monitoring was good and remained stable over time after week 6 (Figures 4 and 5). Even in the last week of the 24-week study, participants completed all active tests on average 4 out of 7 days per week (Figure 4), and recorded at least 4 hours of data via passive monitoring on average 4 out of 7 days per week (Figure 5). The lowest average adherence over 24 weeks was observed for active tests including the 2MWT and the 2MWT only, with participants showing highest average adherence for passive monitoring (Figure 6). A total of 70% (16.68/24 weeks) of participants were adherent to all active tests, 75% (17.95/24 weeks) to all active tests except 2MWT, 71% (17.13/24 weeks) to 2MWT, 79% (18.89/24 weeks) to smartphone- or smartwatch-based passive monitoring, 66% (15.74/24 weeks) to smartphone-based passive monitoring, and 74% (17.69/24 weeks) to smartwatch-based passive monitoring.

    ‎
    Figure 3. Adherence of people with multiple sclerosis to active tests for individual participants: number of performed active tests per week [level of activity (light green: high; dark green/grey: low] over individual study weeks [columns]).
    View this figure
    ‎
    Figure 4. Adherence of people with multiple sclerosis to active tests. 2MWT: Two-Minute Walk Test.
    View this figure
    ‎
    Figure 5. Adherence of people with multiple sclerosis to smartphone and smartwatch passive monitoring. Days with more than 4 hours of passive monitoring on a device are considered as adherent.
    View this figure
    ‎
    Figure 6. Adherence of people with multiple sclerosis to active tests and passive monitoring. The results of the time-to-event survival analysis based on the Kaplan–Meier method along the FLOODLIGHT study. The abandoning event was defined as the last week in which the participant was adherent according to the definitions for active tests and passive monitoring. Active tests performed on days of in-clinic visits were not considered in the adherence calculation, to focus the abandoning analysis on the remote use. Participants leaving the study before the terminal visit were considered as censored. 2MWT: Two-Minute Walk Test.
    View this figure

    Correlation was explored between adherence measures and people with multiple sclerosis population characteristics. Only disease duration showed significant small negative correlation with measures of adherence (Spearman rank correlations; 2MWT adherence: –0.42, P<.001; smartphone passive monitoring adherence: –0.29, P=.02; all active tests except 2MWT adherence: –0.37, P=.003; and smartwatch passive monitoring adherence: –0.27, P=.04), indicating that disease severity and demographics did not appear to play a significant role in adherence.

    Patient Satisfaction

    The average overall satisfaction score among people with multiple sclerosis who completed the study at week 12 (n=64) was 74.1 out of a possible 100 and remained stable at week 24 (study termination/early discontinuation visit [n=68]) with 73.7 out of 100 (Wilcoxon signed-rank test P=.71). There was one significant association between overall satisfaction score and gender (P=.04). Individual questions from the satisfaction questionnaire (Multimedia Appendix 2) were analyzed for their association with people with multiple sclerosis population characteristics, described in Multimedia Appendix 1.

    Implications for the use of the FLOODLIGHT test battery in people with multiple sclerosis were assessed from individual questions from the patient satisfaction questionnaire. When asked to rate the impact of the smartphone, smartwatch, and active tests on daily living, more than 80% (61/72) of people with multiple sclerosis perceived the FLOODLIGHT test battery to have at least acceptable impact on daily activities (Figure 7). Nearly 50% (32/71) of participants had no issue with any of the active tests, and only one-third would prefer to avoid the 2MWT, most likely due to increased burden from execution—for example, having to find a place to perform the test or not wanting to go outside in bad weather (Figure 8). Without providing any data feedback to the people with multiple sclerosis throughout the study, more than 60% (46/72) of participants would have liked to continue using FLOODLIGHT “to understand my MS better and improve my disease management” (Figure 9). Approximately 90% (65/72) of people with multiple sclerosis indicated their interest to see the results of the tests, which will be addressed in future Roche-sponsored studies using FLOODLIGHT (CONSONANCE [NCT03523858] and FLOODLIGHT Open [floodlightopen.com]; Figure 10). Analysis of patient responses to the satisfaction questionnaire is described in Multimedia Appendix 1.

    ‎
    Figure 7. Implications of FLOODLIGHT in people with multiple sclerosis for “impact on daily activities” from the patient satisfaction questionnaire.
    View this figure
    ‎
    Figure 8. Implications of FLOODLIGHT in people with multiple sclerosis for “avoiding one component of FLOODLIGHT" from the patient satisfaction questionnaire. 2MWT: Two-Minute Walk Test.
    View this figure
    ‎
    Figure 9. Implications of FLOODLIGHT in people with multiple sclerosis for “desire to continue using the FLOODLIGHT app” from the patient satisfaction questionnaire.
    View this figure
    ‎
    Figure 10. Implications of FLOODLIGHT in people with multiple sclerosis for “prefer to see results immediately to monitor” from the patient satisfaction questionnaire.
    View this figure

    Principal Findings

    This study demonstrates that the use of smartphones and smartwatches for remote daily active testing and continuous passive monitoring is feasible over 6 months and provides further support to earlier studies, which have shown that healthy participants and people with multiple sclerosis were capable of completing daily tasks on a smartphone [7]. This study provides further evidence for the use of digital technology, including smartphones, for data collection. Other studies in MS have used smartphone apps to (1) assess steps when walking on a treadmill [9]; (2) assess pain, fatigue, anxiety, and quality of life [13]; and (3) assess the feasibility of gathering passive and active performance data [7]. Together with the current analyses, these studies document the focus toward developing digital measures to continuously monitor and assess the MS disease.

    In this protocol, the FLOODLIGHT solution collected metrics on cognition, mood, upper extremity function, and gait and posture by instructing participants to perform a set of daily active tests, which should take approximately 5 minutes in total to complete and capture activity data via passive monitoring over a period of 24 weeks. A previous study has shown that 51% of participants (22/38 of people with multiple sclerosis and 17/38 of healthy participants) completed 12 months of daily data collection, where participants were prompted to complete one assigned test [7]. In the context of the FLOODLIGHT study, we observed that overall adherence to active tests was 70% (16.68/24 weeks), which appears to be higher than the adherence of participants to 12 months of daily data collection (39/76, 51%) from Bove et al [7]. However, comparisons between the studies are limited, as the study design and burden of testing are different—for example, the app from Bove et al contained 19 different tests, of which participants were prompted to complete one each day. As the FLOODLIGHT app was integrated into standalone devices in this study, deployment of the app on participants’ own mobile devices may increase adherence because it removes the need to carry a separate, dedicated device and decreases burden on the individual.

    Limitations

    As this study remains a pilot investigation designed to collect first experiences from continuous sensor data capture, the main limitation is the small sample size and short duration of follow-up. Future ongoing FLOODLIGHT studies (CONSONANCE and FLOODLIGHT Open) will collect longer term data on smartphone-based sensor data capture in a larger number of participants from a broader disability spectrum. Additionally, whether physical and cognitive limitations in people with secondary progressive MS (SPMS) and people with primary progressive MS (PPMS) differentially impacts adherence compared with people with relapsing MS (RMS) cannot be gleaned from the current data set due to the low numbers of advanced patients enrolled in the study; however, this important question warrants future research exploring remote monitoring in patients with more advanced MS. The importance of continuous monitoring in RMS should also not be overlooked, as the sensitivity of this novel approach aiming at detecting progression in a real-world setting may provide an earlier window into disease progression outside of the clinic.

    Comparison With Prior Work

    A recent study assessing the feasibility of the MS TeleCoach, a novel intervention offering telemonitoring of fatigue and telecoaching of physical activity in people with multiple sclerosis, showed that participants were highly engaged, with 76% (57/75) of participants completing the study, and 91% (21/23) of a subset of completers showing a median of quite satisfied in the patient satisfaction questionnaire [27]. During the 12-week study period, use of the MS TeleCoach improved fatigue levels in people with multiple sclerosis with moderate to severe fatigue, suggesting that implementation of digital technologies can enhance patient performance. Together with the data presented here, these results indicate that the use of consumer devices by people with multiple sclerosis for sensor data capture fulfills the prerequisites of people with multiple sclerosis satisfaction and acceptable adherence to daily active tests and passive monitoring for potential integration in long-term clinical trials and treatment monitoring. Regarding future studies, for example FLOODLIGHT Open, attempts will be made to improve participant adherence throughout the study by introducing controlled app variations, such as reminders, types of achievements, and fun metrics.

    Conclusions

    In summary, these analyses showed that people with multiple sclerosis are highly engaged with performing active tests and capturing continuous data via passive monitoring and are satisfied with the FLOODLIGHT test battery. Neither satisfaction nor adherence showed strong correlation with study population characteristics. More than 60% (46/72) of people with multiple sclerosis indicated their interest to continue to use FLOODLIGHT, and approximately 90% (65/72) wanted to see the results of their tests in real time as biofeedback, which was implemented in future studies using the FLOODLIGHT solution. These findings indicate that smartphone-based FLOODLIGHT outcomes may represent a promising avenue to enable a more accurate and continuous assessment of MS disease in clinical trials and real-world practice settings and may eventually also contribute to informing and guiding clinical research and clinical practice in the future.

    Acknowledgments

    We would like to thank all patients, their families, and the investigators who participated in this trial. This research was funded by F Hoffmann–La Roche Ltd, Basel, Switzerland. We would like to thank the following employees from F Hoffmann–La Roche Ltd who supported and contributed to the study: Atieh Bamdadian, Alessandro Barbato, Jan Beckmann, Sandro Fritz, Nicholas Pierce Heinemeier, Timothy Kilchenmann, Lito Kriara, Bernd Laub, Grégoire Pointeau, Caroline Polakowska, Marcin Puhacz, Cedric Simillion, Jens Schjodt-Eriksen, Jöerg Sprengel, Ralf Stubner, and Krzysztof Trybus. Writing and editorial assistance for this manuscript was provided by Heather Latimer from Articulate Science, United Kingdom, and funded by F Hoffmann–La Roche Ltd.

    Qualified researchers may request access to individual patient-level data through the clinical study data request platform (https://clinicalstudydatarequest.com). Further details on Roche’s criteria for eligible studies are available at https://clinicalstudydatarequest.com/Study-Sponsors/Study-Sponsors-Roche.aspx. For further details on Roche’s Global Policy on the Sharing of Clinical Information and how to request access to related clinical study documents, see https://www.roche.com/research_and_development/who_we_are_how_we_work/clinical_trials/our_commitment_to_data_sharing.htm.

    The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: F Hoffmann–La Roche Ltd, Basel, Switzerland, provided financial support for the study and publication of this manuscript.

    Conflicts of Interest

    XM has received speaker honoraria and travel expense reimbursement for participation in scientific meetings, been a steering committee member of clinical trials, or served on advisory boards of clinical trials for Actelion, Biogen, Celgene, Merck, Novartis, Oryzon, Roche, Sanofi Genzyme, and Teva Pharmaceutical. JG has received grants or research support from Biogen, Genentech Inc, and S3 Group and has received compensation for a nonbranded resident and fellow education seminar supported by Biogen. SLH serves on the scientific advisory boards for Annexon, Symbiotix, Bionure, and Molecular Stethoscope, is on the board of trustees for Neurona Therapeutics, and has received travel reimbursement and writing assistance from F Hoffmann–La Roche Ltd for CD20-related meetings and presentations. LJ is an employee of Genentech Inc and a shareholder of F Hoffmann–La Roche Ltd. MB, JS, and CG are employees and shareholders of F Hoffmann–La Roche Ltd. AS, FL, and JvB are employees of F Hoffmann–La Roche Ltd. CB and ML are contractors for F Hoffmann–La Roche Ltd. SB was an employee of F Hoffmann–La Roche Ltd during the completion of the work related to this manuscript. SB is now an employee of Biogen (Cambridge, MA), which was not in any way associated with this study. LM and PM have nothing to disclose.

    ‎
    Multimedia Appendix 1

    Additional trial information.

    PDF File (Adobe PDF File), 278KB
    ‎
    Multimedia Appendix 2

    Satisfaction questionnaire.

    PDF File (Adobe PDF File), 238KB
    ‎
    Multimedia Appendix 3

    Participant flow table.

    PDF File (Adobe PDF File), 138KB
    1. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 1983 Nov;33(11):1444-1452. [Medline]
    2. Bradshaw MJ, Farrow S, Motl RW, Chitnis T. Wearable biosensors to monitor disability in multiple sclerosis. Neurol Clin Pract 2017 Aug;7(4):354-362. [CrossRef] [Medline]
    3. Lavorgna L, Brigo F, Moccia M, Leocani L, Lanzillo R, Clerico M, et al. e-Health and multiple sclerosis: An update. Mult Scler 2018 Nov;24(13):1657-1664. [CrossRef] [Medline]
    4. Busis N. Mobile phones to improve the practice of neurology. Neurol Clin 2010 May;28(2):395-410. [CrossRef] [Medline]
    5. Lipsmeier F, Taylor KI, Kilchenmann T, Wolf D, Scotland A, Schjodt-Eriksen J, et al. Evaluation of smartphone-based testing to generate exploratory outcome measures in a phase 1 Parkinson's disease clinical trial. Mov Disord 2018 Dec;33(8):1287-1297 [FREE Full text] [CrossRef] [Medline]
    6. Prasad S, Ramachandran R, Jennings C. Development of smartphone technology to monitor disease progression in multiple sclerosis. Neurology 2012 Apr 22;78(Meeting Abstracts 1):P01.144. [CrossRef]
    7. Bove R, White CC, Giovannoni G, Glanz B, Golubchikov V, Hujol J, et al. Evaluating more naturalistic outcome measures: a 1-year smartphone study in multiple sclerosis. Neurol Neuroimmunol Neuroinflamm 2015 Dec;2(6):e162 [FREE Full text] [CrossRef] [Medline]
    8. Boukhvalova AK, Kowalczyk E, Harris T, Kosa P, Wichman A, Sandford MA, et al. Identifying and quantifying neurological disability via smartphone. Front Neurol 2018;9:740 [FREE Full text] [CrossRef] [Medline]
    9. Balto JM, Kinnett-Hopkins DL, Motl RW. Accuracy and precision of smartphone applications and commercially available motion sensors in multiple sclerosis. Mult Scler J Exp Transl Clin 2016;2:2055217316634754 [FREE Full text] [CrossRef] [Medline]
    10. Haase R, Schultheiss T, Kempcke R, Thomas K, Ziemssen T. Modern communication technology skills of patients with multiple sclerosis. Mult Scler 2013 Aug;19(9):1240-1241. [CrossRef] [Medline]
    11. Rudick RA, Miller D, Bethoux F, Rao SM, Lee J, Stough D, et al. The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool. J Vis Exp 2014 Jun 30(88):e51318 [FREE Full text] [CrossRef] [Medline]
    12. Gholami F, Trojan DA, Kovecses J, Haddad WM, Gholami B. A Microsoft Kinect-based point-of-care gait assessment framework for multiple sclerosis patients. IEEE J Biomed Health Inform 2017 Dec;21(5):1376-1385. [CrossRef] [Medline]
    13. Kos D, Raeymaekers J, Van Remoortel A, D'hooghe MB, Nagels G, D'Haeseleer M, et al. Electronic visual analogue scales for pain, fatigue, anxiety and quality of life in people with multiple sclerosis using smartphone and tablet: a reliability and feasibility study. Clin Rehabil 2017 Sep;31(9):1215-1225. [CrossRef] [Medline]
    14. Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol 2011 Feb;69(2):292-302 [FREE Full text] [CrossRef] [Medline]
    15. Smith A. The symbol-digit modalities test: A neuropsychologic test of learning and other cerebral disorders. Seattle: Helmuth J, editor. Learning disorders. Seattle: Special Child Publications; 1968:83-91.
    16. Smith A. Symbol Digits Modalities Test: Manual. Los Angeles: Western Psychological Services; 1982.
    17. Rao SM, Losinski G, Mourany L, Schindler D, Mamone B, Reece C, et al. Processing speed test: validation of a self-administered, iPad-based tool for screening cognitive dysfunction in a clinic setting. Mult Scler 2017 Dec;23(14):1929-1937. [CrossRef] [Medline]
    18. Berg KO, Wood-Dauphinee SL, Williams JI, Maki B. Measuring balance in the elderly: validation of an instrument. Can J Public Health 1992;83 Suppl 2:S7-S11. [Medline]
    19. Penner IK, Raselli C, Stöcklin M, Opwis K, Kappos L, Calabrese P. The Fatigue Scale for Motor and Cognitive Functions (FSMC): validation of a new instrument to assess multiple sclerosis-related fatigue. Mult Scler 2009 Dec;15(12):1509-1517. [CrossRef] [Medline]
    20. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline]
    21. Learmonth YC, Motl RW, Sandroff BM, Pula JH, Cadavid D. Validation of patient determined disease steps (PDDS) scale scores in persons with multiple sclerosis. BMC Neurol 2013 Apr 25;13:37 [FREE Full text] [CrossRef] [Medline]
    22. Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technol Assess 2009 Feb;13(12):1-177 [FREE Full text] [CrossRef] [Medline]
    23. Hobart J, Lamping D, Fitzpatrick R, Riazi A, Thompson A. The Multiple Sclerosis Impact Scale (MSIS-29): a new patient-based outcome measure. Brain 2001 May;124(Pt 5):962-973. [CrossRef] [Medline]
    24. Riazi A, Hobart JC, Lamping DL, Fitzpatrick R, Thompson AJ. Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry 2002 Dec;73(6):701-704 [FREE Full text] [CrossRef] [Medline]
    25. Cheng W, Scotland A, Lipsmeier F, Kilchenmann T, Jin L, Schjodt-Eriksen J, et al. Human activity recognition from sensor-based large-scale continuous monitoring of Parkinson's disease patients. 2017 Presented at: Proceedings of the 2nd IEEE/ACM International Conference on Connected Health; July 17-19, 2017; Philadelphia. [CrossRef]
    26. Wang Y, Bohannon RW, Kapellusch J, Garg A, Gershon RC. Dexterity as measured with the 9-Hole Peg Test (9-HPT) across the age span. J Hand Ther 2015;28(1):53-59. [CrossRef] [Medline]
    27. D'hooghe M, Van Gassen G, Kos D, Bouquiaux O, Cambron M, Decoo D, et al. Improving fatigue in multiple sclerosis by smartphone-supported energy management: the MS TeleCoach feasibility study. Mult Scler Relat Disord 2018 May;22:90-96 [FREE Full text] [CrossRef] [Medline]

    ‎
    2MWT: Two-Minute Walk Test
    5UTT: 5 U-Turn Test
    9HPT: 9-Hole Peg Test
    BBS: Berg Balance Scale
    DMQ: Daily Mood Question
    EDSS: Expanded Disability Status Scale
    FAS: full analysis set
    FLAIR: fluid-attenuated inversion recovery
    FSMC: Fatigue Scale for Motor and Cognitive Functions
    HCs: healthy controls
    MRI: magnetic resonance imaging
    MS: multiple sclerosis
    MSIS-29: Multiple Sclerosis Impact Scale–29
    MSST: Multiple Sclerosis Symptom Tracker
    PDDS: Patient Determined Disease Steps
    PHQ-9: Patient Health Questionnaire–9
    PPMS: primary progressive multiple sclerosis
    PRO: patient-reported outcome
    RMS: relapsing multiple sclerosis
    RRMS: relapsing-remitting multiple sclerosis
    SBT: Static Balance Test
    SDMT: Symbol Digit Modalities Test
    SPMS: secondary progressive multiple sclerosis
    T25FW: Timed 25-Foot Walk

    Edited by G Eysenbach; submitted 11.06.19; peer-reviewed by A Nguyen, L Visser, L Lavorgna, T Clavier; comments to author 29.06.19; revised version received 11.07.19; accepted 19.07.19; published 30.08.19

    Copyright

    ©Luciana Midaglia, Patricia Mulero, Xavier Montalban, Jennifer Graves, Stephen L Hauser, Laura Julian, Michael Baker, Jan Schadrack, Christian Gossens, Alf Scotland, Florian Lipsmeier, Johan van Beek, Corrado Bernasconi, Shibeshih Belachew, Michael Lindemann. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 30.08.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.