Passive Mobile Self-tracking of Mental Health by Veterans With Serious Mental Illness: Protocol for a User-Centered Design and Prospective Cohort Study

Background Serious mental illnesses (SMI) are common, disabling, and challenging to treat, requiring years of monitoring and treatment adjustments. Stress or reduced medication adherence can lead to rapid worsening of symptoms and behaviors. Illness exacerbations and relapses generally occur with little or no clinician awareness in real time, leaving limited opportunity to modify treatments. Previous research suggests that passive mobile sensing may be beneficial for individuals with SMI by helping them monitor mental health status and behaviors, and quickly detect worsening mental health for prompt assessment and intervention. However, there is too little research on its feasibility and acceptability and the extent to which passive data can predict changes in behaviors or symptoms. Objective The aim of this research is to study the feasibility, acceptability, and safety of passive mobile sensing for tracking behaviors and symptoms of patients in treatment for SMI, as well as developing analytics that use passive data to predict changes in behaviors and symptoms. Methods A mobile app monitors and transmits passive mobile sensor and phone utilization data, which is used to track activity, sociability, and sleep in patients with SMI. The study consists of a user-centered design phase and a mobile sensing phase. In the design phase, focus groups, interviews, and usability testing inform further app development. In the mobile sensing phase, passive mobile sensing occurs with participants engaging in weekly assessments for 9 months. Three- and nine-month interviews study the perceptions of passive mobile sensing and ease of app use. Clinician interviews before and after the mobile sensing phase study the usefulness and feasibility of app utilization in clinical care. Predictive analytic models are built, trained, and selected, and make use of machine learning methods. Models use sensor and phone utilization data to predict behavioral changes and symptoms. Results The study started in October 2020. It has received institutional review board approval. The user-centered design phase, consisting of focus groups, usability testing, and preintervention clinician interviews, was completed in June 2021. Recruitment and enrollment for the mobile sensing phase began in October 2021. Conclusions Findings may inform the development of passive sensing apps and self-tracking in patients with SMI, and integration into care to improve assessment, treatment, and patient outcomes. Trial Registration ClinicalTrials.gov NCT05023252; https://clinicaltrials.gov/ct2/show/NCT05023252 International Registered Report Identifier (IRRID) DERR1-10.2196/39010


HSR4
YOUNG, A
The study evaluates passive mobile sensing with Veterans in treatment for serious mental illness and allows self-tracking of behaviors and symptoms. Analytics are developed that use passive data to predict behaviors and symptoms. The investigators plan to: 1.Conduct user-centered design of passive mobile self-tracking to support Veterans' management of their mental health. 2.Study the feasibility, acceptability and safety of passive self-tracking of mental health that includes feedback of mental health status to the Veteran. 3.Use mobile sensor and phone utilization data to develop individualized estimates of sociability, activities, and sleep as measured by weekly interviews. 4.Study the predictive value of using data on sociability, activities, and sleep to identify exacerbations of psychiatric symptoms.
The study has substantial significance in that technology could improve the ability of patients, families, and providers to have earlier warning signs of relapse of serious mental illnesses that involve psychotic, manic, or suicidal episodes. Warning signs of relapse include changes in sleep patterns and social patterns (i.e. either social isolation or overactivity). The investigators propose that smartphone application for the passive collection of sensor data can be used to help identify risk for relapse. The advancement in science would be meaningful and large enough to warrant a study such as the one proposed.

Innovation and Impact.
The investigators have developed new technology for a smartphone application that may have important impact in self-management of illness. This project would yield results that could be expected to change current research and clinical practice paradigms. This technology has not been tested in a VA setting/population. This tool would be highly innovative and has potential for clinically meaningful impact. The technology could also be very useful for other research to track outcome to interventions.

Approach.
Approach as written in the application: Methodology: Activities can be assessed with data on movement, location, and habits. Sociability can be assessed with data on communication and public interactions. Sleep can be assessed using data on light, sound, movement, and phone use.
Investigators on this project developed "mWellness," a functional mobile app that monitors and transmits mobile sensor and utilization data. Focus groups and in-lab usability testing will inform further app and intervention development. Mixed methods research will study deployment over 9 months in 60 Veterans who passively self-track their behaviors and psychiatric symptoms. If this project meets intended goals, the VA will have a mobile analytics platform that continuously monitors the behaviors and symptoms of patients with serious mental illness.
Key summary points in the previous critique and key modifications: • More clarity needed regarding primary outcome of exacerbated symptoms Added the Brief Psychiatric Rating Scale and updated the analysis plan. • Heterogeneity of diagnosis is not sufficiently addressed.
• Excluding MDD with psychotic features; narrowing to schizophrenia and bipolar. HSR4

YOUNG, A
• Unclear how this app would be integrated into VA care.
Processes could be automated using the VA Office of Connected Care patient-generated data platform that currently houses video telehealth and other Veteran data to interface with clinicians. No longer needs to connect within VA firewall (i.e. Video Connect and MyHealtheVet). Included a support letter from VA OCC. VA OCC and VA Office of Mental Health and Suicide Prevention provided support letters to sustain the app and future implementation study would be proposed. • Missed opportunity to collect perspectives of VA providers. • Added individual interviews with clinicians and qualitative analyses to inform development.
Other changes to enhance the approach: • Increased sample size from 60 to 100 and extended recruitment period.

•
Increase timeline for initial aim from 4 to 6 months to allow interview with providers. • Added a process for communicating risk of clinical deterioration to patient's clinician.

Feasibility (including Sampling, Project Timeline and Staffing).
Increasing the sample size from 60 to 100 does not negatively impact feasibility. They will be recruiting from a very large patient pool in the Greater LA VA Healthcare System. The investigators have been thoughtful in modifying the timeline due to increased activities in Aim 1 and in recruitment goals.

Implementation.
The investigators have already thought through the logistics of taking the product from test phase to implementation phase and have garnered the support of VA OCC and VA OMHSP.

Investigator Qualifications.
The investigators are experienced, qualified and capable. They have a track record of success.

Multiple PI Leadership Plan.
Not applicable.

Facilities and Resources.
The facilities and resources are well established and robust. The investigators have support at the national level.

Adequacy of Response to Previous Feedback Regarding the Proposed Study.
The investigators have responded to all points and feedback from previous review.

Protection of Human Subjects from Research Risk.
The investigators have provided reassurance at multiple levels that privacy risks are minimal.

Inclusion of Women and Minorities in Research.
Women and minorities will be included. HSR4 YOUNG, A

Budget and Period of Support.
The study has been modified to a 3 year study (from 2 years) and costs are appropriately increased to $776,144.

Sharing Research Data (Data Management and Access Plan (DMAP)).
The investigators plan to make de-identified datasets available under appropriate approvals and data use agreements.
The technology is innovative, clinically meaningful, and needed to fill a gap. The research methods are well described and thorough. The investigators have been responsive to previous review critique.

Key Strengths.
1. The study is focused on finding clinically meaningful self-management tools that can help a manage a difficult-to-treat patient population.
2. Innovative technology and support from VA OCC to make it a reality.
4. Very strong investigative team and environment.
5. Letters of support from VA OCC and VA OMHSP.

Key Weaknesses.
1. The test of predictability of self-tracking symptoms given many confounding variables is always something of concern with innovative research; not a weakness necessarily -just a reality.

Significance.
Understanding how mobile technology can be used to assist individuals with Severe Mental Illness (SMI) as well as others with mental illness is a pressing question. While many individuals will opt out of anything requiring passive tracking, many are willing. This proposal to develop and evaluate the acceptability, feasibility, and validity of passive sensing for prevention of decompensation in SMI is conceptually thoughtful and interesting. The idea of making the passive sensing results visible to the Veteran is particularly interesting as seeing what is being analyzed may increase the comfort of the Veteran to participate as well as providing self-monitoring benefits. The VA is a large, complex, and politically sensitive system which all can lead to barriers to technological interventions. Thus, it is critical that health service technological interventions be developed to effectively co-exist in VA care. VA partners have provided letters supporting their investment. If they can, this will be extremely significant. HSR4 YOUNG, A

Innovation and Impact.
Previous SMI mobile phone interventions have required significant attention from the individual to function properly and many individuals with SMI have other concerns and did not interact sufficientlylimiting the benefit of these mobile apps. This study is innovative in focusing purely on the passive sensing and demanding nothing else from the Veteran with SMI while offering self-monitoring information from the passive sensors, should the Veteran care to view it. Given the operational support for integrating the app into VA care, the impact could be very large.

Approach.
This is a well-written grant by experienced researchers who clearly describe a reasonable set of methods to match their aims which build one upon the next. Starting with user-centered design and inlab testing will be critical. Their plan to parallel the passive sensing data with ongoing reports from the participants on standard measures will provide key information on the validity and utility of the passive sensing data compared to gold standard measures of mechanisms and outcomes.

Feasibility (including Sampling, Project Timeline and Staffing).
The project team provides sufficient detail with regard to the recruitment and staffing. The revised proposal adds sample size and time to the project that will increase feasibility.

Implementation.
The investigators have expanded their implementation plan through greater support from operational partners and more explicit next steps.

Multiple PI Leadership Plan.
Not applicable.

Facilities and Resources.
More than adequate.

Adequacy of Response to Previous Feedback Regarding the Proposed Study.
Not applicable.

Protection of Human Subjects from Research Risk.
Adequate.

Inclusion of Women and Minorities in Research. HSR4
YOUNG, A Adequate.

Budget and Period of Support.
Adequate.

Sharing Research Data (Data Management and Access Plan (DMAP)).
Adequate.

Overall Impression.
This is a promising project to develop a passive sensing app that could be alert to early signs of decompensation in Veterans with SMI. The research team has a thoughtful and systematic approach laid out that reflects appropriate treatment development methodology. The ability for Veterans to review their own data is innovative and potentially impactful.

Key Strengths.
1. Could play a useful role in predicting decompensation allowing for timely intervention.

Strong research team.
3. Strong treatment development methodology.

Key Weaknesses.
1. Presuming the VA systems will allow for future compatibility.

Significance.
The proposed research has very high public health significance. Ten percent of VHA's budget is allocated to the care of Veterans with SMI.
The proposed research targets two HSR&D research priorities (Mental Health and Healthcare Informatics).
The background of the application makes a solid justification for the use of passive monitoring to detect relapse, which is quite common among patients with SMI. A passive sensing approach is needed because interventions focused on active self-monitoring have proven to be ineffective.

Innovation and Impact.
The proposed use of passive monitoring is highly innovative. HSR4

YOUNG, A
The proposed use of AI to analyze the passive monitoring and weekly symptom ratings is highly innovative.
The use of user centered design methods to improve patient-centeredness is novel.
There are no overlapping ongoing studies funded by VA or other federal research agencies.

Approach.
User Centered Design methods are proposed, which should make the mWellness highly Veteran centric.
Use of the Technology Acceptance Model to guide Aim 2 is a strength.
The application provides a detailed description of the app, with special attention to the potential risks to Veterans (which are minimal). The evaluation will assess app functioning and acceptability (data collection/transmission) as well as Veterans' perceptions of the app.
The application provides considerable detail about how the passive data will be collected and analyzed, including how to manage sensors collecting data at different time frequencies, data reduction techniques, and missing data.
A minor weakness of the proposed methods is that there is no quantitative assessment of how the clinicians view the alerts. Will the clinicians react to the alerts (i.e., are they clinically actionable)? Will new appointments be scheduled for patients with alerts? These questions will be assessed qualitatively, which mitigates this weakness somewhat.
Another weakness is that the application lacked detail on establishing baseline behaviors and whether Veterans would be recruited in a "baseline" state or a "relapsed" state. It was not clear whether the AI could handle data from patients recruited when they were in a relapse state?
Finally, while the addition of the self-monitoring dashboard to the mWellness app is a strength, the evaluation plan did not really specify how the investigators would determine whether this additional feature contributes to improved outcomes. The perceived usefulness of dashboard will be assessed qualitatively/retrospectively, which mitigates this weakness somewhat.

Feasibility (including Sampling, Project Timeline and Staffing).
A high percentage of Veterans with SMI use smartphone functionality. A small pilot with UCLA students found no major technical problems with the mWellness app. The investigators have already obtained approval for a mobile phone sensing and predictive analytic research project from their VA IRB, Information Security Officer, and R&D service. There are more than a sufficient number of Veterans with SMI at the GLA VAMC to recruit the proposed sample.

Implementation.
The D&I plan is likely to be effect. The research is strongly supported by the Office of Connected Care.

Investigator Qualifications. HSR4
YOUNG, A Dr. Young is at the forefront of research in mHealth for Veterans with SMI and has been a highly productive investigator over the years. He is proposing to devote a substantial amount of FTE to the project, ensuring its successful completion.
There was obviously substantial involvement from Dr. Majid Sarrafzadeh (AI expert on the team) in the development of the application. Involving computer science experts from UCLA in the research project is a major strength of the application.
Not applicable.

Facilities and Resources.
Research resources at the GLA VAMC (HSR&D COIN, MIRECC, UCLA affiliation) are outstanding.

Adequacy of Response to Previous Feedback Regarding the Proposed Study.
The investigators have been highly responsive to the previous critiques. The application is much improved.

Protection of Human Subjects from Research Risk.
Adequate.

Inclusion of Women and Minorities in Research.
Adequate.

Budget and Period of Support.
Acceptable.

Sharing Research Data (Data Management and Access Plan (DMAP)).
Acceptable.
The proposed research addresses a disorder that consumers 10% of VHA funding and addresses two key HSR&D research priorities. This highly innovative research is being proposed by an outstanding investigative team that includes both VA and UCLA researchers. The main concern about the proposed research (fidelity) has been very well addressed. The approach had a few minor/negligible weaknesses.

Key Strengths.
1. The proposed research has high public health significance and addresses two HSR&D research priorities. HSR4 YOUNG, A 2. The passive monitoring approach is well justified.
3. The user centered design and AI methods are highly innovative.
4. The investigator team is exceedingly strong.

Key Weaknesses.
1. The approach has minor weaknesses with regard to evaluating the contributions of the dashboard, how the AI would handle sensor data from patients in relapse at baseline and determining how clinicians viewed alerts.