This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3093) e-Mental Health and Cyberpsychology (1387) Psychotic Disorders (156)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48634, first published .
The Effects of Remote Cognitive Training Combined With a Mobile App Intervention on Psychosis: Double-Blind Randomized Controlled Trial

The Effects of Remote Cognitive Training Combined With a Mobile App Intervention on Psychosis: Double-Blind Randomized Controlled Trial

The Effects of Remote Cognitive Training Combined With a Mobile App Intervention on Psychosis: Double-Blind Randomized Controlled Trial

Authors of this article:

Melissa Fisher1 Author Orcid Image ;   Kevin Etter1, 2, 3 Author Orcid Image ;   Aimee Murray1 Author Orcid Image ;   Neelu Ghiasi1 Author Orcid Image ;   Kristin LaCross2 Author Orcid Image ;   Ian Ramsay1 Author Orcid Image ;   Ariel Currie1 Author Orcid Image ;   Karrie Fitzpatrick1 Author Orcid Image ;   Bruno Biagianti4 Author Orcid Image ;   Danielle Schlosser2 Author Orcid Image ;   Rachel Loewy2 Author Orcid Image ;   Sophia Vinogradov1 Author Orcid Image
Article Authors Cited by (2) Tweetations (10) Metrics

    Original Paper

    1Department of Psychiatry & Behavioral Sciences, University of Minnesota, Minneapolis, MN, United States

    2Department of Psychiatry & Behavioral Sciences, University of California, San Francisco, San Francisco, CA, United States

    3Edgewood Center for Children and Families, San Francisco, CA, United States

    4Department of Psychology, University of Milan Bicocca, Milano, Italy

    *these authors contributed equally

    Corresponding Author:

    Melissa Fisher, PhD

    Department of Psychiatry & Behavioral Sciences

    University of Minnesota

    F299/2A West

    2450 Riverside Avenue

    Minneapolis, MN, 55454

    United States

    Phone: 1 612 273 9835

    Email: mafisher@umn.edu


    Background: Impairments in cognition and motivation are core features of psychosis and strong predictors of social and occupational functioning. Accumulating evidence indicates that cognitive deficits in psychosis can be improved by computer-based cognitive training programs; however, barriers include access and adherence to cognitive training exercises. Limited evidence-based methods have been established to enhance motivated behavior. In this study, we tested the effects of web-based targeted cognitive and social cognitive training (TCT) delivered in conjunction with an innovative digital smartphone app called Personalized Real-Time Intervention for Motivational Enhancement (PRIME). The PRIME app provides users with a motivational coach to set personalized goals and secure social networking for peer support.

    Objective: This study investigated whether deficits in cognition and motivation in people with a psychosis spectrum disorder (N=100) can be successfully addressed with 30 hours of TCT+PRIME as compared with 30 hours of a computer games control condition (CG) plus PRIME (CG+PRIME). Here, we describe our study procedures, the feasibility and acceptability of the intervention, and the results on all primary outcomes.

    Methods: In this double-blind randomized controlled trial, English-speaking participants completed all cognitive training, PRIME activities, and assessments remotely. Participants completed a diagnostic interview and remote cognitive, clinical, and self-report measures at baseline, posttraining, and at a 6-month follow-up.

    Results: This study included participants from 27 states across the United States and 8 countries worldwide. The study population was 58% (58/100) female, with a mean age of 33.77 (SD 10.70) years. On average, participants completed more than half of the cognitive training regimen (mean 18.58, SD 12.47 hours of training), and logged into the PRIME app 4.71 (SD 1.58) times per week. The attrition rate of 22% (22/100) was lower than that reported in our previous studies on remote cognitive training. The total sample showed significant gains in global cognition (P=.03) and attention (P<.001). The TCT+PRIME participants showed significantly greater gains in emotion recognition (P<.001) and global cognition at the trend level (P=.09), although this was not statistically significant, relative to the CG+PRIME participants. The total sample also showed significant improvements on multiple indices of motivation (P=.02-0.05), in depression (P=.04), in positive symptoms (P=.04), and in negative symptoms at a trend level (P=.09), although this was not statistically significant. Satisfaction with the PRIME app was rated at 7.74 (SD 2.05) on a scale of 1 to 10, with higher values indicating more satisfaction.

    Conclusions: These results demonstrate the feasibility and acceptability of remote cognitive training combined with the PRIME app and that this intervention can improve cognition, motivation, and symptoms in individuals with psychosis. TCT+PRIME appeared more effective in improving emotion recognition and global cognition than CG+PRIME. Future analyses will test the relationship between hours of cognitive training completed; PRIME use; and changes in cognition, motivation, symptoms, and functioning.

    Trial Registration: ClinicalTrials.gov NCT02782442; https://clinicaltrials.gov/study/NCT02782442

    J Med Internet Res 2023;25:e48634

    doi:10.2196/48634

    Keywords

    schizophrenia; psychosis; cognitive training; motivation; mobile intervention; mobile phone 


    Background

    Psychosis spectrum disorders are not only characterized by positive symptoms such as hallucinations and delusions but also by mood, motivational, and cognitive impairments [1]. These latter symptoms affect functioning and quality of life, contribute strongly to disability [2], and have now become primary therapeutic targets [3] because of their strong association with functional outcomes [4,5]. We have previously demonstrated both behavioral gains and improved neural system functioning after neuroscience-informed cognitive training in schizophrenia in both the chronic and early phases of psychotic disorders [6-13]. In young (aged 16-35 years) recent-onset individuals, our multisite double-blind randomized controlled trial showed that 40 hours of cognitive training delivered at home on a laptop resulted in significant improvement in global cognition, verbal memory, and problem-solving compared with a computer games control condition (CG) [8], with durable cognitive gains and improvement in positive symptoms at a 6 month follow-up [12]. Cognitive gains were significantly correlated with enhanced thalamic volume and thalamocortical connectivity as well as reduced cortical thinning in the frontal, temporal, parietal, and occipital lobes [14-16].

    Despite the evidence that cognitive training can lead to benefits in cognition and functioning [17,18], patient adherence to a cognitive training regimen is highly variable. Training is highly effortful, requiring close attention to stimuli and response outcomes, processes that are impaired in psychotic disorders. It is also likely that motivational deficits have an impact on engagement with training [19-21], although these can respond to interventions that scaffold goal planning and reward processing [22].

    Objectives

    Thus, we hypothesized that leveraging an intervention specifically targeted to improving motivated behavior would improve adherence and benefits from cognitive training in individuals with psychosis spectrum disorders. We examined the cognitive and clinical effects of supplementing a 16-week course of targeted cognitive and social cognitive training (TCT) with Personalized Real-Time Intervention for Motivational Enhancement (PRIME)—a smartphone-based app that includes a motivational coach and peer-to-peer interactions. To determine which changes in outcome measures would be attributable to the cognitive training, this intervention was compared with the cognitive and clinical effects of PRIME delivered with a CG.

    As a mobile app, PRIME is designed to enhance motivation (but not cognition) by helping participants set and track SMART (Specific, Measurable, Achievable, Realistic, Timely) goals and by celebrating accomplishments. PRIME pairs participants with a motivational coach trained in cognitive behavioral therapy (CBT) to encourage and reinforce the goals. PRIME participants message and interact with their coach as well as a community of peers. PRIME was developed at the University of California, San Francisco, in partnership with the design company IDEO, by Danielle Schlosser [23]. It uses a user-centered design, developed with a cohort of individuals living with schizophrenia and their families who provided insight into specific values that drive one to make improvements in their quality of life. The stakeholders also provided important feedback on each feature of the app in terms of acceptability and impact on motivated behavior. A single-arm feasibility study of PRIME showed 100% retention across the intervention, with users logging in several times a week, and favorable ratings of features and overall satisfaction [23]. In a larger randomized waitlist-controlled study of young people with schizophrenia, the PRIME group showed significant improvements in social motivation, depression, defeatist beliefs, and self-efficacy relative to the waitlist group [22].

    Here, we describe the procedures for this study, which was conducted entirely remotely across the United States and internationally before and during the COVID-19 pandemic; the features of the interventions; the feasibility and acceptability of the intervention in 100 randomized participants; and the results of this randomized controlled trial on cognition, motivation, symptoms, and functioning. Our primary hypotheses were as follows: (1) TCT+PRIME participants would show greater gains in cognition relative to CG+PRIME and (2) both groups would show improvements in motivation and related indices of motivation due to the effects of PRIME. Exploratory analyses tested the effects of these interventions on symptoms and functioning.


    Study Design and Timeline

    In this recently completed double-blind randomized controlled trial (ClinicalTrials.gov: NCT02782442), participants were randomized to PRIME+30 hours of TCT or PRIME+30 hours of CG in a 1:1 ratio. Before enrolling participants, a random allocation sequence was created using a randomization generator. The sequence was stored in Box and concealed or inaccessible to study members blinded to the group assignment (ie, PRIME coaches and assessment staff). The sequence was accessible to our unblinded study coordinator who enrolled and randomized the participants.

    All of the study activities were completed remotely. Both groups used the PRIME app on their personal smartphones and completed cognitive training exercises or the CG on laptops or computers. The participants were asked to complete the following schedule of study activities: baseline assessments (weeks 0-3), intervention (weeks 4-20), postintervention assessments (weeks 21-23), and a 6-month follow-up assessment (weeks 49-50). The total time for study completion was approximately 1 year. Diagnosis was confirmed at baseline using the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5), Research Version (SCID-5-RV) and conducted through videoconferencing. Assessments of symptoms, functioning, motivation, and cognition at all 3 time points were also completed entirely remotely via videoconference, self-report surveys, and web-based cognitive batteries (refer to the “Measures” section for details). In total, 85% (85/100) of the participants enrolled before the COVID-19 pandemic and 15% (15/100) enrolled during the pandemic. During the COVID-19 pandemic, there was a decrease in enrollment and greater attrition during baseline assessments (ie, before randomization). Our final sample consisted of 100 randomized participants, which was a decrease from our original goal of 120 randomized participants.

    Study Population

    The study sample comprised individuals aged 18 to 59 years with a psychosis spectrum disorder (refer to the inclusion criteria in the next paragraph). Participants were recruited through advertisements on Craigslist, Reddit, Schizophrenia.com [24], and the National Alliance on Mental Illness websites or were self-referred to our research group. Physical flyers and brochures were distributed in community programs and events and at the University of Minnesota clinics.

    All participants met the following inclusion criteria: aged 18 to 60 years; clinical diagnosis of schizophrenia, schizoaffective disorder, schizophreniform disorder, psychotic disorder not otherwise specified, major depressive disorder with psychotic features, or bipolar disorder with psychotic features confirmed by the SCID-5-RV; fluent in spoken and written English; can demonstrate adequate decisional capacity to make a choice about participating in the research study as demonstrated by the University of California San Diego Brief Assessment of Capacity to Consent and in the judgment of the consenting study staff member; in good general physical health; outpatient status without hospitalization at least 1 month before participation; has maintained a stable dose of psychiatric medication for at least 1 month before participation; has a personal smartphone and access to a computer; willing to share contact information for their clinical provider; and if from a non–English-speaking country, the study team must be able to establish a plan for communicating with the participant’s clinical provider. Exclusion criteria were participation in research or therapy involving cognitive training within the past 3 years, a history of severe substance use in the past 3 months determined by DSM-5 criteria, and a history of neurological disorder.

    TCT and CG

    Overview

    This study used 2 computerized programs provided by Posit Science Inc through their BrainHQ portal: a TCT module focused on auditory processing and social cognition and a CG module. The CG module consists of computer games developed by Novel Games that are engaging and enjoyable but not designed to drive neuroplastic change. Participants were asked to use their assigned program (treatment or active control) on a computer for approximately 2 hours per week over the course of 16 weeks.

    Targeted Cognitive and Social Cognitive Training
    Auditory Training Module

    This suite of exercises has been extensively studied by us and has been described in detail by Fisher et al [6]. It was designed to improve the speed and accuracy of auditory information processing while engaging working memory and cognitive control under conditions of close attention and reward. Exercises continuously adjust the difficulty level according to user performance to maintain an approximately 80% correct performance rate.

    Social Cognition Training Module

    This training module consists of exercises designed to ameliorate core deficits in social cognition expressed in schizophrenia [25]. The exercises apply principles of implicit learning to restore the brain’s capacity to process and use socially relevant information and include training to improve affect perception (both visual and vocal), social cue perception (in faces, gazes, social situations), theory of mind, self-referential style, and emotion labeling and working memory. This module has been previously studied by us, and it drives improvements in social cognition as well as measures of motivated behavior [13,25,26]. Refer to Multimedia Appendix 1 for a list of the auditory and social cognition training exercises.

    Description of CG

    CG participants rotated through a series of 13 different enjoyable commercially available computer games (eg, checkers, solitaire, crossword puzzles; Multimedia Appendix 2), playing 4 to 5 games on any given day. Games that have shown to provide face-valid cognitive stimulation and that are rated E (for everyone) by the Entertainment Software Rating Board were chosen. The CG condition was designed to control for computer exposure, contact with research personnel, and monetary payments. The CG condition is administered in exactly the same way as the TCT: the number, availability, and time spent on each game is managed by the same server that manages the treatment group exercises to match the experience between the 2 conditions.

    Description of PRIME

    PRIME is a mobile app designed to support individuals with psychosis to manage negative symptoms and engage in motivated behavior. PRIME coaches were graduate students in counseling, clinical psychology, or social work programs, and some coaches had completed a master’s or higher degree. All coaches had previous training in CBT, motivational interviewing, or Dialectical Behavior Therapy and participated in regular study team meetings supervised by a licensed clinical psychologist (coauthor AM). PRIME techniques include CBT, behavioral activation, mindfulness, psychoeducational approaches, and coaching behaviors associated with increases in overall engagement [23]. This model incorporates a casual, collaborative tone for conversations and texts; short individual messages; and sharing of resources, including short videos and psychoeducation materials, while emphasizing behavioral strategies for any asynchronous exchanges. Text exchanges rely on communication cues that differ from face-to-face interactions [27], often creating ambiguity in meaning and tone. In light of this, PRIME uses an overtly warm tone, including expressive punctuation and positive emojis, aligning with the growing literature on messaging, behavioral change, and preferences found in other clinical populations [28,29] (Figure 1).

    Users create a profile by describing interests and symptoms and selecting long-term goals across social, body (ie, physical health), productivity, and creativity domains. While using the app, users identify which of their goals they would like to work on and are prompted with brief challenges—SMART activities related to the selected goal (Figure 2). Challenges are framed as behavioral experiments; users log predictions about the effort and pleasure associated with an upcoming challenge to compare it with what was actually exerted and derived.

    PRIME also helps track changes in mood, symptoms, and activity levels by pairing visualizations of surveys with motion and fitness data piped from built-in iOS and Android apps (Figure 3). Users complete the Patient Health Questionnaire-9 weekly and answer a personalized question such as “How well did I sleep last night” or “How well did your social interactions go?” each day. These visualizations can serve as feedback to help determine which challenges or other behavioral strategies are having a positive impact.

    Reflecting tenets of the recovery model of psychosis [30,31], users can also interact with a community of peers with similar diagnoses, sharing direct messages as well as offering encouragement by “liking” and commenting on community posts. Each day, an automatically generated post such as “Share a silly selfie!” or “Write down a list of things you are grateful for” encourages participants to share responses. When a participant completes a challenge, an accomplishment post is automatically generated, creating additional opportunities for social reinforcement.

    ‎
    Figure 1. Example screenshots of the Personalized Real-Time Intervention for Motivational Enhancement (PRIME) app (from left to right): goal setting, community feed, coaching and social networking (messaging with coaches and peers), and personalized profile and metrics.
    ‎
    Figure 2. Examples of the Challenges feature (left to right): goal selection, reminders, feedback, and share with community.
    ‎
    Figure 3. Examples of the My Progress feature (left to right): mood survey and mood and physical activity visualizations.

    Measures of Feasibility and Acceptability

    The feasibility was investigated through attrition rates and use patterns on each platform. For TCT and CG, use was defined as the total hours of training completed and training intensity (hours of training completed per week). Acceptability was measured through participant ratings of satisfaction, interest, and enjoyment in PRIME and in the TCT and CG conditions and the anticipated benefits of the TCT and CG conditions. Accuracy in TCT exercises was used as an indicator of engagement and to screen for participants who were not meaningfully participating. For PRIME, use patterns include log-in frequency, number of user-initiated interactions with a motivational coach, number of peer interactions, number of community posts, number of goals set, and number of goals achieved.

    Cognition, Motivation, Symptom, and Functioning Measures

    Cognition was measured using the Penn Computerized Neurocognitive Battery, Measurement and Treatment Research to Improve Cognition in Schizophrenia version. The participants were provided a link to the battery and completed the measures independently. The following self-report measures of motivation and related constructs were completed on the web via REDCap (Research Electronic Data Capture; Vanderbilt University): Motivation and Pleasure Scale (MAPS) Self Report, Motivation State Questionnaire (MSQ), Defeatist Beliefs Scale, Temporal Experience of Pleasure Scale (TEPS), Behavioral Inhibition Scale and Behavioral Activation Scale, Beck Depression Inventory, and University of California Los Angeles Loneliness Scale. Symptoms and functioning were assessed using the following interview-rated measures conducted via videoconference: Quick Scale for the Assessment of Negative Symptoms (QSANS) and Quick Scale for the Assessment of Positive Symptoms (QSAPS), the abbreviated Quality of Life Scale (QLS), and the Role Functioning Scale (RFS). Across all measures, higher scores indicated more of the construct measured. For example, higher z scores on cognitive measures or higher scores on motivation measures indicated better cognitive performance or greater motivation, higher ratings on symptom measures indicated greater symptom severity, and higher ratings on functioning measures indicated better functioning.

    Assessors who completed the diagnostic interview and the interview-rated measures of symptoms and functioning had completed a master’s or doctoral degree in psychology or related fields and completed extensive training including observation of videos and practice ratings, observing assessments conducted by senior staff members, mock interviews, and assessment of participants while being observed by a senior staff member (coauthors MF or AC). Participants, PRIME coaches, and assessment staff were blinded to group assignment. The study coordinator was unblinded to enroll and randomize participants, perform weekly check-ins with participants, and provide support while the participants completed their assigned training program.

    Statistical Analyses

    Independent samples t tests (2-tailed) or chi-squared tests were used to compare groups in terms of demographic variables; attrition rates; cognitive training metrics (ie, total hours of training completed and training intensity defined as hours of training per week); PRIME use (eg, number of log-ins, coach and peer interactions, and goals achieved); and participant ratings of satisfaction, interest, and enjoyment in the cognitive training and PRIME. An intent-to-treat analysis was conducted using a linear mixed-effects model, with group and time as fixed factors. The model parameters were estimated using restricted maximum likelihood. Participant groups were compared in terms of changes in the Penn cognition z scores and measures of motivation, symptoms, and functioning. Effect sizes (Cohen d) were calculated using the mean changes from baseline to posttraining and baseline to 6-month follow-up and the pooled SDs. In measures where the main effects of time or group-by-time interactions were statistically significant, post hoc contrasts were conducted from baseline to posttraining and from baseline to 6-month follow-up. All variables were screened, and outlying values less than −2.5 SD and greater than +2.5 SD from the mean were winsorized. The number of TCT participants with an accuracy threshold of ≥50% was used as an indicator of engagement and to screen for participant guessing on the cognitive training exercises. All analyses were performed using SPSS Statistics version 28 (IBM Corp).

    Ethical Considerations

    The institutional review board of the University of Minnesota reviewed and approved this study (1607S90202). All participants provided informed consent. Data from this study have been deidentified. Participants in this study received US $100 for completing the diagnostic interview and baseline assessments, US $5 per session of cognitive training or computer games completed, US $55 for postintervention assessments, and US $55 for 6-month follow-up assessments.


    Participant Demographics and Locations

    There were no significant differences in the demographic or clinical characteristics between the TCT+PRIME and CG+PRIME groups (Table 1). The age range of the participants was 18 to 60 years, with a slight majority of participants identifying as female (58/100, 58%). The most common diagnoses were schizoaffective disorder (42/100, 42%) and schizophrenia (38/100, 38%). The average onset of symptoms was 19.76 (SD 8.02) years, and the average number of hospitalizations was 4.30 (SD 5.53).

    Participants resided in 27 states nationally and 7 countries internationally (Figure 4).

    Table 1. Demographic and clinical characteristics (N=100).

    		TCTa + PRIMEb (n=48)CGc + PRIME (n=52)t or χ2(P value)
    Age (years)d, mean (SD)33.98 (10.60)33.58 (10.89)0.19 (.85)
    Femalee, % (n/N)56.3 (27/48)59.6 (31/52)0.12 (.73)
    Education (years)d, mean (SD)15.47 (3.00)15.65 (2.52)−0.33 (.74)
    Racial background, n (%)e3.87 (.57)

    		American Indian or Alaska native0 (0)1 (2)

    		Asian5 (10)7 (13)

    		Black or African American8 (17)6 (12)

    		Native Hawaiian or other Pacific Islander2 (4)0 (0)

    		White31 (65)34 (65)

    		>1 race2 (4)3 (6)

    		Declined to answer0 (0)1 (2)
    Diagnosise, n (%)1.98 (.85)

    		Schizophrenia19 (40)19 (37)

    		Schizoaffective21 (44)21 (40)

    		Schizophreniform1 (2)1 (2)

    		Psychosis (not otherwise specified)1 (2)2 (4)

    		Bipolar3 (6)7 (13)

    		Major depressive disorder with psychotic features3 (6)2 (4)
    Age of first symptomsd, mean (SD)19.65 (8.77)19.87 (7.35)0.14 (.89)
    Total hospitalizationsd, mean (SD)4.23 (6.63)4.37 (4.34)−0.12 (.90)

    aTCT: targeted cognitive and social cognitive training.

    bPRIME: Personalized Real-Time Intervention for Motivational Enhancement.

    cCG: computer games control condition.

    dt test for equality of means.

    ePearson chi-square.

    ‎
    Figure 4. (A) Locations of participants by state and (B) locations of participants outside the United States. Numbers indicate how many participants reside in each area. Image created with Tableau [32].

    Participant Retention

    As a measure of acceptability, retention was computed as the percentage of participants who engaged in TCT or CG plus PRIME and who completed baseline and posttraining assessments. The total attrition rate was 22% (22/100). Of the 48 participants in the TCT+PRIME condition, 35 (73%) completed the posttraining assessments, and 13 (27%) did not complete them; of the 52 participants in the in the CG+PRIME condition, 43 (83%) completed the posttraining assessments, and 9 (17%) did not complete them (Figure 5). Most participants who did not complete the posttraining assessment were lost to follow-up (ie, could not be reached after multiple attempts). The between-group difference was not significant (N=100; χ21=1.4; P=.24).

    ‎
    Figure 5. CONSORT (Consolidated Standards of Reporting Trials) diagram of study participants. CG: computer games control condition; TCT: targeted cognitive and social cognitive training.

    TCT and CG Use Metrics

    Participants in the TCT+PRIME group averaged 17.15 (SD 13.89) hours of cognitive training across the study, with an intensity of 1.33 (SD 0.98) hours per week. Participants in the CG+PRIME group completed an average of 19.91 (SD 10.98) hours of computer games, with an intensity of 1.86 (SD 1.52) hours per week. The difference between the groups was not statistically significant for total hours of training (t98=1.10; P=.28) but was significant for intensity (t98=2.05; P=.04). The average accuracy in TCT exercises was 69.6% (SD 7.04%), with only 1 participant scoring <50% accuracy.

    PRIME Use Metrics

    Across both conditions, participants logged into PRIME an average of 63.98 (SD 34.10) times, on average 4 to 5 times per week. The log-in rate significantly decreased from the first half of participation (up to week 8) to the second half (t96=4.56; P<.001), although the average change was less than 1 fewer log-ins per week (mean change 0.83, SD 1.79). In total, participants shared 7612 interactions (messages, comments, and “likes”) with their coach and 887 interactions between peers. CG+PRIME participants initiated more coach interactions relative to TCT+PRIME, and this difference was at a trend level (P=.06), although it was not statistically significant. The average number of community posts per participant was 21.84 (SD 25.59). The average number of goals set per participant was 14.98 (SD 17.03), and the average number of goals achieved was 14.16 (SD 14.66) or 95% (14.16/14.98). The most popular goals were related to physical health and well-being (body), with participants setting an average of 6.93 (SD 7.51) body goals. The number of goals set related to social activities, productivity, and creativity averaged 2 to 3 per category. There were no statistically significant differences in any PRIME use metrics between the TCT+PRIME and CG+PRIME groups (Table 2).

    Table 2. Personalized Real-Time Intervention for Motivational Enhancement (PRIME) use metrics (N=96)a.
    Outcome measuresTCTb + PRIME (n=45), mean (SD)CGc + PRIME (n=51), mean (SD)t (P value)
    Number of log-ins64.44 (30.66)63.57 (37.18)0.13 (.90)
    Total weeks of use13.78 (4.80)13.67 (6.00)0.10 (.92)
    Proportion of log-ins0.67 (0.22)0.67 (0.24)0.10 (.92)
    Log-ins per week4.69 (1.54)4.73 (1.62)−0.13 (.90)
    Log-ins per week (first half)5.27 (1.64)5.06 (1.97)0.59 (.56)
    Log-ins per week (second half)4.35 (2.24)4.31 (2.24)0.09 (.93)
    Number of interactions with coach61.13 (59.39)95.22 (107.03)−1.89 (.06)
    Number of interactions with peers8.47 (13.48)9.49 (18.75)−0.30 (.76)
    Community posts20.44 (24.44)23.08 (26.75)−0.50 (.62)
    Spontaneous posts3.24 (6.54)5.53 (11.29)−1.19 (.24)
    Achievement posts16.02 (19.67)16.78 (18.44)−0.20 (.85)
    Goals set13.78 (16.95)16.39 (17.15)−0.86 (.39)

    		Social2.02 (3.26)2.75 (3.58)−1.03 (.31)

    		Body6.84 (8.10)7.00 (7.02)−0.10 (.92)

    		Productivity2.09 (3.86)2.94 (4.08)−1.05 (.30)

    		Creativity2.42 (4.15)3.71 (5.18)−1.33 (.19)
    Goals achieved11.87 (12.36)16.18 (16.27)−1.45 (.15)

    aDue to technical difficulties, PRIME use metrics were unavailable for 4 participants.

    bTCT: targeted cognitive and social cognitive training.

    cCG: computer games control condition.

    The Effects on Cognition

    Statistically significant main effects of time were observed in global cognition, attention, and emotion recognition, with both groups showing improvements (refer to Table 3 and Multimedia Appendix 3 for P and Cohen d values from baseline to posttraining and baseline to follow-up).

    There was a statistically significant group-by-time interaction in emotion recognition and a non–statistically significant trend in global cognition, with the TCT+PRIME group showing greater improvements relative to the CG+PRIME group (Table 3). In emotion recognition, within-group comparisons showed significant gains in the TCT+PRIME group from baseline to posttraining (P=.05; Cohen d=0.37) and from baseline to follow-up (P=.007; Cohen d=0.56) and significant gains in the CG+PRIME group from baseline to posttraining (P=.001; Cohen d=0.55) but no significant change from baseline to follow-up (P=.28; Cohen d=0.19). In global cognition, within-group contrasts showed significant gains in the TCT+PRIME group from baseline to posttraining (P=.002; Cohen d=0.61) and nonsignificant improvement from baseline to follow-up (P=.15; Cohen d=0.28), and nonsignificant change in the CG+PRIME group from baseline to posttraining (P=.54; Cohen d=0.10) and from baseline to follow-up (P=.66; Cohen d=0.08).

    Table 3. z scores on cognitive and social cognitive outcome measures at baseline, posttraining, and at 6-month follow-up in the targeted cognitive and social cognitive training (TCT) + Personalized Real-Time Intervention for Motivational Enhancement (PRIME) and computer games control condition (CG) + PRIME groups.
    Outcome measuresaTCT + PRIME (n=48)CG + PRIME (n=52)Main effects of time, P valueGroup × time interaction, P value

    		Baseline, mean (SE)Posttraining, mean (SE)6-month follow-up, mean (SE)Baseline mean (SE)Posttraining, mean (SE)6 month follow-up, mean (SE)

    Global cognition0.04 (0.09)0.33 (0.10)0.24 (0.12)−0.03 (0.09)0.004 (0.09)0.01 (0.11).03.09
    Motor speed−0.02 (0.17)0.28 (0.20)0.34 (0.18)0.27 (0.16)0.17 (0.18)0.37 (0.16).17.38
    Speed of processing0.41 (0.09)0.50 (0.09)0.47 (0.08)0.34 (0.08)0.34 (0.08)0.47 (0.07).33.53
    Attention−0.07 (0.16)0.24 (0.14)0.35 (0.08)−0.20 (0.15)0.04 (0.12)0.27 (0.07)<.001.72
    Working memory−0.40 (0.18)0.04 (0.25)−0.33 (0.30)−0.25 (0.17)−0.50 (0.22)−0.34 (0.27).83.29
    Verbal learning−0.22 (0.18)−0.04 (0.21)−0.46 (0.30)−0.26 (0.17)−0.45 (0.19)−0.63 (0.27).14.44
    Verbal memory−0.40 (0.17)−0.21 (0.23)−0.50 (0.28)−0.41 (0.16)−0.59 (0.20)−0.50 (0.26).84.40
    Visual learning−0.32 (0.17)−0.01 (0.20)−0.27 (0.22)−0.31 (0.16)−0.30 (0.18)−0.39 (0.20).37.68
    Visual memory−0.37 (0.16)−0.32 (0.21)−0.10 (0.21)−0.44 (0.16)−0.29 (0.19)−0.33 (0.20).33.66
    Problem-solving0.29 (0.16)0.53 (0.18)0.36 (0.19)0.39 (0.15)0.35 (0.16)0.08 (0.17).10.37
    Emotion recognition0.79 (0.19)1.13 (0.16)1.19 (0.20)−0.11 (0.18)0.39 (0.14)0.14 (0.18).005<.001

    aPenn Computerized Neurocognitive Battery: global cognition (composite z score across all measures), motor speed (Motor Praxis Test), speed of processing (Digit Symbol Test), attention (Short Continuous Performance Test), working memory (Short Letter N-Back Test), verbal learning (Word Memory Test), verbal memory (Word Memory Delayed Test), visual learning (Short Visual Object Learning Test), visual memory (Short Visual Object Learning Delayed Test), problem-solving (Matrix Analysis Test), and Emotion Recognition test.

    The Effects on Motivation Indices

    In the total sample, statistically significant main effects of time indicated improvements in MAPS social pleasure, MAPS work and recreation, MAPS motivation to engage in activities, MSQ overall motivation, MSQ self-efficacy, defeatist beliefs, and TEPS consummatory pleasure (Table 4; Multimedia Appendix 3).

    There was a statistically significant group-by-time interaction in MSQ Extrinsic Motivation. Within-group comparisons showed nonsignificant change in the TCT+PRIME group from baseline to posttraining (P=.48; Cohen d=0.13) and from baseline to follow-up (P=.87; Cohen d=0.03). The CG+PRIME group showed nonsignificant change from baseline to posttraining (P=.18; Cohen d=0.23) and a significant decrease from baseline to follow-up (P=.04; Cohen d=0.37). All other group-by-time interactions were not significant.

    Table 4. Scores on motivation indices, symptom ratings, and functional outcomes at baseline, posttraining, and at 6-month follow-up in the targeted cognitive and social cognitive training (TCT) + Personalized Real-Time Intervention for Motivational Enhancement (PRIME) and computer games control condition (CG) + PRIME groups.
    Outcome measuresTCT + PRIME (n=48)CG + PRIME (n=52)Main effects of time, P valueGroup × time interaction, P value

    		Baseline, mean (SE)Posttraining, mean (SE)6-month follow-up, mean (SE)Baseline, mean (SE)Posttraining, mean (SE)6-month follow-up, mean (SE)

    Motivation indices

    		MAPSa social pleasure1.94 (0.15)2.25 (0.17)2.18 (0.18)1.99 (0.15)2.26 (0.16)2.01 (0.16).02.82

    		MAPS work and recreation2.17 (0.16)2.58 (0.17)2.42 (0.16)2.03 (0.15)2.35 (0.15)2.28 (0.14).02.79

    		MAPS motivation about close relationships2.12 (0.15)2.34 (0.17)2.26 (0.20)2.11 (0.14)2.22 (0.15)2.17 (0.17).33.94

    		MAPS motivation to engage in activities1.90 (0.14)2.15 (0.16)2.07 (0.16)1.78 (0.13)2.09 (0.14)2.07 (0.14).04.91

    		MSQb overall motivation2.85 (0.15)3.14 (0.16)3.29 (0.18)3.08 (0.14)3.24 (0.14)3.23 (0.16).04.53

    		MSQ intrinsic motivation2.98 (0.14)3.18 (0.16)3.07 (0.18)3.15 (0.14)3.20 (0.15)3.25 (0.16).47.74

    		MSQ extrinsic motivation-pressure1.97 (0.15)2.13 (0.18)2.01 (0.17)2.51 (0.14)2.27 (0.16)1.98 (0.15).13.04

    		MSQ self-efficacy2.83 (0.16)2.87 (0.19)3.13 (0.20)2.86 (0.15)2.98 (0.17)3.17 (0.18).03.98

    		Defeatist beliefs3.28 (0.17)3.09 (0.18)3.18 (0.18)3.34 (0.16)3.10 (0.17)3.03 (0.17).05.74

    		TEPSc anticipatory pleasures4.15 (0.15)4.18 (0.18)4.25 (0.18)4.34 (0.15)4.38 (0.17)4.31 (0.17).92.61

    		TEPS consummatory pleasure4.25 (0.15)4.45 (0.18)4.34 (0.20)4.37 (0.14)4.65 (0.16)4.36 (0.18).02.64

    		Behavioral Inhibition Scale1.87 (0.08)1.86 (0.10)1.99 (0.11)1.72 (0.08)1.85 (0.09)1.78 (0.10).32.19

    		BASd reward responsivity1.60 (0.07)1.56 (0.09)1.66 (0.11)1.67 (0.07)1.65 (0.08)1.69 (0.10).56.85

    		BAS drive2.16 (0.10)2.17 (0.12)2.12 (0.14)2.23 (0.09)2.03 (0.11)2.16 (0.12).37.33

    		BAS fun seeking2.05 (0.09)2.13 (0.12)2.15 (0.13)2.02 (0.08)1.98 (0.11)1.92 (0.11).94.52
    Symptom ratings

    		QSANSe32.49 (2.87)26.31 (3.17)25.75 (3.20)31.98 (2.76)30.39 (2.95)31.75 (2.97).09.31

    		QSAPSf16.58 (1.86)17.92 (2.24)16.62 (2.02)18.05 (1.79)18.67 (2.08)14.36 (1.85).04.48

    		Beck Depression Inventory20.68 (2.19)19.47 (2.56)18.61 (2.42)24.02 (2.08)18.01 (2.29)18.63 (2.15).05.42

    		UCLAg Loneliness Scale2.28 (0.11)2.29 (0.12)2.36 (0.14)2.26 (0.10)2.40 (0.11)2.30 (0.12).46.46
    Functional outcomes

    		QLSh social functioning2.91 (0.26)3.13 (0.30)3.24 (0.27)2.84 (0.25)3.12 (0.28)3.06 (0.25).14.95

    		QLS occupational functioning3.77 (0.31)3.43 (0.33)3.45 (0.36)3.25 (0.30)3.57 (0.31)3.54 (0.34).99.13

    		QLS intrapsychic foundations3.90 (0.14)4.16 (0.16)4.12 (0.16)3.94 (0.13)3.99 (0.15)3.83 (0.15).26.43

    		QLS environmental engagement5.58 (0.08)5.75 (0.10)5.75 (0.11)5.58 (0.08)5.42 (0.09)5.45 (0.10).97.04

    		QLS mean item total3.85 (0.14)4.04 (0.16)4.05 (0.15)3.80 (0.13)3.90 (0.15)3.81 (0.14).22.59

    		RFSi work productivity4.67 (0.24)4.39 (0.25)4.61 (0.25)4.62 (0.23)4.79 (0.24)4.68 (0.23).89.25

    		RFS independent living5.25 (0.19)5.23 (0.20)5.28 (0.20)5.25 (0.18)5.28 (0.19)5.38 (0.19).8.98

    		RFS family network5.73 (0.17)5.53 (0.20)5.58 (0.21)5.96 (0.16)5.72 (0.18)5.54 (0.20).04.47

    		RFS social network4.56 (0.26)5.02 (0.28)4.87 (0.27)4.92 (0.25)4.87 (0.26)4.93 (0.25).46.53

    aMAPS: Motivation and Pleasure Scale.

    bMSQ: Motivation State Questionnaire.

    cTEPS: Temporal Experience of Pleasure Scale.

    dBAS: Behavioral Activation Scale.

    eQSANS: Quick Scale for the Assessment of Negative Symptoms.

    fQSAPS: Quick Scale for the Assessment of Positive Symptoms.

    gUCLA: University of California Los Angeles.

    hQLS: Quality of Life Scale.

    iRFS: Role Functioning Scale.

    The Effects on Symptoms and Functioning

    In the total sample, statistically significant main effects of time indicated improvements in the Beck Depression Inventory, QSAPS, and improvement at the trend level (although this was not statistically significant) in the QSANS (Table 4; Multimedia Appendix 3). In the RFS Family Network, a significant main effect of time indicated a decline in the total sample.

    There was a significant group-by-time interaction in the QLS Environmental Engagement subscale. Within-group comparisons showed significant improvement in the TCT+PRIME group from baseline to posttraining (P=.02; Cohen d=0.43) and from baseline to follow-up (P=.03; Cohen d=0.42). The CG+PRIME group showed a significant decrease from baseline to posttraining (P=.05; Cohen d=0.31) and a decrease at the trend level, although this was not statistically significant, from baseline to follow-up (P=.09; Cohen d=0.30). All other group-by-time interactions were not significant.

    Satisfaction, Interest, and Enjoyment With TCT, CG, and PRIME

    In total, 72% (72/100) of the participants evaluated the TCT and CG modules and the features of PRIME during an exit interview (Table 5). When asked about attitudes and experiences regarding cognitive training or computer games on a Likert scale ranging from 0 (none of the time) to 4 (all the time), participants rated interest in cognitive training and computer games similarly (difference in means=0.02; P=.94). Participants in the TCT group rated the exercises as more difficult than those in the CG group (difference in means=0.72; P=.004); however, the TCT group tended to find the instructions for cognitive training easier to understand compared with the CG group (difference in means=0.39; P=.07). In terms of anticipated impact of the exercises (“The training will help me do well at school or work”), participants in the TCT group rated cognitive training as slightly more meaningful relative to the CG group, although this difference was not statistically significant (difference in means=0.46; P=.15).

    Regarding overall satisfaction with PRIME, the TCT and CG groups responded similarly, with a mean rating of 7.74 (SD 2.05) on a scale of 1 (completely disagree) to 10 (completely agree). Among features, participants rated satisfaction with daily discussion topics and the relatability of coach community posts slightly more highly compared with helpfulness of the mood tracking feature. There were no significant differences between the groups.

    Table 5. Participants’ ratings of targeted cognitive and social cognitive training (TCT), computer games control condition (CG), and Personalized Real-Time Intervention for Motivational Enhancement (PRIME).
    Outcome measuresTCT + PRIME (n=33), mean (SD)CG + PRIME (n=39), mean (SD)t (P value)
    The traininga

    		...is interesting2.03 (1.05)2.05 (1.34)0.07 (.94)

    		...is enjoyable1.88 (1.05)2.21 (1.36)1.15 (.26)

    		...is easy1.97 (1.08)2.69 (1.00)2.95 (.004)

    		...is effective2.06 (1.27)1.97 (1.29)−0.28 (.78)

    		...is having a positive effect on daily life2.22 (1.50)1.82 (1.32)−1.19 (.24)

    		...will improve my thinking and memory2.38 (1.29)2.05 (1.34)−1.03 (.31)

    		...will help me do well at school or work2.25 (1.24)1.79 (1.38)−1.44 (.15)

    		The training sessions felt helpful to me2.12 (1.34)1.87 (1.32)−0.79 (.43)

    		The training instructions are easy to understand3.52 (0.76)3.13 (0.98)−1.85 (.07)

    		I felt I was making progress2.30 (1.05)2.08 (1.28)−0.80 (.43)
    PRIMEb

    		I felt satisfied with the daily discussion topics7.00 (2.70)7.16 (2.50)0.26 (.80)

    		Moments posted by the coaches felt relatable to me7.30 (2.54)7.44 (2.23)0.24 (.81)

    		Tracking my mood in PRIME was helpful7.00 (2.91)6.82 (3.16)−0.25 (.81)

    		Overall satisfaction with PRIME7.73 (2.04)7.74 (2.09)0.03 (.97)

    aEach statement was rated on a scale ranging from 0 to 4 (0=none of the time, 1=a little bit of the time, 2=about half the time, 3=most of the time, and 4=all of the time).

    bEach statement was rated on a scale ranging from 1 (completely disagree) to 10 (completely agree).


    This is the first randomized controlled trial to explore how cognitive training, supported by a novel smartphone-based app designed to improve motivational deficits, can be delivered remotely to individuals with psychosis spectrum disorders. All procedures were conducted entirely online, enabling participation from individuals both nationally and internationally.

    Participants

    The participants in this study were on average aged 34 (SD 10.70) years, 35% (35/100) were people of color, 58% (58/100) identified as women, and 46% (46/100) had an average of several years of postsecondary education (mean of 15.57 years of education in the total sample, SD 2.75). This is in contrast to our prior studies of TCT in adults with persistent schizophrenia spectrum disorders where participants were recruited from community outpatient mental health clinics, where the average age was 43 (SD 12.85) years, 50% were people of color, 28% identified as women, and the average number of years of education was 13.5 (SD 2.24) [13,33]. These results indicate that web-based recruitment results in a sample that is younger, less racially diverse, with a larger number of women, with greater number of years of education, and with average cognitive performance relative to studies using traditional brick-and-mortar recruitment methods from community clinics.

    This is consistent with the study by Moseson et al [34] who compared 3 studies on cognitive health, diabetes, and hypertension that used web-based recruitment methods with 19 studies in the same fields that used traditional recruitment methods (eg, in-person screening and consent in clinics). The web-based recruitment studies had slightly younger participants, more female participants, and were split on enrollment of racial minorities, with some of the web-based recruitment studies showing less diversity relative to the traditional recruitment studies and others showing more diversity. Web-based recruitment also resulted in a more geographically diverse sample and faster recruitment. Education level was not compared in this study.

    These and our results suggest that although there are some benefits to web-based recruitment (eg, an increase in recruitment of women, a more geographically diverse sample, and increased efficiency), there are also limitations. Web-based recruitment led to a less racially diverse sample in our study and in some of the studies by Moseson et al [34]. Moseson et al [34] suggested that investigators should use this knowledge to carefully consider strategies for increasing outreach and recruitment of individuals from underrepresented populations. Participants in our study also showed greater years of education and were required to have a personal smartphone and access to a computer. Although we did not compare socioeconomic status, it is likely that web-based recruitment may have led to a sample with higher socioeconomic status that had greater access to digital devices. Future studies should consider loaning devices to increase the recruitment of diverse individuals.

    Intervention Adherence

    The dropout rate of this study was 22% (22/100), which is lower than that in our previous studies of TCT in adults with persistent schizophrenia and with recent-onset psychotic disorders where training was completed remotely (where it was 30% [12]). Previous studies on TCT social cognition exercises in adults with psychosis spectrum disorders that also included group therapy and peer-to-peer messaging showed similarly low dropout rates (22% [35] and 20% [26]), whereas studies involving PRIME alone have shown dropout rates <10% [22,23]. These results suggest that the use of socially supportive and interactive therapeutic interventions such as PRIME may increase adherence to cognitively demanding and effortful treatments such as TCT.

    Although retention in both the TCT and CG groups was higher in this study than in similar studies without PRIME, the TCT participants showed a lower weekly training intensity, likely reflecting the greater effort and difficulty of engaging in TCT exercises. The completion of the training remotely may have also contributed to the lower training intensity. Our prior study of TCT completed in person in the research setting showed an average training intensity of approximately 5 hours per week [33], whereas prior studies of TCT cognitive or social cognitive exercises completed remotely show training intensities ranging from <1 hour per week [26] to approximately 3 hours per week [12,13.

    To retain blinding, the coaches were instructed to redirect discussions that might reveal specific details of the specific computer module a participant was engaged in (TCT or CG). Although critical to the design of a randomized controlled trial, this practice created obstacles to meaningful conversations about the effortful nature of cognitive training and inhibited motivational troubleshooting regarding the training itself. In a separate study of first-episode psychosis, we are currently investigating the effects of cognitive training provided with PRIME versus treatment as usual using a regression discontinuity design. In this study, discussion of the training between participants, coaches, and peers is encouraged and embedded in the treatment planning for the individual [36].

    PRIME Use

    Most participants used PRIME multiple times per week with only a slight drop as the study progressed, a finding that was consistent between the TCT and CG groups. Although direct messaging with a coach was by far the most used feature of PRIME, users frequently expressed value and inspiration in peer interactions and peer community posts. Body and creativity goals (averaging 6.86 and 3.07 goals set, respectively) were the most popular categories selected by participants, whereas social goals (averaging 2.40 goals set) were the least popular category. This is noteworthy considering how impairments in social functioning are often referenced as primary treatment goals of those living with psychosis spectrum disorders [37], where defeatist beliefs and negative appraisals often exacerbate actual performance difficulties and social aversion [38-40]. Future research can examine ways to harness popular PRIME body goals such as “go on a walk” to create low-risk social interactions.

    TCT+PRIME Show Durable Gains in Emotion Recognition

    Our results indicate significantly greater improvement in emotion recognition, and greater improvement at the trend level, although this was not statistically significant, in global cognition in the TCT+PRIME group relative to the CG+PRIME group. In emotion recognition, both groups showed improvement from baseline to posttraining; however, these effects were only durable at the follow-up in the TCT+PRIME group, with a medium effect size. In global cognition, the total sample showed a significant gain; however, within-group contrasts revealed that only the TCT+PRIME group showed improvement from baseline to posttraining, with an effect size in the medium range, whereas the CG+PRIME group showed negligible change. At follow-up, the gains in the TCT+PRIME group showed some durability with an effect size in the small range; however, these results did not reach statistical significance. Both groups also showed durable gains in attention, likely owing to the attentional demands of the TCT and CG exercises.

    The TCT+PRIME group results are consistent with those of our prior study of cognitive training combined with social cognition training versus cognitive training alone (TCT only) in 111 participants with schizophrenia spectrum disorders [13]. Both groups showed improvements in cognition and symptoms; however, the cognitive training combined with social cognition training group showed greater improvement in social cognition and reward processing relative to the TCT-only group. Effect sizes on social cognitive measures were larger in this prior study and likely owing to the greater number of hours of training completed (70 hours recommended vs 30 hours recommended in this study). Although the effects were not as large in this study, our findings suggest that durable gains in emotion recognition can be achieved with substantially fewer hours of training.

    Our prior studies on cognitive training versus a CG in chronic schizophrenia and recent-onset schizophrenia have also shown larger effects at posttraining and follow-up [7,12]. Again, this could be because of the greater number of hours of training completed in these studies (ie, 40-50 recommended hours). However, this is also likely owing to differences in our samples. The sample in this study did not show cognitive deficits at baseline and had an average of 15 years of education, whereas in our prior studies, participants showed baseline global cognition approximately 1 SD below the mean of healthy controls and an average of 13 years of education. This is consistent with a meta-analysis [17] that found that individuals with schizophrenia with lower premorbid IQ, fewer years of education, and greater baseline symptom severity showed greater gains from cognitive training, likely because there is more room for improvement.

    Finally, the differences in these effects across studies could also be the result of differences in the measures. In our prior studies, the MATRICS Consensus Cognitive Battery was administered in person [12,13,33]. In this study, a computerized battery of measures analogous to the MATRICS Consensus Cognitive Battery was used and completed remotely.

    PRIME Provides Durable Improvements in Motivation and Related Constructs

    The total sample showed improvements in a majority of the motivational domains, and these improvements were sustained at the 6-month follow-up. These included changes in experiential pleasure (MAPS social pleasure, MAPS work and recreation, and TEPS consummatory pleasure), MAPS motivation to engage in activities, and ratings of overall motivation; improvement in feelings of self-efficacy; and a reduction in defeatist beliefs. Notably, these benefits also appear to be reflected in the negative symptom measures (QSANS, Beck Depression Inventory; refer to the “PRIME Induces Improvement in Symptoms” section). These results are consistent with a prior study of PRIME alone (ie, no cognitive training) in 38 young people with schizophrenia [22]. Participants in the PRIME condition showed significantly greater improvements in self-efficacy, defeatist beliefs, and depression, and improvements at the trend level, although these were not statistically significant, on the MAPS relative to a waitlist control condition.

    There was a significant group-by-time interaction in the MSQ Extrinsic Motivation subscale, with nonsignificant change in the TCT+PRIME group and a significant decrease in the CG+PRIME group from baseline to follow-up. This subscale measures feelings of pressure from friends or family to make improvements in one’s social life, academic or work life, and general health. The CG+PRIME group ratings indicated feeling less external pressure at the 6-month follow-up, whereas the TCT+PRIME group showed no significant change. It is possible that feelings of external pressure may not have changed in the TCT+PRIME group because of the demands of the cognitive training program. In our exit interview, the TCT+PRIME group rated the exercises as significantly more difficult than the CG+PRIME group. This is consistent with a recent study of BrainHQ exercises versus computer control games in survivors of breast cancer [41]. Cognitive training was seen as challenging, engaging, and gave a sense of accomplishment, whereas control games were seen as a way of taking mind off issues, enjoyable, and easy to navigate. The barriers to cognitive training included an awareness of failing, whereas control games were deemed to be too repetitive.

    PRIME Induces Improvement in Symptoms

    Symptoms improved over time in the total sample. Specifically, symptom reductions were observed in the Beck Depression Inventory and QSAPS, whereas the QSANS showed a decrease at the trend level, although this was not statistically significant, indicating improvement in depressive, positive, and negative symptoms. This pattern is consistent with our previous studies of computerized cognitive training in individuals with chronic schizophrenia [13] and recent-onset schizophrenia [12] where reductions in symptoms were evident at a 6-month follow-up in those who completed the cognitive training. What is unique about this study compared with previous studies is that the control group did not receive cognitive training, but it still showed a modest improvement in symptoms. This difference may be attributable to the PRIME app, as both groups had access to the resources on the app. Our results on depressive symptom reduction have been found in previous research comparing PRIME with treatment as usual or waitlist [22]. Finally, meta-analyses of cognitive training in psychosis have shown comparable small effect sizes at posttraining on symptoms [17,42] but not at follow-up [42].

    The overall pattern of most of the functional outcome measures showed no evidence of a meaningful difference, which parallels the findings of our previous studies [7,12,13]. Two exceptions were found in the QLS Environmental Engagement subscale and the RFS Family Network. The QLS Environmental Engagement subscale showed a significant group-by-time interaction, with the TCT+PRIME group showing improvement and the CG+PRIME group showing a decline from baseline to posttraining and from baseline to follow-up. However, there were no statistically significant changes in the main functional outcomes of the QLS (ie, social and occupational functioning). The RFS Family Network showed a decline across the total sample. Most of the decrease occurred from baseline to posttraining while participants were using the PRIME app. Increased engagement with the PRIME community may have led to a decline in family involvement. Furthermore, although the change was significant, the mean rating at follow-up was still in the moderate range, indicating that participants, on average, were still seeing a family member more than once a month across all time points.

    Improving everyday functioning likely requires both cognitive training and functional skills training [43]. Future studies are needed to determine the effects of remotely delivered cognitive training provided with remotely delivered psychosocial treatments. A recent review and meta-analysis of remotely delivered, evidence-based psychosocial treatments for schizophrenia spectrum disorders found a limited number of studies, with effect sizes on functioning in the small range, and similar to the effect sizes of these treatments when provided in person [44]. One promising approach is metacognitive training, which combines psychoeducation, cognitive bias modification, and strategy teaching, with effect sizes ranging from small to moderate on functioning, and evidence of durable improvements in symptoms and functioning up to 1 year postintervention [45].

    Satisfaction, Interest, and Enjoyment With Cognitive Training and PRIME

    Responses between the TCT and CG groups regarding specific training experiences were not significantly different, with the exception of ease of training, which was rated more highly by the CG group. Although participants were typically neutral in their reports of enjoyment and interest in the exercises, cognitive training appeared to be engaging enough to avoid random guessing, with only 1 participant falling below the 50% accuracy. Group differences were observed for questions probing immediate or anticipated impact of the training (eg, “The training will help me do well at school or work.”) that tended to elicit higher ratings in the TCT group, although these differences were not statistically significant. Participants in both groups tended to rate the features of PRIME favorably, averaging 7.7 out of 10 in overall satisfaction.

    Previous studies of cognitive training in other diagnoses have also found that participants’ ratings of control and active cognitive training conditions are similar in terms of enjoyment and engagement and in terms of perceived cognitive gains [46,47]. Keefe et al [46] noted that increasing the enjoyability of digital training interventions could lead to greater adherence and greater cognitive gains. In turn, daily functioning could be affected by encouraging efforts to deploy those improved cognitive skills. In a recent commentary, Harvey [48] notes this as potentially very important, given that improvement in objective cognitive performance and subjective assessments of performance are associated with improvements in everyday functioning [49]. Thus, important future directions include (1) increasing the enjoyability of digital cognitive training interventions and (2) providing interventions, such as metacognitive training, that might increase participants’ insights into cognitive improvements and encouragement of the use of these improvements in daily functioning.

    Strengths and Limitations

    The strengths of this study include our relatively large sample size and racial, geographic, clinical, and gender diversity. PRIME is an innovative digital intervention targeting domains in psychotic disorders that have few alternative treatments currently available. All elements of the study were fully remote and delivered to English-speaking individuals in 8 different countries. The browser interface of cognitive training and the mobile interface of PRIME make both interventions highly scalable, although access to a laptop, a smartphone, and the internet are required. The remote nature of the interventions also created limitations in assessing the quality of engagement with TCT or CG, as we were unable to assess active and distracted engagement or random inputs (although this was likely not the case, as only 1 participant showed a cognitive training accuracy level <50%). Additional limitations arise from our sample, which was recruited on the web and consisted of highly educated participants with average baseline cognitive performance. Future studies should consider strategies to increase the recruitment of a more diverse sample.

    Conclusions

    This study aimed to identify and inform scalable remote interventions that address both cognitive and motivational impairments in psychosis using neuroscience-informed cognitive training exercises and a novel smartphone-based app. Our retention rate of 78% (78/100) and the national and international geography of participation in our study indicate that this approach is acceptable, feasible, tolerable, and scalable and that cognitive training and the PRIME app can induce improvements in emotion recognition, motivation, and symptoms that persist 6 months after the interventions. Future analyses will test models of the relationship between hours of cognitive training completed; PRIME use; and changes in cognition, motivation, symptoms, and functioning.

    Acknowledgments

    This work was supported by the Stanley Medical Research Institute (grant 15T-010) and the National Center for Advancing Translational Sciences of the National Institutes of Health (award UL1-TR002494). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    Data Availability

    The data sets generated during this study are available from the corresponding author upon reasonable request.

    Authors' Contributions

    SV conceptualized and acquired funding for this study. Data curation and formal analysis were performed by MF. The investigation was conducted by MF, KE, AM, NG, KLC, and AC. The methodology was developed by MF, AM, AC, BB, and SV. NG was responsible for the primary administration of the study. RL provided resources, including the Personalized Real-Time Intervention for Motivational Enhancement (PRIME) app and data storage, along with overseeing PRIME coaches and assessment staff from the University of California, San Francisco. BB from Posit Science provided consultation and access to BrainHQ cognitive training exercises and computer games for the control group. DS provided the PRIME app. MF, AM, AC, KF, and SV supervised this study. IR provided validation of the cognitive training and control games completed. KE prepared visualizations of the PRIME app and locations of participants. MF, KE, and AM drafted the manuscript, and all authors were involved in reviewing and editing the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Cognitive and social cognitive training exercises.

    DOCX File , 13 KB
    Multimedia Appendix 2

    Commercially available computer games.

    DOCX File , 14 KB
    Multimedia Appendix 3

    Effect sizes (Cohen d) from baseline to posttraining and baseline to 6-month follow-up in measures showing statistically significant or trend-level, but not significant, main effects of time.

    DOCX File , 14 KB
    Multimedia Appendix 4

    CONSORT-eHEALTH (V 1.6.1).

    PDF File (Adobe PDF File), 1279 KB
    1. Kahn RS, Sommer IE, Murray RM, Meyer-Lindenberg A, Weinberger DR, Cannon TD, et al. Schizophrenia. Nat Rev Dis Primers. Nov 12, 2015;1:15067. [CrossRef] [Medline]
    2. Harvey PD. Assessing disability in schizophrenia: tools and contributors. J Clin Psychiatry. Oct 2014;75(10):e27. [CrossRef] [Medline]
    3. Aleman A, Lincoln TM, Bruggeman R, Melle I, Arends J, Arango C, et al. Treatment of negative symptoms: where do we stand, and where do we go? Schizophr Res. Aug 2017;186:55-62. [CrossRef] [Medline]
    4. Fett AK, Viechtbauer W, Dominguez MD, Penn DL, van Os J, Krabbendam L. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci Biobehav Rev. Jan 2011;35(3):573-588. [CrossRef] [Medline]
    5. Green MF, Kern RS, Braff DL, Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the "right stuff"? Schizophr Bull. 2000;26(1):119-136. [CrossRef] [Medline]
    6. Fisher M, Holland C, Merzenich MM, Vinogradov S. Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. Am J Psychiatry. Jul 2009;166(7):805-811. [FREE Full text] [CrossRef] [Medline]
    7. Fisher M, Holland C, Subramaniam K, Vinogradov S. Neuroplasticity-based cognitive training in schizophrenia: an interim report on the effects 6 months later. Schizophr Bull. Jul 2010;36(4):869-879. [FREE Full text] [CrossRef] [Medline]
    8. Fisher M, Loewy R, Carter C, Lee A, Ragland JD, Niendam T, et al. Neuroplasticity-based auditory training via laptop computer improves cognition in young individuals with recent onset schizophrenia. Schizophr Bull. Jan 2015;41(1):250-258. [FREE Full text] [CrossRef] [Medline]
    9. Loewy R, Fisher M, Schlosser DA, Biagianti B, Stuart B, Mathalon DH, et al. Intensive auditory cognitive training improves verbal memory in adolescents and young adults at clinical high risk for psychosis. Schizophr Bull. Jul 2016;42 Suppl 1(Suppl 1):S118-S126. [FREE Full text] [CrossRef] [Medline]
    10. Subramaniam K, Luks TL, Fisher M, Simpson GV, Nagarajan S, Vinogradov S. Computerized cognitive training restores neural activity within the reality monitoring network in schizophrenia. Neuron. Feb 23, 2012;73(4):842-853. [FREE Full text] [CrossRef] [Medline]
    11. Subramaniam K, Luks TL, Garrett C, Chung C, Fisher M, Nagarajan S, et al. Intensive cognitive training in schizophrenia enhances working memory and associated prefrontal cortical efficiency in a manner that drives long-term functional gains. Neuroimage. Oct 01, 2014;99:281-292. [FREE Full text] [CrossRef] [Medline]
    12. Loewy R, Fisher M, Ma S, Carter C, Ragland JD, Niendam TA, et al. Durable cognitive gains and symptom improvement are observed in individuals with recent-onset schizophrenia 6 months after a randomized trial of auditory training completed remotely. Schizophr Bull. Jan 21, 2022;48(1):262-272. [FREE Full text] [CrossRef] [Medline]
    13. Miley K, Fisher M, Nahum M, Howard E, Rowlands A, Brandrett B, et al. Six month durability of targeted cognitive training supplemented with social cognition exercises in schizophrenia. Schizophr Res Cogn. Dec 26, 2019;20:100171. [FREE Full text] [CrossRef] [Medline]
    14. Ramsay IS, Fryer S, Boos A, Roach BJ, Fisher M, Loewy R, et al. Response to targeted cognitive training correlates with change in thalamic volume in a randomized trial for early schizophrenia. Neuropsychopharmacology. Feb 2018;43(3):590-597. [FREE Full text] [CrossRef] [Medline]
    15. Ramsay IS, Roach BJ, Fryer S, Fisher M, Loewy R, Ford JM, et al. Increased global cognition correlates with increased thalamo-temporal connectivity in response to targeted cognitive training for recent onset schizophrenia. Schizophr Res. Apr 2020;218:131-137. [FREE Full text] [CrossRef] [Medline]
    16. Ramsay IS, Fryer S, Roach BJ, Boos A, Fisher M, Loewy R, et al. Response to targeted cognitive training may be neuroprotective in patients with early schizophrenia. Psychiatry Res Neuroimaging. Jun 30, 2021;312:111285. [FREE Full text] [CrossRef] [Medline]
    17. Vita A, Barlati S, Ceraso A, Nibbio G, Ariu C, Deste G, et al. Effectiveness, core elements, and moderators of response of cognitive remediation for schizophrenia: a systematic review and meta-analysis of randomized clinical trials. JAMA Psychiatry. Aug 01, 2021;78(8):848-858. [FREE Full text] [CrossRef] [Medline]
    18. Lejeune JA, Northrop A, Kurtz MM. A meta-analysis of cognitive remediation for schizophrenia: efficacy and the role of participant and treatment factors. Schizophr Bull. Jul 08, 2021;47(4):997-1006. [FREE Full text] [CrossRef] [Medline]
    19. Mote J, Minzenberg MJ, Carter CS, Kring AM. Deficits in anticipatory but not consummatory pleasure in people with recent-onset schizophrenia spectrum disorders. Schizophr Res. Oct 2014;159(1):76-79. [FREE Full text] [CrossRef] [Medline]
    20. Kring AM, Barch DM. The motivation and pleasure dimension of negative symptoms: neural substrates and behavioral outputs. Eur Neuropsychopharmacol. May 2014;24(5):725-736. [FREE Full text] [CrossRef] [Medline]
    21. McCarthy JM, Treadway MT, Bennett ME, Blanchard JJ. Inefficient effort allocation and negative symptoms in individuals with schizophrenia. Schizophr Res. Feb 2016;170(2-3):278-284. [FREE Full text] [CrossRef] [Medline]
    22. Schlosser DA, Campellone TR, Truong B, Etter K, Vergani S, Komaiko K, et al. Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophr Bull. Aug 20, 2018;44(5):1010-1020. [FREE Full text] [CrossRef] [Medline]
    23. Schlosser D, Campellone T, Kim D, Truong B, Vergani S, Ward C, et al. Feasibility of PRIME: a cognitive neuroscience-informed mobile app intervention to enhance motivated behavior and improve quality of life in recent onset schizophrenia. JMIR Res Protoc. Apr 28, 2016;5(2):e77. [FREE Full text] [CrossRef] [Medline]
    24. Home page. Schizophrenia. URL: http://schizophrenia.com/ [accessed 2023-10-24]
    25. Nahum M, Fisher M, Loewy R, Poelke G, Ventura J, Nuechterlein KH, et al. A novel, online social cognitive training program for young adults with schizophrenia: a pilot study. Schizophr Res Cogn. Mar 01, 2014;1(1):e11-e19. [FREE Full text] [CrossRef] [Medline]
    26. Dabit S, Quraishi S, Jordan J, Biagianti B. Improving social functioning in people with schizophrenia-spectrum disorders via mobile experimental interventions: Results from the CLIMB pilot trial. Schizophr Res Cogn. Jul 28, 2021;26:100211. [FREE Full text] [CrossRef] [Medline]
    27. Dijkstra A. The psychology of tailoring-ingredients in computer-tailored persuasion. Soc Personal Psychol Compass. Feb 21, 2008;2(2):765-784. [FREE Full text] [CrossRef]
    28. Muench F, van Stolk-Cooke K, Morgenstern J, Kuerbis AN, Markle K. Understanding messaging preferences to inform development of mobile goal-directed behavioral interventions. J Med Internet Res. Feb 05, 2014;16(2):e14. [FREE Full text] [CrossRef] [Medline]
    29. Woolford SJ, Barr KL, Derry HA, Jepson CM, Clark SJ, Strecher VJ, et al. OMG do not say LOL: obese adolescents' perspectives on the content of text messages to enhance weight loss efforts. Obesity (Silver Spring). Dec 2011;19(12):2382-2387. [FREE Full text] [CrossRef] [Medline]
    30. Frese FJ3, Stanley J, Kress K, Vogel-Scibilia S. Integrating evidence-based practices and the recovery model. Psychiatr Serv. Nov 2001;52(11):1462-1468. [CrossRef] [Medline]
    31. Frese FJ3, Knight EL, Saks E. Recovery from schizophrenia: with views of psychiatrists, psychologists, and others diagnosed with this disorder. Schizophr Bull. Mar 2009;35(2):370-380. [FREE Full text] [CrossRef] [Medline]
    32. Tableau. URL: https://www.tableau.com/
    33. Fisher M, Mellon SH, Wolkowitz O, Vinogradov S. Neuroscience-informed auditory training in schizophrenia: a final report of the effects on cognition and serum brain-derived neurotrophic factor. Schizophr Res Cogn. Mar 01, 2016;3:1-7. [FREE Full text] [CrossRef] [Medline]
    34. Moseson H, Kumar S, Juusola JL. Comparison of study samples recruited with virtual versus traditional recruitment methods. Contemp Clin Trials Commun. Sep 2020;19:100590. [FREE Full text] [CrossRef] [Medline]
    35. Biagianti B, Schlosser D, Nahum M, Woolley J, Vinogradov S. Creating live interactions to mitigate barriers (CLIMB): a mobile intervention to improve social functioning in people with chronic psychotic disorders. JMIR Ment Health. Dec 13, 2016;3(4):e52. [FREE Full text] [CrossRef] [Medline]
    36. Roisum R, Jenkins D, Fisher M, Currie A, Ma S, Lindgren C, et al. Targeting cognition and motivation in coordinated specialty care for early psychosis: a grant report. J Psychiatr Brain Sci. 2020;5:e200023. [FREE Full text] [CrossRef] [Medline]
    37. Bellack AS, Green MF, Cook JA, Fenton W, Harvey PD, Heaton RK, et al. Assessment of community functioning in people with schizophrenia and other severe mental illnesses: a white paper based on an NIMH-sponsored workshop. Schizophr Bull. May 2007;33(3):805-822. [FREE Full text] [CrossRef] [Medline]
    38. Granholm E, Ben-Zeev D, Link PC. Social disinterest attitudes and group cognitive-behavioral social skills training for functional disability in schizophrenia. Schizophr Bull. Sep 2009;35(5):874-883. [FREE Full text] [CrossRef] [Medline]
    39. Granholm E, Holden J, Link PC, McQuaid JR, Jeste DV. Randomized controlled trial of cognitive behavioral social skills training for older consumers with schizophrenia: defeatist performance attitudes and functional outcome. Am J Geriatr Psychiatry. Mar 2013;21(3):251-262. [FREE Full text] [CrossRef] [Medline]
    40. Grant PM, Beck AT. Defeatist beliefs as a mediator of cognitive impairment, negative symptoms, and functioning in schizophrenia. Schizophr Bull. Jul 2009;35(4):798-806. [FREE Full text] [CrossRef] [Medline]
    41. Von Ah D, Crouch A, Storey S. Acceptability of computerized cognitive training and global cognitive stimulating-based games delivered remotely: results from a randomized controlled trial to address cancer and cancer-related cognitive impairment in breast cancer survivors. Cancer Med. Jun 2023;12(11):12717-12727. [FREE Full text] [CrossRef] [Medline]
    42. Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am J Psychiatry. May 2011;168(5):472-485. [CrossRef] [Medline]
    43. Bowie CR, McGurk SR, Mausbach B, Patterson TL, Harvey PD. Combined cognitive remediation and functional skills training for schizophrenia: effects on cognition, functional competence, and real-world behavior. Am J Psychiatry. Jul 2012;169(7):710-718. [CrossRef] [Medline]
    44. Best MW, Romanowska S, Zhou Y, Wang L, Leibovitz T, Onno KA, et al. Efficacy of remotely delivered evidence-based psychosocial treatments for schizophrenia-spectrum disorders: a series of systematic reviews and meta-analyses. Schizophr Bull. Jul 04, 2023;49(4):973-986. [CrossRef] [Medline]
    45. Penney D, Sauvé G, Mendelson D, Thibaudeau É, Moritz S, Lepage M. Immediate and sustained outcomes and moderators associated with metacognitive training for psychosis: a systematic review and meta-analysis. JAMA Psychiatry. May 01, 2022;79(5):417-429. [FREE Full text] [CrossRef] [Medline]
    46. Keefe RS, Cañadas E, Farlow D, Etkin A. Digital intervention for cognitive deficits in major depression: a randomized controlled trial to assess efficacy and safety in adults. Am J Psychiatry. Jul 2022;179(7):482-489. [CrossRef] [Medline]
    47. Mahncke HW, DeGutis J, Levin H, Newsome MR, Bell MD, Grills C, et al. A randomized clinical trial of plasticity-based cognitive training in mild traumatic brain injury. Brain. Aug 17, 2021;144(7):1994-2008. [FREE Full text] [CrossRef] [Medline]
    48. Harvey PD. Digital therapeutics to enhance cognition in major depression: how can we make the cognitive gains translate into functional improvements? Am J Psychiatry. Jul 2022;179(7):445-447. [CrossRef] [Medline]
    49. Rnic K, Jung YE, Torres I, Chakrabarty T, LeMoult J, Vaccarino AL, et al. Association between discrepancy in objective and subjective cognitive abilities and treatment response in patients with major depressive disorder: a CAN-BIND-1 study report. J Affect Disord. Dec 01, 2021;295:1095-1101. [CrossRef] [Medline]

    ‎
    CBT: cognitive behavioral therapy
    CG: computer games control condition
    DSM-5: Diagnostic and Statistical Manual of Mental Disorders, 5th Edition
    MAPS: Motivation and Pleasure Scale
    MSQ: Motivation State Questionnaire
    PRIME: Personalized Real-Time Intervention for Motivational Enhancement
    QLS: Quality of Life Scale
    QSANS: Quick Scale for the Assessment of Negative Symptoms
    QSAPS: Quick Scale for the Assessment of Positive Symptoms
    REDCap: Research Electronic Data Capture
    RFS: Role Functioning Scale
    SCID-5-RV: Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, 5th Edition, Research Version
    SMART: Specific, Measurable, Achievable, Realistic, Timely
    TCT: targeted cognitive and social cognitive training
    TEPS: Temporal Experience of Pleasure Scale

    Edited by T de Azevedo Cardoso; submitted 01.05.23; peer-reviewed by P Harvey, Y Jeem; comments to author 07.09.23; revised version received 14.09.23; accepted 26.09.23; published 13.11.23.

    Copyright

    ©Melissa Fisher, Kevin Etter, Aimee Murray, Neelu Ghiasi, Kristin LaCross, Ian Ramsay, Ariel Currie, Karrie Fitzpatrick, Bruno Biagianti, Danielle Schlosser, Rachel Loewy, Sophia Vinogradov. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.11.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.