Examining Use Behavior of a Goal-Supporting mHealth App in Primary Care Among Patients With Multiple Chronic Conditions: Qualitative Descriptive Study

Background Although mobile health (mHealth) apps are increasingly being used to support patients with multiple chronic conditions (multimorbidity), most mHealth apps experience low interaction and eventual abandonment. To tackle this engagement issue, when developing an mHealth program, it is important to understand the social-behavioral factors that affect patients’ use behavior. Objective The aim of this study was to explore the social and behavioral factors contributing to patients’ use behavior of an mHealth app called the electronic Patient-Reported Outcome (ePRO). The ePRO app supports goal-oriented care delivery in interdisciplinary primary care models. Methods A descriptive qualitative study was used to analyze interview data collected for a larger mixed methods pragmatic trial. The original 15-month trial was conducted in 6 primary care teams across Ontario, Canada, between 2018 and 2019. The eligibility criteria for patients were being aged ≥60 years with ≥10 visits within the previous 12 months of study enrollment. For this analysis, patients were classified as long-term or short-term users based on their length of use of the ePRO app during the trial. The Social Cognitive Theory by Bandura was used to categorize social-behavioral factors that contributed to patients’ decision to continue or discontinue using the app. Results The patient-provider relationship emerged as a key factor that shaped patients’ experiences with the app and subsequent decision to continue using the app. Other factors that contributed to patients’ decision to continue using the app were personal and social circumstances, perceived usefulness, patients’ previous experience with goal-related behaviors, and confidence in one’s capability. There was an overlap of experience between long- and short-term app users but, in general, long-term users perceived the app to be more useful and their goals to be more meaningful than short-term app users. This observation was complicated by the fact that patient health-related goals were dynamic and changed over time. Conclusions Complex patients’ use behavior of a goal-supporting mHealth app is shaped by an array of sociobehavioral factors that can evolve. To tackle this dynamism, there should be an emphasis on creating adaptable health technologies that are easily customizable by patients and able to respond to their changing contexts and needs. Trial Registration ClinicalTrials.gov NCT02917954; https://clinicaltrials.gov/ct2/show/NCT02917954

The CONSORT-EHEALTH checklist is intended for authors of randomized trials evaluating web-based and Internet-based applications/interventions, including mobile interventions, electronic games (incl multiplayer games), social media, certain telehealth applications, and other interactive and/or networked electronic applications. Some of the items (e.g. all subitems under item 5 -description of the intervention) may also be applicable for other study designs.
The goal of the CONSORT EHEALTH checklist and guideline is to be a) a guide for reporting for authors of RCTs, b) to form a basis for appraisal of an ehealth trial (in terms of validity) CONSORT-EHEALTH items/subitems are MANDATORY reporting items for studies published in the Journal of Medical Internet Research and other journals / scientific societies endorsing the checklist.
As the CONSORT-EHEALTH checklist is still considered in a formative stage, we would ask that you also RATE ON A SCALE OF 1-5 how important/useful you feel each item is FOR THE PURPOSE OF THE CHECKLIST and reporting guideline (optional).
Mandatory reporting items are marked with a red *.
In the textboxes, either copy & paste the relevant sections from your manuscript into this form -please include any quotes from your manuscript in QUOTATION MARKS, or answer directly by providing additional information not in the manuscript, or elaborating on why the item was not relevant for this study. e.g. "Stress", "Diabetes", or define the target group in brackets after the condition, e.g. "Autism (Parents of children with)", "Alzheimers (Informal Caregivers of)" "patients with multiple chronic conditions" You're editing your response. Sharing this URL allows others to also edit your response. yes: all primary outcomes were significantly better in intervention group vs control partly: SOME primary outcomes were significantly better in intervention group vs control no statistically significant difference between control and intervention potentially harmful: control was significantly better than intervention in one or more outcomes inconclusive: more research is needed

FILL OUT A NEW RESPONSE
Other: Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms. Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "goal-supporting mHealth app" Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Patients with multiple chronic conditions" Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The full trial procedure has been described in previous published paper which is cited in this manuscript.  Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons.  Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The positive benefit of using apps/web-based platforms to support complex patients is documented in the current literature. 10,11 For example, when patients with chronic illness use mHealth apps they are more likely to be engaged with health-promoting behavior such as fruit consumption and physical activity. 12 However, the benefits are more likely to be realized if technologies are used as intended. Instead, most mHealth apps experience high attrition 13,14, defined as when an individual disengages from a health technology after initially committing to using the technology for a trial. 15 Attrition has been considered a major challenge in mHealth-based interventions. 15-17 Previous research has identified that only a small number of participants use mHealth apps in the long term and the majority of the patients abandon the app after a short period. 16,18 The reason behind high attrition varies. Critical factors that drive attrition can include having available social support and capital, trust in technology, intention, and ability to use the app. 19-21 A meta-analysis evaluating the rates of attrition in mHealth interventions shows that many attrition-focused studies had relatively short intervention (less than 2 months) or short follow-up periods, ranging from 2 weeks to 2 months. 15 This synthesis work suggests a need to advance knowledge on attrition by exploring socio-cognitive factors that contribute to patients' app usage in the long term and real-world settings. 15 " Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Attrition has been considered a major challenge in mHealth-based interventions. 15-17 Previous research has identified that only a small number of participants use mHealth apps in the long term and the majority of the patients abandon the app after a short period. 16,18 The reason behind high attrition varies. Critical factors that drive attrition can include having available social support and capital, trust in technology, intention, and ability to use the app. 19-21 A meta-analysis evaluating the rates of attrition in mHealth interventions shows that many attrition-focused studies had relatively short intervention (less than 2 months) or short follow-up periods, ranging from 2 weeks to 2 months. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We conducted a descriptive qualitative sub-study, drawing on patients' interview data collected as part of a larger 15-month multi-site pragmatic stepped-wedge trial of the electronic Patient Reported Outcomes (ePRO) tool.22,23 The stepped-wedge design randomized 6 sites into two intervention groups. The first group used the ePRO app for 12 months after a 3-month control period, and the second group used the app for 9 months after a 6-month control period. 22 Out of the six case sites three sites were identified as case sites where qualitative interviews were conducted. The case sites were selected based on geographic location and the site's interest in participating in the qualitative part of the study alongside outcome data.22,23 The patient interviews, demographic surveys, and research memos collected in these three case sites were used to answer the research question of this paper." ? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No changes were made in the trial, hence, unreported. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Using FHT EMRs, eligible patients were identified. Then, the list of eligible patients was given to FHT providers to assess whether the patients met the following additional criteria: 1) perceived willingness to engage in goals of care conversation; 2) ability to use a smartphone or tablet in English or have a caregiver who could do this on their behalf; 3) capable of providing consent to participate; 4) willing to complete surveys every 3 months thereafter until the trial concluded. 51A detailed description of the recruitment procedure has been described elsewhere. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "the list of eligible patients was given to FHT providers to assess whether the patients met the following additional criteria: 1) perceived willingness to engage in goals of care conversation; 2) ability to use a smartphone or tablet in English or have a caregiver who could do this on their behalf; " You're editing your response. Sharing this URL allows others to also edit your response.

FILL OUT A NEW RESPONSE
subitem not at all important 1 2 3 4 5 essential subitem not at all important a c e a s s e s s m e n t s : Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these.
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The recruitment procedure has been published in the original pragmatic trial paper which is cited in the manuscript. But in short, we have recruited FHTs first and then the patients within it. "Using FHT EMRs, eligible patients were identified. Then, the list of eligible patients was given to FHT providers to assess whether the patients met the following additional criteria: 1) perceived willingness to engage in goals of care conversation; 2) ability to use a smartphone or tablet in English or have a caregiver who could do this on their behalf; 3) capable of providing consent to participate; 4) willing to complete surveys every 3 months thereafter until the trial concluded. 51A detailed description of the recruitment procedure has been described elsewhere. 44,51 Eligible patients were approached by their FHT staff (i.e. care coordinators, administrators) and asked if they would be willing to speak to a research team member about the project. Recruitment occurred during a scheduled office visit or by phone. " Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " The ePRO intervention was implemented in 6 Family Health Teams (FHTs) located across the province of Ontario, Canada from 2018-2019. Ontario is the largest province in Canada, with the highest population density, and the majority of services provided in the primary care teams are funded by the Ontario Ministry of Health and Long term Care. FHTs are designed to provide integrated, multidisciplinary primary care and are typically led by physicians or nurse practitioners. 22 " Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
C l e a r s e l e c t i o n You're editing your response. Sharing this URL allows others to also edit your response.

FILL OUT A NEW RESPONSE
subitem not at all important Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative sub-study, we did not have any online questionnaires. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We have cited the papers (protocol and full assessment of the paper) that have more information about the developers of the app. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Our full pragmatic trial reports on the history/development process of the app, which is cited in the manuscript.
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The app did not go through any major changes or events during the 15 month trial. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

-
As a qualitative study we have provided details on the decisions that were made during data collection, and analysis to ensure that readers can follow the logic trails.

-
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "One such theory is Bandura's SCT 27 which explains human behaviors through a model of interactions among behavioral, environmental, and social factors. This model has been used extensively to uncover which social and behavioral constructs may influence patients' usage behavior of a mHealth app. 28-30 SCT is particularly well-suited to examine the patients' usage behavior of a mHealth app like ePRO because this app enables and encourages patients to work toward their health-related goals such as: walking every day for 50 minutes.31 And previous literature suggests that this sort of mHealth-enabled healthpromoting trials and individuals' engagement with the mHealth is deeply influenced by their social-cognitive factors. 28,32 Therefore, the theoretical constructs of SCT allowed us to understand the social-cognitive related factors that contribute to the process by which patients decide to continue or discontinue app usage. For example, one of the SCT domains reciprocal determinism is helpful to identify how personal, environmental and behavioral factors can influence one's decision to continue/discontinue app use. Similarly, the behavior capability and goal efficacy domain were helpful to identify how one's skills and confidence can influence their decision of app usage. " Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability Copy and paste relevant sections from manuscript text "Based on patients' app-automated usage logs, patients were categorized into two categories: long-term users and short-term users. Of the 22 interviewed individuals, 9 were short-term users and 13 were long-term users. Participants who did not use the ePRO app after initial onboarding or used it for less than 3 months were categorized into the "shortterm user" group. In contrast, the participants who used the ePRO app for more than 3 months were categorized into the "long-term user" group. The 3-month cut-off period was determined because the app experienced a sharp decline in usage at 3 months. 34 This 3month cut-off period is also consistent with the previous literature. 42 " You're editing your response. Sharing this URL allows others to also edit your response. FILL Patients' demographic information was collected through a demographic survey at the beginning of the study. The first set of interviews was conducted at the midpoint of the trial, 4-6 months after patients started using the app (timing of the interviews depended on if they were in the 12 or 9-month use group). The second round of interviews was conducted at the end of the study trial. The semi-structured interview guide addressed the following topics: 1) perception and experience of using the ePRO app; 2) patients' relationship with their care team; 3) perception and experience of setting goals through ePRO; 4) impact of ePRO in patients' daily lives. Interviews were 25-40 minutes long and were conducted by one of four research team members (FT, TA, JS, and CSG). Each interview was audiotaped and transcribed using a commercial transcription service. Transcripts were checked for accuracy against recordings by a member of the research team. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative sub-study we only used participants who received the intervention. However, the original trial has been published elsewhere which contains information about randomization, and allocation sequence and cited in the manuscript. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative sub-study we only used participants who received the intervention. However, the original trial has been published elsewhere which has contains information about randomization and cited in the manuscript. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This study used a stepped-wedge cluster randomized trial, hence, the random allocation happened at the cluster level. We have provided brief overview on how randomization at the cluster level impacted this qualitative sub-study. " The stepped-wedge design randomized 6 sites into two intervention groups. The first group used the ePRO app for 12 months after a 3-month control period, and the second group used the app for 9 months after a 6-month control period. 22 Out of the six case sites three sites were identified as case sites where qualitative interviews were conducted. The case sites were selected based on geographic location and the site's interest in participating in the qualitative part of the study alongside outcome data.22,23 The patient interviews, demographic surveys, and research memos collected in these three case sites were used to answer the research question of this paper." However, a more detailed information about randomization has been described elsewhere and cited in the manuscript. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Our previously published paper contain information about random allocation sequence and we did not repeat the information in this manuscript. "A detailed description of the recruitment procedure has been described elsewhere. " However, the published paper is cited in this manuscript. Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator". Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative sub-study we interviewed participants when they have received the intervention. Aligning with stepped-wedge controlled trial, all participants in the intervention group acted as their own control. However, we did not interview participants when they were in the control group. Hence, the comparator group was not relevant to the current study. A detail description the trial design has been published elsewhere and cited in the manuscript. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative sub-study we did not conduct any statistical analysis of primary and secondary outcomes.
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4] Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Ethics approval was received from the University of Toronto Health Sciences Research Ethics Board and the Research Ethics Boards of three participating primary care practices. All patient participants provided informed verbal and written consent before initiation of study activities." The additional information about consent can be found in the original trial paper which is cited in the manuscript. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a low intensity program, we did not take any extra steps to reduce the likelihood of harm. However, participants had access to their usual primary care team during intervention and were to free to stop using the app any time.
You're editing your response. Sharing this URL allows others to also edit your response. FILL OUT A NEW RESPONSE Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This information regarding individuals who were randomly assigned and received intended treatment has been provided in full details in previous paper which has been cited in this paper. As a qualitative sub-study, we did not analyze any primary outcome. Additionally, we have only interviewed participants in the treatment term who has received the intervention.
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The losses and exclusions have been described in previous article which is cited in the paper.
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement.
You're editing your response. Sharing this URL allows others to also edit your response. Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The full recruitment procedure is published in a previous article and cited in this paper. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The trial ended before Covid-19 pandemic on 2018, hence, the trial did not experience any critical "secular events".
You're editing your response. Sharing this URL allows others to also edit your response. FILL OUT A NEW RESPONSE  D  o  e  s  y  o  u  r  p  a  p  e  r  a  d  d  r  e  s  s  C  O  N  S  O  R  T  s  u  b  i  t  e  m  1  4  b ? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The trial did not end or stop early. The trial ended at the end of the pre-planned study period. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We have provided participant demographic information including age, sex, computer literacy, income and education level in Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We have provided participant demographic information including age, sex, computer literacy, income and education level in Table 2.
Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention. Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative research study, we did not report any primary and secondary outcome in terms of effect size and precision.
You're editing your response. Sharing this URL allows others to also edit your response. In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative research study, we did not report any primary and secondary outcomes.
You're editing your response. Sharing this URL allows others to also edit your response. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative paper, we have conducted thematic analysis of the interview data. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a qualitative study, we did not conduct sub-group analysis.
You're editing your response. Sharing this URL allows others to also edit your response. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Our qualitative data analysis shows some unintended effects of the app on patients. "Additionally, patients reflected that their confidence and skills to perform a goal changes over time depending on their health. When patients felt that they were not able to perform their goals because of health and life circumstances and they did not have "enough" to report on the app, they discontinued using the app. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "This study used descriptive qualitative methods and re-storying analytic techniques to explore the social and behavioral factors contributing to the patients' usage behavior of the ePRO tool. Study findings show that patient-provider relationships, patients' social relationships, and patient personal circumstances all play a central role in their decision to continue or discontinue the use of the ePRO app." "Customizable technologies: The findings of this article support previous study findings that health technologies are often discontinued and abandoned because a technology lack features of meaningful customization which is not part of users' already existing devices such as personal phones. 50 Additionally, the study findings suggest that health-related goals change over time for patients with multiple chronic conditions, and so designing apps that offer patient-driven customization and modification techniques will be helpful to repurpose the same technology at multiple time points of lifecycle.66 For example, one patient (Patient #15) shared that their ability to achieve their goals has changed over time because of emerging health issues but they were unsure how to modify the goals in the ePRO app. This design feature in the ePRO was intentional based on a previous exploratory trial of the app (which was under 4 months). 38 In this exploratory trial, it was determined by patients and providers that setting goals together was an important motivator for patients. However, the finding of the current study contradicts the previous one. This contradiction may be caused by the prolonged use of ePRO, meaning after using the app for a while patients felt confident in the app to take charge of their goals. This further shows the importance of longitudinal evaluation of mHealth apps compared to shorter follow up time because patients' confidence, skills, and health needs from the app changes over time which may not be captured in a shorter trial. 15 Importance of chronic care paradigm shift from self-management to co-management of chronic conditions: In the current chronic care paradigm, the task of goal management is often left to patients. 3,51 Our study findings highlight that discussions around goal-oriented care are a one-time occurrence for study participants which was facilitated by introducing the ePRO app. After setting goals with patients, providers often leave it up to patients to be responsible for their own goals. On the other hand, patients do not bring up the topic of goals in their discussion as they perceive that their providers "are too busy" to attend to patients' goals, and providers' time could be better spent on other condition-related concerns. The study finding reflects that there is a need for an ongoing conversation between patient-provider about patient-centered goals to ensure the goals and associated devices/tools are appropriate for the patient's needs and serve the purpose that the goal/device set out to do. Similarly, the interview data suggests that patients considered their providers' enthusiasm for the ePRO intervention important and influenced patients' interest in two ways, which are: (a) monitoring of patient data by providers was considered important and (b) providers' encouragement to keep using the ePRO app. 52 This finding highlights the need for further education and training tools for health care providers on how to effectively have a goaloriented conversation with patients and within interprofessional teams. 10,53 " Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Firstly, due to scheduling conflict or loss-to follow-up of participants, we were not able to interview all participants at two different time points. However, the interviews that were conducted were in-depth and had rich information. Additionally, the patient population of this study was recruited from FHTs across Ontario. Hence, the findings of this study may not be transferable to other models of primary care such as community health centers or solo practice environments. Furthermore, the participant demography suggests that the study patient population were less complex and well-resourced meaning on average patients had a low number of chronic conditions, had high income and educational attainment level, which might not be representative of general complex patients. Therefore, the finding of the study may not be transferable for patients living in resource-poor communities or who have lower income or education levels. Additionally, the underrepresentation of low-income individuals is a common occurrence across multiple research studies and requires attention in study design to facilitate this population's participation. 55 Additionally, because of using deductive coding strategy, we may have missed constructs that would have emerged using the inductive method. " Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The pragmatic approach of this study which consisted of a broad exclusion criteria, minimal follow up from the research team and no manipulation of patient's usual care team would make the study highly applicable to similar patient population and care setting. You're editing your response. Sharing this URL allows others to also edit your response.
FILL Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study