This paper is in the following e-collection/theme issue:

Personal Health Records, Patient-Accessible Electronic Health Records, Patient Portals (538) mHealth for Wellness, Behavior Change and Prevention (2728) Design and Formative Evaluation of Mobile Apps (661) Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (782)

Published on in Vol 24, No 6 (2022): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/34191, first published .
Electronic Health Record–Based Recruitment and Retention and Mobile Health App Usage: Multisite Cohort Study

Electronic Health Record–Based Recruitment and Retention and Mobile Health App Usage: Multisite Cohort Study

Electronic Health Record–Based Recruitment and Retention and Mobile Health App Usage: Multisite Cohort Study

Authors of this article:

Janelle W Coughlin1, 2 Author Orcid Image ;   Lindsay M Martin3 Author Orcid Image ;   Di Zhao2, 4 Author Orcid Image ;   Attia Goheer4 Author Orcid Image ;   Thomas B Woolf5 Author Orcid Image ;   Katherine Holzhauer3 Author Orcid Image ;   Harold P Lehmann3 Author Orcid Image ;   Michelle R Lent6 Author Orcid Image ;   Kathleen M McTigue7 Author Orcid Image ;   Jeanne M Clark2, 3 Author Orcid Image ;   Wendy L Bennett2, 3 Author Orcid Image
Article Authors Cited by (11) Tweetations (10) Metrics

    Original Paper

    1Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, United States

    2Welch Center for Prevention, Epidemiology and Clinical Research, Johns Hopkins University, Baltimore, MD, United States

    3Division of General Internal Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, United States

    4Department of Health Policy and Management, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States

    5Department of Physiology, Johns Hopkins University School of Medicine, Baltimore, MD, United States

    6School of Professional and Applied Psychology, Philadelphia College of Osteopathic Medicine, Philadelphia, PA, United States

    7Division of General Internal Medicine, University of Pittsburgh, Pittsburgh, PA, United States

    Corresponding Author:

    Janelle W Coughlin, PhD

    Department of Psychiatry and Behavioral Sciences

    Johns Hopkins University School of Medicine

    5510 Nathan Shock Drive Suite 100

    Baltimore, MD, 21224

    United States

    Phone: 1 410 550 7988

    Email: jwilder3@jhmi.edu


    Background: To address the obesity epidemic, there is a need for novel paradigms, including those that address the timing of eating and sleep in relation to circadian rhythms. Electronic health records (EHRs) are an efficient way to identify potentially eligible participants for health research studies. Mobile health (mHealth) apps offer available and convenient data collection of health behaviors, such as timing of eating and sleep.

    Objective: The aim of this descriptive analysis was to report on recruitment, retention, and app use from a 6-month cohort study using a mobile app called Daily24.

    Methods: Using an EHR query, adult patients from three health care systems in the PaTH clinical research network were identified as potentially eligible, invited electronically to participate, and instructed to download and use the Daily24 mobile app, which focuses on eating and sleep timing. Online surveys were completed at baseline and 4 months. We described app use and identified predictors of app use, defined as 1 or more days of use, versus nonuse and usage categories (ie, immediate, consistent, and sustained) using multivariate regression analyses.

    Results: Of 70,661 patients who were sent research invitations, 1021 (1.44%) completed electronic consent forms and online baseline surveys; 4 withdrew, leaving a total of 1017 participants in the analytic sample. A total of 53.79% (n=547) of the participants were app users and, of those, 75.3% (n=412), 50.1% (n=274), and 25.4% (n=139) were immediate, consistent, and sustained users, respectively. Median app use was 28 (IQR 7-75) days over 6 months. Younger age, White race, higher educational level, higher income, having no children younger than 18 years, and having used 1 to 5 health apps significantly predicted app use (vs nonuse) in adjusted models. Older age and lower BMI predicted early, consistent, and sustained use. About half (532/1017, 52.31%) of the participants completed the 4-month online surveys. A total of 33.5% (183/547), 29.3% (157/536), and 27.1% (143/527) of app users were still using the app for at least 2 days per month during months 4, 5, and 6 of the study, respectively.

    Conclusions: EHR recruitment offers an efficient (ie, high reach, low touch, and minimal participant burden) approach to recruiting participants from health care settings into mHealth research. Efforts to recruit and retain less engaged subgroups are needed to collect more generalizable data. Additionally, future app iterations should include more evidence-based features to increase participant use.

    J Med Internet Res 2022;24(6):e34191

    doi:10.2196/34191

    Keywords

    mHealth; mobile apps; recruitment; engagement; retention; timing of eating; timing of sleep; obesity; EHR 


    Obesity and its related medical comorbidities are highly prevalent public health conditions [1-5]. The strongest evidence for preventing and treating obesity targets health behaviors to modify dietary composition, reduce calories, and increase physical activity [6-8]. Although reducing calories and increasing physical activity result in short-term weight loss, there is a need to identify lifelong behavioral patterns that promote longer-term weight loss and maintenance of healthy weight [9-11]. Aligning the timing of eating and sleeping with intrinsic circadian rhythm (ie, a shorter duration of eating, often called time-restricted eating or feeding) has not yet been thoroughly examined in population-based studies, but has the potential to provide a new paradigm to prevent obesity and metabolic conditions [12-15].

    Mobile devices and mobile health (mHealth) apps are ubiquitous, readily available approaches to collect real-time data on health behaviors, such as dietary intake, physical activity, and sleep [16-18]. mHealth apps are often designed and marketed to provide behavioral tracking and lifestyle modification support [19-22]. They also provide a convenient and efficient method for collecting information to advance knowledge about the relationship between obesity-related behavioral patterns and health outcomes [23-25].

    Although mHealth research has grown exponentially in the last few decades, study attrition is a major problem, and there is a need to identify successful, low-burden, and efficient recruitment and retention strategies [26,27]. The era of COVID-19, in particular, has additionally highlighted the importance of remote research procedures. Electronic health record (EHR)–based recruitment strategies provide potentially efficient (ie, low touch and low participant burden) methods for identifying and recruiting high volumes of patients meeting predetermined medical criteria for population-based research studies [28-30].

    This study presents a secondary analysis from a 6-month, multisite, cohort study that used the EHR to identify and recruit participants to use a mobile app (Daily24), designed to assess timing of eating and sleep [31]. The main goal of the parent observational study was to evaluate the longitudinal association between timing of eating and weight changes over time. Because of the growing interest in both EHR-based recruitment strategies and mHealth data collection methods [19,32-34], the goal of this descriptive analysis is to do the following:

    1. Describe the EHR-based recruitment and electronic consent (e-consent) methods and response rates for enrolling in the study and downloading the mobile app.
    2. Describe engagement strategies, app use, and retention rates during the 6-month study.
    3. Evaluate demographic and behavioral predictors of Daily24 app use.

    We hypothesized that people who are younger, have greater education, and have higher BMIs would be more likely to use the app. This study has the potential to inform the field of behavioral health in methodology, uptake, and engagement of mHealth approaches for observational research.


    Recruitment

    We recruited a cohort of adult patients from three health care systems in the PaTH Clinical Research Network, part of PCORnet (National Patient-Centered Research Network). The three health care systems included the Johns Hopkins Health System, the Geisinger Health System, and the University of Pittsburgh Medical Center [35-37].

    Ethics Approval

    Institutional Review Board (IRB) approval was obtained from the Johns Hopkins School of Medicine (IRB00174516), which had a reliance agreement with the other institutions’ IRBs.

    EHR-Based Participant Eligibility Criteria

    Potential participants were identified using EHR-based eligibility criteria (ie, “computable phenotype” [38]) to query the EHR. Each site also obtained a list of potentially eligible participants who previously consented to complete PaTH cohort studies at these sites [37]. Eligibility criteria included the following: at least 18 years of age and a minimum of one weight measurement and one height measurement recorded in the EHR between July 2017 and July 2019. Participants were excluded if they were deceased.

    Recruitment Messaging Via Email and the Patient Portal

    Potentially eligible participants were sent recruitment messages via email or the patient portal (ie, Epic MyChart) from February to July 2019. Each partnering health care system tailored its own strategy to recruit participants from the large pool of potentially eligible patients who were identified using the computable phenotype. One site used patient portal recruitment almost exclusively, focusing on patients who had a health system visit in the last week. The other two sites sent email recruitment letters through their primary care and weight management practices, with messages signed by the clinic medical directors. Multimedia Appendix 1 shows a sample recruitment message, which included a brief study description and link to a web-based e-consent form.

    e-Consent and Enrollment Process

    We designed a web-based e-consent process in REDCap (Research Electronic Data Capture) beginning with a study description, including participant expectations and duration (see Figure 1). Upon confirming interest, participants proceeded to the e-consent form, which included a supplemental audio clip of the consent form being read aloud, followed by a short quiz to ensure comprehension. Consenting participants provided identifying information (ie, full name, date of birth, and email), enabling staff to link each participant to the EHR for future analyses (not reported herein). Once consented, participants received a link to complete online baseline surveys (Figure 1). Participants were considered enrolled in the cohort after completing baseline surveys, at which time they received instructions on how to download and use the Daily24 mobile app.

    ‎
    Figure 1. Screenshots of web-based electronic recruitment and onboarding: electronic consent (top left), baseline surveys (top right), and POWER 28 and POWER week information (bottom).
    View this figure

    Daily24 Mobile App, Registration, and Download

    The Daily24 mobile app was custom designed by our research team to collect information from participants about the timing of eating and sleep, including wake time, sleep time, timing of each eating occasion, and estimate of amount eaten (ie, small, medium, or large meal; small or large snack; or drink, except water, without food) during a 24-hour window (Figure 2). The design of the app is described elsewhere [31]. We benefited from the input of patient and end-user stakeholders in the design of the mobile app, as well as recruitment and retention methods, and we pilot-tested the app [31,39].

    Following enrollment in the cohort, participants received a text message on their mobile phones with a unique link to the Daily24 registration form. This unique link contained a token (ie, 11-character universally unique identifier) that enabled the study team to connect participants’ data between the mobile app and their online enrollment information and surveys, while preserving privacy. Registration included an overview of how to use the app, study timeline, and incentives (see next section), followed by selection of a unique “Daily24 name” from a list of randomly generated combined nouns (eg, “FloatHarbor”) that, once selected, was the participant’s Daily24 username. Participants then received a link to download the Daily24 app via iOS (Apple Store) or Android (Google Play).

    ‎
    Figure 2. Screenshots of the Daily24 app: empty sleep ring (top left), complete sleep ring (top middle), empty food ring (top right), meal size selection (bottom left), complete food ring (bottom middle), and complete day (bottom right).
    View this figure

    Engagement Strategies to Promote Use of the Daily24 Mobile App

    Although we encouraged participants to enter as much data as possible over the 6-month study, we developed and applied strategies aimed at maximizing app use during their first 4 weeks of participation (ie, 28 days after downloading the app, called “POWER 28”) and 1 week per month for the remaining 5 months of the study, called “POWER weeks” (Figure 1). These highly targeted usage days for the study were equivalent to 63 days (POWER 28 + POWER weeks × 5 weeks). Engagement strategies included a leaderboard, badges, raffles, and text reminders. The leaderboard displayed the number of consecutive days tracked on one tab (ie, streak) and total number of all days tracked on the other tab. Earned badges encouraged various types of app use, including one-time badges, streak badges, and POWER week badges (Figure 3). We raffled off US $25 gift cards weekly throughout the study, with those earning more badges having greater odds of winning the raffle. We used emails, SMS text messages, and in-app notifications to encourage usage and to remind participants where they were in their POWER 28 and when a POWER week was coming up. The logic for these messages was triggered both by time (ie, close to a POWER week) as well as by lack of a response (ie, an event missing data). If a participant was on track with logging events, we simply encouraged their continued involvement and did not send additional reminders.

    ‎
    Figure 3. Screenshots of the badges earned to encourage usage in the Daily24 app.
    View this figure

    Data Collection

    Daily24 app usage data were collected using Amazon Web Services. Self-reported online surveys were administered using REDCap at baseline and at the end of 4 months using standard measures assessing demographics, mHealth use, height, and weight, as well as eating, physical activity, and sleep habits. Technology use and health app use were assessed using the Pew Social Media Update 2016 [40] and a survey measuring characteristics of health app use [16], respectively. Nutritional and eating assessments included the National Health and Nutrition Examination Survey 2009-2010 Dietary Screener Questionnaire [41], which provided estimates of fruit, vegetable, and sugar-sweetened beverage intake over the last 30 days. Physical activity was assessed using the self-administered, short version of the International Physical Activity Questionnaire [42]; physical activity levels were categorized as low, medium, or high over the last 7 days. Sleep measurements included the single sleep quality item from the Pittsburgh Sleep Quality Index [43] and study-created questions about frequency of daytime naps.

    To facilitate and encourage baseline survey completion, participants received automated reminders at 15 minutes, 24 hours, and 48 hours after consent and had up until 90 days after receiving the initial survey link to complete the survey. Personalized survey-engagement strategies included a combination of staff emails, text messages, and US $100 raffle gift cards. Participants had up until 60 days to complete the 4-month measures, but this paper only reports baseline survey descriptive results. Data collection was completed in January 2020.

    App Usage Categories

    App users were defined as using the Daily24 app for at least one day, which was captured by having at least one meal and sleep entry, on at least one day, and pushing “done for the day” on the screen. Nonusers either did not register or download the app or did not push “done for the day” on any day. App use was further categorized into three non–mutually exclusive ways:

    1. Immediate use, defined as using the app for 7 days or more during the POWER 28.
    2. Consistent use, defined as using the app for 28 days or more during the entire 6-month study, which was based on using the app equal to or more than the median overall days of use for the entire 6-month study.
    3. Sustained use, defined as using the app for at least 2 days during the last POWER week (month 6) of the study.

    Statistical Analyses

    This was a secondary analysis of data from a parent cohort study. We used descriptive statistics (Student t tests or χ2 tests) for baseline characteristics for all participants and by app use versus nonuse categories. App use was also described in median days of use for the total study, median days used in targeted and nontargeted usage days, and frequency of 2 or more and 7 or more days of use during each month of the study. We selected these two categories based on the following logic:

    1. Two or more days: this was selected to represent a low threshold of app use that was not identical to the minimal definition of being an app user.
    2. Seven or more days: this was selected because we focused on POWER weeks during months 2 to 6 and wanted to capture those who achieved at least one week of usage.

    We evaluated the association between baseline characteristics, with app usage as the dependent variable, using multivariable logistic regression models. Multivariable logistic regression was also used to model the association between baseline characteristics with immediate, consistent, and sustained app use. We used two models with progressive adjustment. Model 1 adjusted for key demographics, including age, sex, race, education, household income, and children younger than 18 years old. Model 2 additionally adjusted for key behavioral factors that could influence engagement, including physical activity, fruit and vegetable servings, sleep quality, and BMI. Covariates were nonmissing, prespecified, and based on a priori hypotheses.


    Enrollment and App Use

    Figure 4 shows the enrollment flow of eligible participants. Electronic recruitment messages were sent to 70,661 potentially eligible participants, with 1253 participants (1.77%) completing the e-consent process and 1021 (1.44%) enrolling by completing baseline surveys. A total of 4 participants withdrew, leaving 1017 participants included in the analytic sample. Participant characteristics are reported in Table 1. The majority of the 1017 participants were female (n=790, 77.68%), White (n=788, 77.48%), and college graduates (n=749, 73.65%), and the mean age was 51.1 (SD 15.0) years.

    ‎
    Figure 4. Recruitment and retention flow. REDCap: Research Electronic Data Capture.
    View this figure
    Table 1. Description of study participants and potential confounders at baseline.
    Characteristics and confoundersAll participants (N=1017)Non–app users (n=470)App usersa (n=547)P valueb
    Age (years), mean (SD)51.1 (15.0)53.2 (14.6)49.3 (15.0)<.001
    Gender, n (%)

    		Male224 (22.0)115 (24.5)109 (19.9).07

    		Female790 (77.7)355 (75.5)435 (79.5)

    		Prefer not to answer3 (0.3)0 (0)3 (0.5)
    Race, n (%)

    		White788 (77.5)351 (74.7)437 (79.9).14

    		Black149 (14.7)82 (17.4)67 (12.2)

    		Asian29 (2.9)13 (2.8)16 (2.9)

    		Pacific Islander, American Indian, or others17 (1.7)10 (2.1)7 (1.3)

    		Two or more races34 (3.3)14 (3.0)20 (3.7)
    Site, n (%)

    		Site A51 (5.0)23 (4.9)28 (5.1).004

    		Site B282 (27.7)105 (22.3)177 (32.4)

    		Site C200 (19.7)96 (20.4)104 (19.0)
    Educational level, n (%)

    		High school or less63 (6.2)40 (8.5)23 (4.2)<.001

    		Some college205 (20.2)109 (23.2)96 (17.6)

    		College graduate749 (73.6)321 (68.3)428 (78.2)
    Annual household income (US $), n (%)

    		<35,000120 (11.8)70 (14.9)50 (9.1).02

    		35,000 to <50,000109 (10.7)53 (11.3)56 (10.2)

    		50,000 to <75,000148 (14.6)66 (14.0)82 (15.0)

    		≥75,000550 (54.1)234 (49.8)316 (57.8)

    		Don’t know/choose not to answer90 (8.8)47 (10.0)43 (7.9)
    Any child <18 years old, n (%)248 (24.4)129 (27.4)119 (21.8).04
    Height (cm), mean (SD)168.9 (50.2)170.6 (73.3)167.4 (8.5).31
    Weight (kg), mean (SD)85.8 (23.8)86.8 (25.1)85.0 (22.5).23
    BMIc, mean (SD)30.5 (7.9)30.8 (8.2)30.3 (7.6).29
    BMI categories, n (%)

    		Underweight (<18.5)14 (1.4)7 (1.5)7 (1.3).75

    		Normal (18.5 to <25)250 (24.6)111 (23.6)139 (25.4)

    		Overweight (25 to <30)288 (28.3)129 (27.4)159 (29.1)

    		Obese (≥30)465 (45.7)223 (47.4)242 (44.2)
    Fruit or vegetable cup equivalent, mean (SD)2.9 (1.5)2.8 (1.6)3.0 (1.4).18
    Added sugars tsp equivalent from sugar-sweetened beverages, mean (SD)0.8 (1.3)1.0 (1.5)0.7 (1.2).004
    Physical activity, n (%)

    		Low20 (5.1)11 (6.3)9 (4.1).53

    		Medium221 (55.8)94 (53.7)127 (57.5)

    		High155 (39.1)70 (40.0)85 (38.5)
    Sleep quality, n (%)

    		Very good188 (18.5)76 (16.2)112 (20.5).02

    		Fairly good496 (48.8)222 (47.2)274 (50.1)

    		Fairly bad274 (26.9)136 (28.9)138 (25.2)

    		Very bad59 (5.8)36 (7.7)23 (4.2)
    Nap frequency, n (%)

    		<1 per week581 (57.1)267 (56.8)314 (57.4).03

    		1 per week165 (16.2)69 (14.7)96 (17.6)

    		2-3 per week176 (17.3)79 (16.8)97 (17.7)

    		4-6 per week57 (5.6)28 (6.0)29 (5.3)

    		Daily38 (3.7)27 (5.7)11 (2.0)
    Number of health apps used in past month, n (%)

    		0212 (20.8)127 (27.0)85 (15.5)<.001

    		1-5705 (69.3)299 (63.6)406 (74.2)

    		>5100 (9.8)44 (9.4)56 (10.2)
    App use reasons, n (%)

    		Track how much exercise I get665 (65.4)269 (57.2)396 (72.4)<.001

    		Track what I eat/improve what I eat531 (52.2)212 (45.1)319 (58.3)<.001

    		Weight loss476 (46.8)206 (43.8)270 (49.4).08

    		Track a health measure203 (20.0)86 (18.3)117 (21.4).22

    		Track how much sleep I get346 (34.0)132 (28.1)214 (39.1)<.001

    aApp user is defined as downloading the app and recording at least one entry on at least one day.

    bThe P value for a group of variables is reported in the row of the first variable.

    cBMI is calculated as weight in kilograms divided by height in meters squared.

    Out of 1017 participants, 547 (53.79%) were app users (ie, downloaded the app and recorded at least one entry on at least one day). When examining app users by use category, 412 (75.3%), 274 (50.1%), and 139 (25.4%) were categorized as immediate, consistent, and sustained users, respectively. Of the sustained users, 116 (83.5%) used the app at least one day every month of the study, and 133 (95.7%) used the app at least one day for 5 out of the 6 months. In comparison to non–app users (471/1017, 46.31%), app users were younger (mean 49.3 vs 53.3 years; P<.001), more likely to be college graduates (78.2% vs 68.3%; P<.001), had greater annual income (>US $50,000: 398/547, 72.8% vs 300/470, 63.8%; P=.02), and were less likely to have children younger than 18 years old (21.8% vs 27.4%; P=.04). There were no differences between app users and nonusers regarding weight, height, mean BMI, and BMI category. App users were less likely to drink sugar-sweetened beverages (mean sugar tsp equivalent: 0.7 vs 1.0; P=.004), reported better sleep quality (fairly good or very good: 386/547, 70.6% vs 298/470, 63.4%; P=.02), and were less likely to take daily naps (2.0% vs 5.7%; P=.03). They were also more likely to use health apps overall (462/547, 84.5% vs 343/470, 73.0%; P<.001), and to use them for the purpose of tracking exercise (396/547, 72.4% vs 269/470, 57.2%), eating (319/547, 58.3% vs 212/470, 45.1%), and sleep (214/547, 39.1% vs 132/470, 28.1%; P<.001 for all).

    The median amount of app use was 28 (IQR 7-75) days over the 6-month study, 20 (IQR 7-35) days during the targeted 63 days of the study, and 6 (IQR 0-41) days during the 117 nontargeted days of the study. Table 2 describes app use by study month. During study month 1, the vast majority of app users (92.3%) used the app for 2 or more days and 76.2% used it for 7 or more days. Usage decreased over time in the cohort study. By month 6, 27.1% of app users used the app for 2 or more days and 20.1% used it 7 or more days.

    Table 2. Monthly Daily24 app use during the 6-month cohort study by users who completed at least one day of app use.
    MonthaParticipants who used the app (n=547), n (%)

    		Used ≥2 daysUsed ≥7 days
    Month 1505 (92.3)417 (76.2)
    Month 2269 (49.2)214 (39.1)
    Month 3 213 (38.9)166 (30.3)
    Month 4183 (33.5)138 (25.2)
    Month 5b (n=536)157 (29.3)133 (24.8)
    Month 6b (n=527)143 (27.1)106 (20.1)

    aA study month is defined as 4 weeks (28 days). To enable all study months to begin on a Monday, the time between the end of POWER 28 and start of month 2 ranged from 15 to 21 days. 

    bDue to late registration, some participants were not able to reach months 5 and 6 of the study.

    Predictors of Usage of the Daily24 App

    Table 3 shows the multivariable regression model for app use versus nonuse. Younger age, White (vs non-White) race, greater education, higher household income, not having children less than 18 years of age, and having used 1 to 5 apps in the past were statistically significantly associated with app use (vs non–app use). Black participants were one-third less likely to use the app than White participants, whereas those with greater than a college education and a higher income (≥US $75,000 vs <US $35,000) were statistically significantly more likely to use the app. Those with children under the age of 18 years were 45% less likely to use the app, and those who had used 1 to 5 apps in the past month were 70% more likely to use the app compared to those who had not used apps in the past month.

    Table 4 shows multivariable regression models for immediate, consistent, and sustained use. Older age and lower BMI were statistically significantly associated with increased immediate, consistent, and sustained app use. Having children less than 18 years old was statistically significantly associated with decreased immediate use, and better sleep quality was associated with increased immediate and consistent app use.

    Table 3. Multivariable regression models for Daily24 app use versus nonuse.
    Risk factorsModel 1aModel 2b

    		ORc (95% CI)P valueOR (95% CI)P value
    Demographic risk factors

    		Age, per 10-year increase0.77 (0.70-0.85)<.0010.78 (0.71-0.86)<.001

    		Gender


    		MaleRefd (1)
    		Ref (1)


    		Female1.32 (0.96-1.81).091.22 (0.88-1.69).23

    		Race


    		WhiteRef (1)
    		Ref (1)


    		Black0.66 (0.45-0.96).030.67 (0.46-0.98).04


    		Other0.80 (0.49-1.31).380.82 (0.50-1.34).43

    		Educational level


    		<CollegeRef (1)
    		Ref (1)


    		≥College1.39 (1.03-1.89).031.36 (1.00-1.86).05

    		Household income (US $)


    		<35,000Ref (1)
    		Ref (1)


    		35,000 to <50,0001.58 (0.91-2.72).101.40 (0.80-2.44).24


    		50,000 to <75,0002.01 (1.20-3.38).011.82 (1.07-3.07).03


    		≥75,0002.30 (1.47-3.61)<.0012.00 (1.26-3.17).003

    		Any child <18 years old


    		NoRef (1)
    		Ref (1)


    		Yes0.53 (0.39-0.73)<.0010.55 (0.40-0.75)<.001
    Behavioral risk factors

    		Physical activity


    		Low or medium—e—Ref (1)


    		High——0.93 (0.61-1.43).75

    		Fruit and vegetable cups, per 1-cup increase——1.04 (0.95-1.14).41

    		Sleep quality


    		Very good or fairly good——Ref (1)


    		Very bad or fairly bad——0.79 (0.59-1.04).09

    		Number of health apps used in past month


    		0——Ref (1)


    		1-5——1.70 (1.22-2.37).002


    		>5——1.40 (0.84-2.35).20

    		BMIf, per 1-unit increase——1.00 (0.98-1.02).99

    aModel 1 was adjusted for age, sex, race, education, household income, and having children younger than 18 years old.

    bModel 2 included model 1 parameters and was adjusted for physical activity, fruit and vegetable cups, sleep quality, and BMI.

    cOR: odds ratio.

    dRef: reference.

    eNot calculated since these parameters were not included in model 1.

    fBMI is calculated as weight in kilograms divided by height in meters squared.

    Table 4. Multivariable regression modelsa for immediate, consistent, and sustained Daily24 app use (n=547).
    Risk factorsImmediate use: using app for ≥7 days during POWER 28 (n=412)Consistent use: using app for ≥28 days for 6 months (n=274)Sustained use: using app for ≥2 days during POWER week 5 (n=139)

    		ORb (95% CI)P valueOR (95% CI)P valueOR (95% CI)P value
    Demographic risk factors

    		Age, per 10-year increase1.28 (1.09-1.50).0031.40 (1.22-1.61)<.0011.54 (1.31-1.82)<.001

    		Gender


    		MaleRefc (1)
    		Ref (1)
    		Ref (1)


    		Female0.69 (0.38-1.26).230.60 (0.37-0.98).040.74 (0.44-1.25).26

    		Race


    		WhiteRef (1)
    		Ref (1)
    		Ref (1)


    		Black0.70 (0.38-1.30).260.86 (0.48-1.52).600.92 (0.46-1.84).82


    		Other0.79 (0.38-1.66).530.67 (0.33-1.36).271.03 (0.45-2.38).94

    		Educational level


    		<CollegeRef (1)
    		Ref (1)
    		Ref (1)


    		≥College1.00 (0.59-1.71).990.99 (0.61-1.59).961.46 (0.82-2.60).20

    		Household income (US $)


    		<35,000Ref (1)
    		Ref (1)
    		Ref (1)


    		35,000 to <50,0000.94 (0.38-2.29).891.05 (0.46-2.42).910.86 (0.31-2.37).77


    		50,000 to <75,0000.83 (0.36-1.92).661.18 (0.54-2.56).680.87 (0.35-2.16).76


    		≥75,0001.00 (0.47-2.14).990.76 (0.38-1.53).440.62 (0.27-1.41).25

    		Any child <18 years old


    		NoRef (1)
    		Ref (1)
    		Ref (1)


    		Yes0.56 (0.34-0.91).020.68 (0.43-1.07).100.70 (0.38-1.28).24
    Behavioral risk factors

    		Physical activity


    		Low or mediumRef (1)
    		Ref (1)
    		Ref (1)


    		High0.68 (0.35-1.32).251.01 (0.55-1.83).981.30 (0.68-2.51).43

    		Fruit and vegetable cups, per 1-cup increase0.88 (0.75-1.03).101.03 (0.90-1.19).640.98 (0.84-1.15).82

    		Sleep quality


    		Very good or fairly goodRef (1)
    		Ref (1)
    		Ref (1)


    		Very bad of fairly bad0.59 (0.38-0.93).020.63 (0.42-0.95).030.74 (0.45-1.21).23

    		Number of health apps used in past month


    		0Ref (1)
    		Ref (1)
    		Ref (1)


    		1-50.86 (0.45-1.63).641.12 (0.66-1.92).671.46 (0.81-2.62).20


    		>51.03 (0.42-2.51).951.04 (0.49-2.24).910.60 (0.21-1.70).34

    		BMId, per 1-unit increase0.96 (0.94-0.99).010.95 (0.93-0.98).0010.95 (0.92-0.99) .01

    aThe model was adjusted for age, sex, race, education, household income, having children younger than 18 years old, physical activity, fruit and vegetable cups, sleep quality, and BMI.

    bOR: odds ratio.

    cRef: reference.

    dBMI is calculated as weight in kilograms divided by height in meters squared.

    Survey Completion and Retention in the EHR-Based Cohort Study

    Out of 1017 enrolled participants, 328 (32.25%) completed the 4-month follow-up surveys within 72 hours of receiving the link. Of the remaining 689 participants (67.75%), study staff were able to reach out to 610 participants (88.5%) through personalized emails, text messages, and US $100 raffle invitations delivered up to one week prior to study completion. Of the 610 contacted participants, 113 (18.5%) completed their surveys after one contact, 56 (9.2%) after two contacts, and 35 (5.7%) after three contacts, increasing the overall number of 4-month survey completers to 532 (52.31%).


    Principal Findings

    EHRs and patient portals are readily available through most health care systems, and use of mHealth apps is fairly ubiquitous [16,44]. This study reports on EHR-based recruitment of adults from three health systems to use the Daily24 mobile app to record daily timing of meals, snacks, and sleep for 6 months. We emailed research invitations to over 70,000 potentially eligible participants identified through the EHR using efficient identification (ie, computable phenotype) and messaging methods (ie, emails sent directly through the EHR patient portal or to personal email addresses). A total of 1.4% of participants completed e-consent forms and baseline questionnaires in a period of 6 months, a yield that is slightly lower than reports for other EHR-based recruitment methods [32-34]. In a 2019 single-institution study that included 13 separate EHR-based recruitment strategies using the patient portal recruitment service, the average response rate for patient portal messages was 2.9% [32]. Our lower yield might be explained by the study’s expectation to download and actively use an app for 6 months or have no guaranteed compensation be provided for participation [45] (ie, raffles of gift cards). Patients may also be more likely to respond to mHealth research with a behavioral intervention [46] or to disease-related versus wellness-related research [32,45]. In the above study by Miller and colleagues [32], recruitment response rates were higher (3.4%) among condition-specific studies (ie, those with a more inclusive comprehensive phenotype) versus general health studies (1.4%). The latter response rate was identical to this study’s recruitment yield, which was also not specific to a health condition. Furthermore, while our app included gaming elements (eg, badges and a leaderboard) [47,48] to increase data entry, we intentionally did not include behavioral techniques (eg, goal setting and personalized behavioral prompts) aimed at behavior change, given the study’s primary goal to naturalistically examine the relationship between timing of eating and sleep and weight and medical conditions (findings forthcoming).

    Once enrolled, 54% of participants who downloaded the app entered timing of eating or sleep data on at least one day. While the frequency criteria to classify someone as an app user in this study was fairly low (ie, at least one completed day), other studies have used a similarly low frequency to define usage [49]; however, comparisons between studies can be challenging due to disparate study designs and modes of interacting with apps (ie, passive vs active data collection) [50]. For example, in the Asthma Mobile Health Study (AMHS), 85.21% (6470/7593) of enrolled participants (ie, downloaded an asthma health app, e-consented, and verified email) were considered baseline users (ie, at least one in-app survey entry). However, enrollment occurred after the app was already downloaded, and individuals who downloaded the app (N=40,683 in the United States over 6 months) were recruited through a large media blitz versus academic recruitment [49]. Eligibility was also based on having a medical condition (ie, disease related), and the app included behavioral components (eg, goal setting).

    Although criteria for defining usage categories differ across studies, our immediate (75%; ≥7 days in the first month) and consistent (50%; ≥28 days over 6 months) rates were higher than the “robust” cohort rates (30%; 5 or more surveys over 6 months) reported in the AMHS; in the case of sustained users (25%; ≥2 days during month 6), our rates were fairly comparable to those in the AMHS [49]. We attribute being able to initially engage three-quarters of our app users, and to retain a quarter of our users, to the food and sleep wheels in Daily24 being fast and easy to use, whereas other apps may include more survey items or require more detailed dietary intake entry [31,51]. Future iterations of the app should employ evidence-based strategies and features for increasing engagement (eg, push notifications with tailored health messages) [52-54].

    This study’s usage data provides important information about predictors of health app use to guide the design of future observational studies using apps. Our finding that those who were younger, more formally educated, and wealthier were more likely to be app users is consistent with past research [16,49]. This study also found that White participants were more likely to be app users, a finding that is consistent with some research [55]. However, that finding is not consistent with a cross-sectional survey study of 1604 mobile phone users in the United States [16], which found that being Latino or Hispanic (P<.05) or African American (P=.07, trend) were related to a greater likelihood to download a health app. Inconsistencies in findings may be related to different assessment methods (ie, actual app usage vs self-reported use), recruitment methods (ie, national survey vs regional EHR recruitment), and racial and ethnic distribution in recruitment regions [16,56]. Not having children younger than 18 years of age was also associated with app use. While this is perhaps a correlate of being younger, it is also an intuitive finding that those with children may have less time for mHealth app use, supporting the well-documented importance of ease and efficiency of data entry in mHealth apps [16]. While younger age was associated with app use overall, being older was associated with early, consistent, and sustained use. The AMHS study similarly found that among robust users, increasing age was significantly associated with a greater likelihood to use the asthma health app daily [49]. An adherence and retention study of a web-based alcohol intervention also found that being older and not having children predicted a greater likelihood of logging in [57]. We also found that having a lower BMI was associated with early, consistent, and sustained use. Past research has found that having a BMI in the obese range is associated with greater health app use [16], influencing our hypothesis that those with higher BMIs would be more motivated to download and use a health app. While we did not find a significant association between weight status and app use overall, we did find that those with lower BMIs were more likely to use health apps across time. Given this observational study design, we identified an association between BMI and health app use (ie, those with a lower BMI were more likely to engage in sustained tracking and health monitoring), but we do not know causality or temporality. Future research exploring a causal relationship is needed to determine if apps targeting timing of eating and sleep may have an effect on behaviors that influence weight [25,56].

    Limitations

    There are several limitations of this study. First, this was an observational cohort study and was not designed with a comparison group to assess differences in app use among participants instructed to log for 6 months without additional guidance on targeted tracking days, compared to our approach of emphasizing tracking on POWER 28 and POWER week days. In designing the study, to optimize longitudinal tracking, we decided to preidentify targeted days to decrease participant burden and, more importantly, to increase the likelihood that we would collect data on some days across each of the 6 months rather than risk the typical pattern of heavier use up front followed by drop-off [18,49]. This approach appeared to be effective. Although we did observe drop-off across each study month, with the biggest decline being from months 1 and 2, about one-quarter of the participants were still using the app during month 6, and those who were using the app during month 6 were using it in the identified POWER week. However, without a two-arm study design, we cannot fully conclude that this was the ideal approach. Second, although we designed badges and a leaderboard to create gaming elements and increase motivation [47,48], we are unable to ascertain if those who earned badges were more motivated individuals in general or were motivated by the badges. Badges were earned based on various categories of usage (eg, first log-in, track 7 days in a row, and 4 days of your POWER week) and were automatically entered into our raffle (ie, participants did not have to enter their badges into the raffle themselves); thus, it is challenging to know whether badges and the resulting raffle were an effective gamification approach. Third, we do not have detailed information on the reasons that a little less than half of the participants did not go on to download the app. With our app being designed by researchers rather than more highly funded industry, we suspect that the onboarding process may have had some cumbersome features. The biggest obstacles may have been problems with the two-factor authentication process, which required participants to receive an SMS code on their device and correctly enter it to verify their identity. Additionally, many people forgot, misplaced, or mis-entered the password they chose when registering for the app and were without an automated password reset option. Although we had research staff available for tech support, it was available only during work hours and via phone or email. Fourth, our sample was largely comprised of White participants, more formally educated participants, and those of middle- to upper-socioeconomic status; thus, the generalizability to other racial, ethnic, and socioeconomic groups is limited. Future research involving EHR-based recruitment independent of technology use might consider partnering with communities from racialized and lower-socioeconomic subgroups to understand how recruitment efforts and health apps can be adapted to improve their impact for marginalized communities. Finally, while our recruitment methods were efficient in terms of participant identification, messaging, and enrollment, we are unable to comment on the cost-effectiveness of EHR enrollment. Each of these health systems have made significant investments into building and maintaining their EHRs and infrastructure to enable these recruitment methods for research purposes. In addition, for this study, we leveraged existing health information technology infrastructure from the PaTH network [30], which enabled efficiency from both a time and resource perspective. However, for this methodology to be used more broadly in a variety of settings, greater institutional and community partnerships and resources are needed.

    Conclusions

    Health apps aimed at weight loss and related behaviors are among the most highly used mHealth apps [22]. Time-restricted feeding is a novel and promising approach for obesity and related disease management; however, it is largely untested in humans, to a great extent due to the challenges of helping individuals modify their behavior to a shorter window of eating [15,58,59]. This report is a first step in describing efficient EHR recruitment of patients from three large health institutions and the use of an mHealth app to enter information about timing of eating and sleep patterns. Next steps include incorporating behavioral techniques into the app, potentially with health coaching, to assist individuals achieve greater alignment with their circadian rhythms and to determine whether this is a feasible and effective weight loss intervention.

    Acknowledgments

    We would like to thank Jonathan Martinez for his role in recruitment and retention and Hao Da for her role in data management. We would also like to acknowledge Mengru Liao for her design of the Daily24 web registration flow and contribution to the design of the REDCap e-consent flow. This work was funded by the American Heart Association. The authors acknowledge assistance for clinical data coordination and retrieval from the Johns Hopkins School of Medicine’s Core for Clinical Research Data Acquisition, supported, in part, by the Johns Hopkins Institute for Clinical and Translational Research (UL1TR001079).

    Conflicts of Interest

    TBW founded DaiWare, but has no current commercial projects associated with DaiWare. JMC has served on a Scientific Advisory Board for Novo Nordisk and Boehringer Ingelheim.

    ‎
    Multimedia Appendix 1

    Sample recruitment message.

    PDF File (Adobe PDF File), 77 KB
    1. Ng M, Fleming T, Robinson M, Thomson B, Graetz N, Margono C, et al. Global, regional, and national prevalence of overweight and obesity in children and adults during 1980-2013: A systematic analysis for the Global Burden of Disease Study 2013. Lancet 2014 Aug 30;384(9945):766-781 [FREE Full text] [CrossRef] [Medline]
    2. Khan SS, Ning H, Wilkins JT, Allen N, Carnethon M, Berry JD, et al. Association of body mass index with lifetime risk of cardiovascular disease and compression of morbidity. JAMA Cardiol 2018 Apr 01;3(4):280-287 [FREE Full text] [CrossRef] [Medline]
    3. Williams EP, Mesidor M, Winters K, Dubbert PM, Wyatt SB. Overweight and obesity: Prevalence, consequences, and causes of a growing public health problem. Curr Obes Rep 2015 Sep;4(3):363-370. [CrossRef] [Medline]
    4. Arroyo-Johnson C, Mincey KD. Obesity epidemiology worldwide. Gastroenterol Clin North Am 2016 Dec;45(4):571-579 [FREE Full text] [CrossRef] [Medline]
    5. Dwivedi AK, Dubey P, Cistola DP, Reddy SY. Association between obesity and cardiovascular outcomes: Updated evidence from meta-analysis studies. Curr Cardiol Rep 2020 Mar 12;22(4):25. [CrossRef] [Medline]
    6. Ramage S, Farmer A, Eccles KA, McCargar L. Healthy strategies for successful weight loss and weight maintenance: A systematic review. Appl Physiol Nutr Metab 2014 Jan;39(1):1-20 [FREE Full text] [CrossRef] [Medline]
    7. Cleven L, Krell-Roesch J, Nigg CR, Woll A. The association between physical activity with incident obesity, coronary heart disease, diabetes and hypertension in adults: A systematic review of longitudinal studies published after 2012. BMC Public Health 2020 May 19;20(1):726 [FREE Full text] [CrossRef] [Medline]
    8. Wadden TA, Tronieri JS, Butryn ML. Lifestyle modification approaches for the treatment of obesity in adults. Am Psychol 2020;75(2):235-251 [FREE Full text] [CrossRef] [Medline]
    9. McHill AW, Wright KP. Role of sleep and circadian disruption on energy expenditure and in metabolic predisposition to human obesity and metabolic disease. Obes Rev 2017 Feb;18 Suppl 1:15-24. [CrossRef] [Medline]
    10. Nedeltcheva A, Scheer FAJL. Metabolic effects of sleep disruption, links to obesity and diabetes. Curr Opin Endocrinol Diabetes Obes 2014 Aug;21(4):293-298 [FREE Full text] [CrossRef] [Medline]
    11. Dong TA, Sandesara PB, Dhindsa DS, Mehta A, Arneson LC, Dollar AL, et al. Intermittent fasting: A heart healthy dietary pattern? Am J Med 2020 Aug;133(8):901-907 [FREE Full text] [CrossRef] [Medline]
    12. LeBlanc ES, Patnode CD, Webber EM, Redmond N, Rushkin M, O'Connor EA. Behavioral and pharmacotherapy weight loss interventions to prevent obesity-related morbidity and mortality in adults: Updated evidence report and systematic review for the US Preventive Services Task Force. JAMA 2018 Sep 18;320(11):1172-1191. [CrossRef] [Medline]
    13. Hall KD, Kahan S. Maintenance of lost weight and long-term management of obesity. Med Clin North Am 2018 Jan;102(1):183-197 [FREE Full text] [CrossRef] [Medline]
    14. MacLean PS, Wing RR, Davidson T, Epstein L, Goodpaster B, Hall KD, et al. NIH working group report: Innovative research to improve maintenance of weight loss. Obesity (Silver Spring) 2015 Jan;23(1):7-15 [FREE Full text] [CrossRef] [Medline]
    15. Panda S. The arrival of circadian medicine. Nat Rev Endocrinol 2019 Feb;15(2):67-69. [CrossRef] [Medline]
    16. Krebs P, Duncan DT. Health app use among US mobile phone owners: A national survey. JMIR Mhealth Uhealth 2015 Nov 04;3(4):e101 [FREE Full text] [CrossRef] [Medline]
    17. Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, et al. Mobile health technology evaluation: The mHealth evidence workshop. Am J Prev Med 2013 Aug;45(2):228-236 [FREE Full text] [CrossRef] [Medline]
    18. Dorsey ER, Chan Y, McConnell MV, Shaw SY, Trister AD, Friend SH. The use of smartphones for health research. Acad Med 2017;92(2):157-160. [CrossRef]
    19. Bradway M, Gabarron E, Johansen M, Zanaboni P, Jardim P, Joakimsen R, et al. Methods and measures used to evaluate patient-operated mobile health interventions: Scoping literature review. JMIR Mhealth Uhealth 2020 Apr 30;8(4):e16814 [FREE Full text] [CrossRef] [Medline]
    20. Feldman DI, Theodore Robison W, Pacor JM, Caddell LC, Feldman EB, Deitz RL, et al. Harnessing mHealth technologies to increase physical activity and prevent cardiovascular disease. Clin Cardiol 2018 Jul;41(7):985-991 [FREE Full text] [CrossRef] [Medline]
    21. Zhao J, Freeman B, Li M. Can mobile phone apps influence people's health behavior change? An evidence review. J Med Internet Res 2016 Oct 31;18(11):e287 [FREE Full text] [CrossRef] [Medline]
    22. Milne-Ives M, Lam C, De Cock C, Van Velthoven MH, Meinert E. Mobile apps for health behavior change in physical activity, diet, drug and alcohol use, and mental health: Systematic review. JMIR Mhealth Uhealth 2020 Mar 18;8(3):e17046 [FREE Full text] [CrossRef] [Medline]
    23. Park S, Hwang J, Choi Y. Effect of mobile health on obese adults: A systematic review and meta-analysis. Healthc Inform Res 2019 Jan;25(1):12-26 [FREE Full text] [CrossRef] [Medline]
    24. Wang Y, Min J, Khuri J, Xue H, Xie B, Kaminsky LA, et al. Effectiveness of mobile health interventions on diabetes and obesity treatment and management: Systematic review of systematic reviews. JMIR Mhealth Uhealth 2020 Apr 28;8(4):e15400 [FREE Full text] [CrossRef] [Medline]
    25. Ghelani DP, Moran LJ, Johnson C, Mousa A, Naderpoor N. Mobile apps for weight management: A review of the latest evidence to inform practice. Front Endocrinol (Lausanne) 2020;11:412 [FREE Full text] [CrossRef] [Medline]
    26. Kearney A, Daykin A, Shaw ARG, Lane AJ, Blazeby JM, Clarke M, et al. Identifying research priorities for effective retention strategies in clinical trials. Trials 2017 Aug 31;18(1):406 [FREE Full text] [CrossRef] [Medline]
    27. Brueton V, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev 2013 Dec 03(12):MR000032 [FREE Full text] [CrossRef] [Medline]
    28. Cowie MR, Blomster JI, Curtis LH, Duclaux S, Ford I, Fritz F, et al. Electronic health records to facilitate clinical research. Clin Res Cardiol 2017 Jan;106(1):1-9 [FREE Full text] [CrossRef] [Medline]
    29. Lai YS, Afseth JD. A review of the impact of utilising electronic medical records for clinical research recruitment. Clin Trials 2019 Apr;16(2):194-203. [CrossRef] [Medline]
    30. Bennett WL, Bramante CT, Rothenberger SD, Kraschnewski JL, Herring SJ, Lent MR, et al. Patient recruitment into a multicenter clinical cohort linking electronic health records from 5 health systems: Cross-sectional analysis. J Med Internet Res 2021 May 27;23(5):e24003 [FREE Full text] [CrossRef] [Medline]
    31. Woolf TB, Goheer A, Holzhauer K, Martinez J, Coughlin JW, Martin L, et al. Development of a mobile app for ecological momentary assessment of circadian data: Design considerations and usability testing. JMIR Form Res 2021 Jul 23;5(7):e26297 [FREE Full text] [CrossRef] [Medline]
    32. Miller H, Gleason K, Juraschek S, Plante T, Lewis-Land C, Woods B, et al. Electronic medical record-based cohort selection and direct-to-patient, targeted recruitment: Early efficacy and lessons learned. J Am Med Inform Assoc 2019 Nov 01;26(11):1209-1217 [FREE Full text] [CrossRef] [Medline]
    33. Pfaff E, Lee A, Bradford R, Pae J, Potter C, Blue P, et al. Recruiting for a pragmatic trial using the electronic health record and patient portal: Successes and lessons learned. J Am Med Inform Assoc 2019 Jan 01;26(1):44-49 [FREE Full text] [CrossRef] [Medline]
    34. Gleason KT, Ford DE, Gumas D, Woods B, Appel L, Murray P, et al. Development and preliminary evaluation of a patient portal messaging for research recruitment service. J Clin Transl Sci 2018 Feb 25;2(1):53-56 [FREE Full text] [CrossRef] [Medline]
    35. Fleurence RL, Curtis LH, Califf RM, Platt R, Selby JV, Brown JS. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc 2014;21(4):578-582 [FREE Full text] [CrossRef] [Medline]
    36. Forrest CB, McTigue KM, Hernandez AF, Cohen LW, Cruz H, Haynes K, et al. PCORnet® 2020: Current state, accomplishments, and future directions. J Clin Epidemiol 2021 Jan;129:60-67 [FREE Full text] [CrossRef] [Medline]
    37. Amin W, Tsui F, Borromeo C, Chuang CH, Espino JU, Ford D, PaTH network team. PaTH: Towards a learning health system in the Mid-Atlantic region. J Am Med Inform Assoc 2014;21(4):633-636 [FREE Full text] [CrossRef] [Medline]
    38. Pathak J, Kho AN, Denny JC. Electronic health records-driven phenotyping: Challenges, recent advances, and perspectives. J Am Med Inform Assoc 2013 Dec;20(e2):e206-e211 [FREE Full text] [CrossRef] [Medline]
    39. Goheer A, Holzhauer K, Martinez J, Woolf T, Coughlin JW, Martin L, et al. What influences the "when" of eating and sleeping? A qualitative interview study. Appetite 2021 Jan 01;156:104980. [CrossRef] [Medline]
    40. Greenwood S, Perrin A, Duggan M. Social Media Update 2016. Washington, DC: Pew Research Center; 2016 Nov 11.   URL: https://www.pewresearch.org/internet/2016/11/11/social-media-update-2016/ [accessed 2022-05-19]
    41. Thompson FE, Midthune D, Kahle L, Dodd KW. Development and evaluation of the National Cancer Institute's Dietary Screener Questionnaire scoring algorithms. J Nutr 2017 Jun;147(6):1226-1233 [FREE Full text] [CrossRef] [Medline]
    42. Craig CL, Marshall AL, Sjöström M, Bauman AE, Booth ML, Ainsworth BE, et al. International physical activity questionnaire: 12-country reliability and validity. Med Sci Sports Exerc 2003 Aug;35(8):1381-1395. [CrossRef] [Medline]
    43. Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psychiatry Res 1989 May;28(2):193-213. [CrossRef]
    44. Office of the National Coordinator for Health Information Technology. Percent of hospitals, by type, that possess certified health IT. Health IT quick-stat #52. HealthIT.gov. 2018 Sep.   URL: https://www.healthit.gov/data/quickstats/percent-hospitals-type-possess-certified-health-it [accessed 2022-05-19]
    45. Pratap A, Neto EC, Snyder P, Stepnowsky C, Elhadad N, Grant D, et al. Indicators of retention in remote digital health studies: A cross-study evaluation of 100,000 participants. NPJ Digit Med 2020;3:21 [FREE Full text] [CrossRef] [Medline]
    46. Lee J, Choi M, Lee SA, Jiang N. Effective behavioral intervention strategies using mobile health applications for chronic disease management: A systematic review. BMC Med Inform Decis Mak 2018 Feb 20;18(1):12 [FREE Full text] [CrossRef] [Medline]
    47. Edwards EA, Lumsden J, Rivas C, Steed L, Edwards LA, Thiyagarajan A, et al. Gamification for health promotion: Systematic review of behaviour change techniques in smartphone apps. BMJ Open 2016 Oct 04;6(10):e012447 [FREE Full text] [CrossRef] [Medline]
    48. King D, Greaves F, Exeter C, Darzi A. 'Gamification': Influencing health behaviours with games. J R Soc Med 2013 Mar;106(3):76-78 [FREE Full text] [CrossRef] [Medline]
    49. Chan YY, Wang P, Rogers L, Tignor N, Zweig M, Hershman SG, et al. The Asthma Mobile Health Study, a large-scale clinical observational study using ResearchKit. Nat Biotechnol 2017 Apr;35(4):354-362 [FREE Full text] [CrossRef] [Medline]
    50. Perez MV, Mahaffey KW, Hedlin H, Rumsfeld JS, Garcia A, Ferris T, et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. N Engl J Med 2019 Nov 14;381(20):1909-1917. [CrossRef]
    51. Chen J, Berkman W, Bardouh M, Ng CYK, Allman-Farinelli M. The use of a food logging app in the naturalistic setting fails to provide accurate measurements of nutrients and poses usability challenges. Nutrition 2019 Jan;57:208-216. [CrossRef] [Medline]
    52. Bidargaddi N, Almirall D, Murphy S, Nahum-Shani I, Kovalcik M, Pituch T, et al. To prompt or not to prompt? A microrandomized trial of time-varying push notifications to increase proximal engagement with a mobile health app. JMIR Mhealth Uhealth 2018 Nov 29;6(11):e10123 [FREE Full text] [CrossRef] [Medline]
    53. Szinay D, Jones A, Chadborn T, Brown J, Naughton F. Influences on the uptake of and engagement with health and well-being smartphone apps: Systematic review. J Med Internet Res 2020 May 29;22(5):e17572 [FREE Full text] [CrossRef] [Medline]
    54. Wei Y, Zheng P, Deng H, Wang X, Li X, Fu H. Design features for improving mobile health intervention user engagement: Systematic review and thematic analysis. J Med Internet Res 2020 Dec 09;22(12):e21687 [FREE Full text] [CrossRef] [Medline]
    55. Bender MS, Choi J, Arai S, Paul SM, Gonzalez P, Fukuoka Y. Digital technology ownership, usage, and factors predicting downloading health apps among Caucasian, Filipino, Korean, and Latino Americans: The digital link to health survey. JMIR Mhealth Uhealth 2014 Oct 22;2(4):e43 [FREE Full text] [CrossRef] [Medline]
    56. Robbins R, Krebs P, Jagannathan R, Jean-Louis G, Duncan DT. Health app use among us mobile phone users: Analysis of trends by chronic disease status. JMIR Mhealth Uhealth 2017 Dec 19;5(12):e197 [FREE Full text] [CrossRef] [Medline]
    57. Murray E, White IR, Varagunam M, Godfrey C, Khadjesari Z, McCambridge J. Attrition revisited: Adherence and retention in a web-based alcohol trial. J Med Internet Res 2013 Aug 30;15(8):e162 [FREE Full text] [CrossRef] [Medline]
    58. Jamshed H, Beyl R, Della Manna D, Yang E, Ravussin E, Peterson C. Early time-restricted feeding improves 24-hour glucose levels and affects markers of the circadian clock, aging, and autophagy in humans. Nutrients 2019 May 30;11(6):1234 [FREE Full text] [CrossRef] [Medline]
    59. Arble D, Bass J, Behn C, Butler M, Challet E, Czeisler C, et al. Impact of sleep and circadian disruption on energy balance and diabetes: A summary of workshop discussions. Sleep 2015 Dec 01;38(12):1849-1860 [FREE Full text] [CrossRef] [Medline]

    ‎
    AMHS: Asthma Mobile Health Study
    e-consent: electronic consent
    EHR: electronic health record
    IRB: Institutional Review Board
    mHealth: mobile health
    PCORnet: National Patient-Centered Research Network
    REDCap: Research Electronic Data Capture

    Edited by R Kukafka; submitted 15.10.21; peer-reviewed by B Schreiweis, Y Chu; comments to author 18.12.21; revised version received 01.03.22; accepted 21.03.22; published 10.06.22

    Copyright

    ©Janelle W Coughlin, Lindsay M Martin, Di Zhao, Attia Goheer, Thomas B Woolf, Katherine Holzhauer, Harold P Lehmann, Michelle R Lent, Kathleen M McTigue, Jeanne M Clark, Wendy L Bennett. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.06.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.