Review
Abstract
Background: Mobile health (mHealth) is widely used as an innovative approach to delivering physical activity (PA) programs. Users’ adherence to mHealth programs is important to ensure the effectiveness of mHealth-based programs.
Objective: Our primary aim was to review the literature on the methods used to assess adherence, factors that could affect users’ adherence, and the investigation of the association between adherence and health outcomes. Our secondary aim was to develop a framework to understand the role of adherence in influencing the effectiveness of mHealth PA programs.
Methods: MEDLINE, PsycINFO, EMBASE, and CINAHL databases were searched to identify studies that evaluated the use of mHealth to promote PA in adults aged ≥18 years. We used critical interpretive synthesis methods to summarize the data collected.
Results: In total, 54 papers were included in this review. We identified 31 specific adherence measurement methods, which were summarized into 8 indicators; these indicators were mapped to 4 dimensions: length, breadth, depth, and interaction. Users’ characteristics (5 factors), technology-related factors (12 factors), and contextual factors (1 factor) were reported to have impacts on adherence. The included studies reveal that adherence is significantly associated with intervention outcomes, including health behaviors, psychological indicators, and clinical indicators. A framework was developed based on these review findings.
Conclusions: This study developed an adherence framework linking together the adherence predictors, comprehensive adherence assessment, and clinical effectiveness. This framework could provide evidence for measuring adherence comprehensively and guide further studies on adherence to mHealth-based PA interventions. Future research should validate the utility of this proposed framework.
doi:10.2196/30817
Keywords
Introduction
Background
There is strong evidence that physical activity (PA) is associated with improvements in physical health, mental health, and well-being [
]. However, many people are at risk of inactivity and there is poor uptake of, and adherence to, PA interventions [ ]. Promoting PA and maintaining people’s adherence are crucial public health issues.Technological innovation (eg, mobile health [mHealth]) is developing rapidly and is being widely applied in the health care field [
]. mHealth mainly focuses on the delivery and monitoring of health care services [ ] and could also be an alternative approach to delivering PA interventions, overcoming the limitations of traditional PA approaches (eg, classes or workshops) [ ]. Compared with traditional approaches, mHealth can use vivid video and pictures and may be more attractive and acceptable [ ]. The use of mHealth can help deliver exercise programs to a wide audience at a low cost. In addition, mHealth technologies can provide timely feedback, reminders and support, continuous monitoring, and outcomes assessment [ , ].There is a common issue with innovative health technologies, including mHealth, which is users’ adherence to mHealth programs. For example, people who download an exercise app do not always use, or continue to use, the app. Research suggests that suboptimal exposure to the PA program lessens the effects of these interventions [
]. Measuring users’ adherence to mHealth and exploring the factors that could influence users’ adherence and the association between adherence and intervention outcomes are important to understand how PA and other outcomes can be improved.Although there are systematic reviews summarizing evidence on the adherence to technology-based interventions, previous reviews are rather generic, do not differentiate mHealth from other technology-based interventions, do not focus on PA, and do not address the issue that adherence to a PA intervention may differ from adherence to other interventions such as those for medications or therapy. For example, Donkin et al [
] summarized measurement methods for adherence to any e-therapies and evaluated the association of adherence with intervention outcomes. Perski et al [ ], focusing on digital behavior change interventions, developed a conceptual framework to explain the impacts of potential factors on people’s engagement with digital behavior change interventions. By contrast, other reviews may focus on PA but restrict themselves to specific digital technologies. For example, Attig et al [ ], focusing on wearable trackers for PA, summarized reported reasons for abandoning their use, such as usability issues and privacy concerns.mHealth is a commonly used innovative solution to deliver health interventions. The characteristics of instant access, portability, and direct feedback make mHealth different from other technologies such as desktop computers [
]. We believe that these characteristics could affect the adherence of users of mHealth-based interventions in a way that is different from adherence to other technologies. Therefore, it would be better to consider mHealth-based interventions more specifically rather than grouping them together with other, generic technologies. Given this, we consider that the measurement of users’ adherence to mHealth-based PA should not only reflect generic technologies’ features, where applicable, but also incorporate mHealth-specific factors. For example, the automatically recorded number of days when mHealth devices are worn and the automatically recorded daily amount of PA, such as step counts, are commonly used indicators in measuring users’ adherence to mHealth-based PA interventions [ ]. However, such indicators have not been well considered in existing reviews that cover information technologies in general. Overall, there is a lack of evidence on determinants of the adherence to mHealth-based PA and the influence of adherence on health outcomes; in addition, there seems to be no agreed measurement method of adherence to mHealth devices that aim to improve PA engagement.A scoping review of the literature was therefore carried out to explore how adherence to mHealth aiming at improving PA engagement is measured, to investigate which factors affect users’ adherence, and the association between adherence and intervention outcomes. A framework is needed to identify the association among factors, adherence measurement, and health outcomes. The framework can be used in future to guide the measurement of adherence to mHealth-based PA programs and facilitate further research on the effectiveness of mHealth-based programs.
Objectives
The aims of this study were to synthesize evidence about (1) how adherence to mHealth PA interventions has been measured in the literature, (2) the factors that influence the adherence, and (3) the association between adherence to mHealth PA interventions and health outcomes. An additional aim was to propose an operational concept of adherence to mHealth-based PA programs and a framework for identifying the links among the determinants of adherence, adherence measurements, and intervention outcomes.
Methods
The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines were followed for reporting this review [
]. Refer to for the PRISMA-ScR checklist. The review protocol was not registered on the web.Literature Search and Search Strategy
To identify relevant studies, the initial electronic searches were run in September 2020 in 4 databases: MEDLINE, PsycINFO, EMBASE, and CINAHL [
]. Update searches were performed in February 2022. Refer to for the search strategies. There were no restrictions on publication year, but searches were limited to English language publications.Eligibility Criteria
We considered a study eligible if it met the criteria presented in
. Following the World Health Organization’s definitions of mHealth [ ], we defined mHealth-based PA programs as interventions that use mobile devices to deliver PA. The devices could be smartphones, smartwatches, PDAs such as wristbands, and other wireless technologies. In addition, there were no restrictions on how adherence to mHealth interventions was defined and assessed in this review as long as the authors described the measurement or definition of adherence. Furthermore, this review used the PA definition provided by the World Health Organization [ ]: any bodily movement produced by skeletal muscles that requires energy expenditure.Items | Inclusion criteria | Exclusion criteria |
Types of study |
|
|
Types of participants |
|
|
Types of interventions |
|
|
Types of outcomes |
|
|
amHealth: mobile health.
bPA: physical activity.
Study Selection
We used EndNote (Clarivate Analytics) to manage records identified through the electronic searches. After removing duplicate records, we screened titles and abstracts at first to identify potentially eligible studies and then screened their full texts to include eligible studies. Given that this is a scoping review, only 1 reviewer (YY) was involved in this process. However, any problem was resolved by consulting another researcher (CT or EB).
Data Extraction
We used predefined data extraction forms and extracted the following details: (1) characteristics of the included studies: study design, study population, sample size, the description of the intervention, mHealth used, the goal of the intervention, control program, follow-up duration, and outcome measurement; and (2) factors that could influence adherence and the relationship between adherence and outcomes.
Data Synthesis
The critical interpretive synthesis approach was used to synthesize both qualitative and quantitative data [
]. Concepts identified in the full texts of included studies were labeled. The research questions were used as a top-down coding frame. We coded text fragments that were explicitly or implicitly related to any of the following three topics: (1) adherence measurement, (2) predictors of adherence, and (3) association between adherence and intervention outcomes. Synthetic constructs (ie, concepts that explain similar themes) were developed from the codes, and relationships between the synthetic constructs were specified.Framework Development Methods
On the basis of the aforementioned synthetic constructs, we developed an integrative adherence framework by following the theoretical causal pathway to map the synthetic constructs of the scoping review. This framework can show the links among the determinants affecting adherence, multidimensional adherence measurements, and the association between adherence and health outcomes.
Results
Search Results
The electronic database searches retrieved a total of 10,902 records. Title and abstract screening of these 10,902 records resulted in 150 (1.38%) requiring full-text inspection. Of these 150 papers, 54 (36%) were included in this review (
).Characteristics of the Included Studies
The characteristics of the included studies are summarized and reported in
(refer to [ - ] for full details). Most of the interventions were delivered through smartphone apps. The sample sizes of the included studies ranged from 10 to 16,948 (median 86). The average age of participants in the included studies ranged from 23.6 (SD 4.6) years to 73.2 (SD 7.3) years (median 51.7, SD 11.2).Summary characteristics of the included studies.
Types of studies
- Randomized controlled trial design (33 studies) [ , - , , , , , , , - , - , , , , , , - , ], including 9 pilot studies with small sample sizes [ , , , , , , , , ]
- Pre-post quasi-experimental studies with 1 arm (10 studies) [ , , , , , , , , , ]
- Observational study design (4 studies) [ , , , ]
- Subgroup analyses of the intervention groups of the randomized controlled trial (2 studies) [ , ]
- Nonrandomized controlled pilot trial (3 studies) [ , , ]
- Nonrandomized 2-arm, matched case-control trial (1 study) [ ]
- Cross-sectional web-based survey (1 study) [ ]
Types of participants
- General population (such as university students and staff or healthy adults; 10 studies) [ , , , , , , , , , ]
- People with specific characteristics (17 studies)
- Physically inactive community dwelling (4 studies) [ , , , ]
- People who were overweight (4 studies) [ , , , ]
- Older adults (4 studies) [ , , , ]
- Pregnant or postpartum women who were overweight (2 studies) [ , ]
- Shift workers (1 study) [ ]
- Mothers (1 study) [ ]
- Nurses (1 study) [ ]
- People with specific diseases (27 studies)
- Diabetes or at high risk of diabetes (9 studies) [ , , , , , , , , ]
- Cancer (5 studies) [ , , , , ]
- Cardiac event (4 studies) [ , , , ]
- Musculoskeletal conditions (3 studies) [ , , ]
- Pulmonary disease (2 studies) [ , ]
- Stroke (1 study) [ ]
- Parkinson disease (1 study) [ ]
- Excessive weight (in patients in primary care; 1 study) [ ]
- Patients awaiting surgery (1 study) [ ]
Types of mobile health
- Delivering interventions through a smartphone app (52 studies) [ - , - ]
- Using a wrist-worn activity tracker (1 study) [ ]
- Using a PDA (1 study) [ ]
Functions of mobile health
- Helping users to self-monitor and document their behavior (such as physical activity, diet, and weight; 49 studies) [ - , - , - , - , - ]
- Providing feedback, reminders, and social support (38 studies) [ , - , , , - , - , , , , , - , ]
- Analytic and assessment features and setting activity targets or plan (38 studies) [ , , - , , , , , - , , , - , - , - , ]
- Providing behavior change education and instruction (30 studies) [ , , , , , , - , , - , - , , , , , , , , , , , ]
- Game-based function (5 studies) [ , , , , ]
Types of outcomes
- The feasibility of the mobile health interventions
- Adherence (27 studies) [ , , , - , - , - , , - , - , , , - , ]
- Engagement (19 studies) [ , , , , , - , - , - , , ]
- Retention (10 studies) [ , , , , , , , , , ]
- Acceptability (9 studies) [ , , , , , , , , ]
- Usefulness or usability (7 studies) [ , , , , , , ]
- Satisfaction (5 studies) [ , , , , ]
- Recruitment (4 studies) [ , , , ]
- Uptake (4 studies) [ , , , ]
- Completion (3 studies) [ , , ]
- Safety (2 studies) [ , ]
- Program use (2 studies) [ , ]
- Adoption (1 study) [ ]
- Implementation (1 study) [ ]
- Maintenance (1 study) [ ]
- Fidelity (1 study) [ ]
- Change in health behavior
- Physical activity level (step count; 19 studies) [ , , , , , , , , , , , , , , , - ]
- Dieting (4 studies) [ , , , ]
- Clinical indicators
- Change in weight or BMI (13 studies) [ , , , , , , , , , , , , ]
- Physical function and walking or exercise capacity (10 studies) [ , , , , , , , , , ]
- Quality of life (9 studies) [ , , , , , , , , ]
- Glycated hemoglobin levels or fasting blood glucose (5 studies) [ , , , , ]
- Perceptions of treatment effectiveness (3 studies) [ , , ]
- Blood pressure (2 studies) [ , ]
- Oxygen uptake peak (1 study) [ ]
- Psychological indicators
- Physical activity motivation (4 studies) [ , , , ]
- Depression, anxiety stress, and mood (4 studies) [ , , , ]
- Self-efficacy (5 studies) [ , , , , ]
- Education about heart-related health (1 study) [ ]
- Adverse events (2 studies) [ , ]
- Cognitive performance (1 study) [ ]
- Disease knowledge (1 study) [ ]
Follow-up
- Range: 3 weeks to 24 months (median 12 weeks)
Summary of Adherence Measurement Methods
We identified 31 specific adherence measurement methods used (
). The top 3 most frequently used methods were manually entering self-monitored health behavior data into the device (17 studies), recording PA data (eg, step count) automatically recorded on the devices (10 studies), and recording the frequency of daily access to the app (8 studies).These 31 measurement methods were related to 8 measurement indicators that generally reflect 4 measurement dimensions: length, breadth, depth, and interaction (
). Among the 4 dimensions, the breadth dimension was the most frequently measured (35 studies). Of the 54 included studies, 31 (57%) measured adherence in only 1 dimension, 15 (28%) measured adherence in 2 dimensions, and 6 (11%) measured adherence in 3 dimensions, whereas 2 (4%) included all 4 dimensions.Dimensions and measurement indicators | Specific methods reported in the included studies | |
Length: the time users spend on the mHealtha devices (reported by 15 studies) | ||
Device use time and frequency |
| |
Duration of use until attrition |
| |
Breadth: the proportion of mHealth functions and features used out of the total available (reported by 35 studies) | ||
Device functions used |
| |
Completion of modules |
| |
Depth: how well the program has been used (reported by 13 studies) | ||
Meeting tasks or challenges |
| |
Behavior change (eg, physical activity level and diet habits) |
| |
Interaction: how users interact with the intervention programs (reported by 19 studies) | ||
Active interaction |
| |
Passive interaction |
|
amHealth: mobile health.
Factors That Affect Adherence to mHealth PA Programs
In the included studies, there are 3 factors affecting the adherence reported: user characteristics, technology-related factors, and contextual factors (
). For the 5 specific user characteristics, the included studies showed inconsistent evidence on the influence of age, education status, and weight on adherence, whereas the studies by Ryan et al [ ] and Guertler et al [ ] consistently suggested that men had higher adherence. The study by Edney et al [ ] reported that being overweight reduced adherence. The included studies consistently showed that almost all 12 technology-related factors (9 mHealth functions and 3 specific factors related to the experience of using mHealth devices) increased users’ adherence. Only 1 contextual factor was identified in this review: weekdays have higher adherence than weekends [ ].Association with physical activity app adherence | ||||
User characteristics | ||||
Age |
| |||
Sex |
| |||
Weight |
| |||
Education |
| |||
Baseline physical activity |
| |||
Technology-related factors | ||||
mHealtha functions |
| |||
User experience |
| |||
Contextual factors | Weekdays have higher adherence than weekends (1 study) [ | ]
amHealth: mobile health.
Association Between Adherence and Intervention Outcomes
Associations between adherence and intervention outcomes were reported in 13 studies. Higher adherence was reported as having associations with higher PA in 5 studies [
, , , , ] or with physical function in 1 study [ ], with greater weight loss in 6 studies [ , , , , , ], and with greater reductions in glycated hemoglobin levels in 2 studies [ , ]. Höchsmann et al [ ] reported that participants’ adherence was positively associated with intrinsic PA motivation inventory change scores.Proposed Framework of the Adherence to mHealth PA Interventions
The framework proposed by Perski et al [
] explicitly links potential influences on engagement and relationships between engagement and intervention effectiveness. We drew on the same structure for our adherence framework. We mapped our aforementioned review findings into a similar framework to illustrate interactions among the factors affecting adherence to mHealth PA interventions, the 4 dimensions of the adherence measurement, and intervention effectiveness ( ).Generally, most specific factors were related to aspects of user characteristics, the technology itself, and contexts that could increase users’ adherence. The adherence to mHealth could be reflected by indicators associated with 4 dimensions (length, breadth, depth, and interaction) and could be measured largely by automatically recorded data and self-reported methods. Higher adherence could improve intervention effectiveness on health behaviors as well as psychological and clinical outcomes.
Discussion
Principal Findings
In this scoping review, we synthesized evidence from 54 studies. The findings are as follows:
- Users’ personal characteristics (eg, sex and education status), mHealth devices’ features, and contextual factors were reported to increase users’ adherence to mHealth PA interventions.
- Adherence was reportedly reflected or measured in 4 dimensions and their corresponding indicators: breadth (device functions used and completion of modules), depth (meeting tasks or challenges and behavior change), length (device use time and length of use), and interaction (active or passive interaction with the program elements).
- Higher adherence was associated with better outcomes in terms of health behaviors (eg, increasing PA), PA motivation, and clinical outcomes (eg, decrease in glycated hemoglobin levels, blood pressure, and weight).
These findings were summarized to propose an initial comprehensive adherence framework based on a theoretical causal pathway with three parts: (1) the factors that can affect users’ adherence to mHealth PA interventions, (2) multidimensional adherence measurements, and (3) the association between adherence and intervention outcomes.
Adherence Measurement Issues
This scoping review identified 3 issues related to the measurement of adherence to mHealth PA programs in the literature. First, there is little consensus on how adherence should be defined and measured. In total, 31 specific adherence measurement methods were identified in the literature. The heterogeneity of the methods of assessing adherence makes comparison across studies difficult. Second, adherence measurement should be evidence based or theory informed; however, only the study by Adu et al [
] referred to a framework to measure adherence. Third, adherence to PA interventions is a complex definition rather than a single-dimension method [ ], but many of the studies (31/54, 57%) included in this review measured adherence using only 1 dimension.This review attempts to address the aforementioned issues through developing a new adherence conceptual framework based on the review findings. This framework, with 4 dimensions, is expected to inform the comprehensive measurement of users’ adherence to mHealth PA interventions. The four dimensions are as follows:
- The length dimension can reflect whether the users still use the mHealth devices and how much time they spend on the devices. This dimension should be considered essential in measuring adherence and can be reflected by the time to attrition, the frequency of access to the devices, and the duration of use.
- The breadth dimension can reflect how many device functions are involved in PA and how many modules are completed by participants. This dimension can help understand the usability of each function of an mHealth device and the engagement and involvement of users. It was the most frequent consideration in measuring adherence in the included studies. This dimension is especially worth considering when the mHealth program has multiple functions or a number of intervention modules.
- The depth dimension can assess how well users adhere to the task of an mHealth program. It can be measured by whether participants complete the program tasks or meet the PA target. For mHealth programs included in this review, tasks were usually related to behavior changes that could be considered to reflect the depth dimension. For example, the device recorded or self-reported PA time.
- The interaction dimension reflects how users interact with mHealth programs. In the included studies, the most frequently used methods to assess interaction included recording and counting the number of posts written or users’ responses and assessing the users’ access to educational materials.
Evidence on the topic of adherence or the related concept of engagement was included in 3 previous research studies [
, , ]; however, all 3 differ from this review in terms of the focus of the interventions. Perski et al [ ] developed a conceptual framework to highlight potential influences on engagement with a digital behavior change intervention and relationships between engagement and target behaviors. The authors’ framework specified potential direct and indirect influences on engagement but did not aim to show how the engagement could be measured. The framework we have developed in this review focuses on the topic of adherence measurement and also explores the factors that influence users’ adherence to mHealth. In addition, Perski et al [ ] considered the engagement of users with a broad range of digital behavior change interventions rather than focusing on mHealth-based PA interventions. Attig et al [ ] emphasized the exploration of factors that are related to the abandonment of an activity tracker rather than conceptualizing a framework for abandonment. Attig et al [ ] identified less-intensive device use, less device interaction, and amount of PA achieved as important factors that affect abandonment of activity tracker use. We agree with the importance of these factors and have included them in our proposed adherence framework. Couper et al [ ] analyzed data from a randomized controlled trial that aimed to evaluate the effectiveness of a web-based intervention in promoting dietary changes. They produced a 2D adherence measurement approach that consisted of breadth and depth. Breadth was defined as how widely users could access all available functions on the website, and depth was defined as how deeply users were engaged in the web-based material. Couper et al [ ] considered that the breadth-depth engagement led to intervention outcomes: users’ retention and behavior changes. However, we consider retention and behavior change as specific aspects of length and depth, respectively. This is because both retention and behavior change are only intermediate, moderating effects in terms of mHealth use patterns, rather than determining the influence of users’ adherence on clinical outcomes resulting from PA, such as specific balance and strength outcomes. Given this, we consider our evidence-based adherence framework to be specifically relevant to mHealth-delivered PA interventions.When choosing methods to measure adherence, researchers need to consider the following issues:
- The characteristics of the mHealth program (eg, whether the mHealth program has multiple functions). The length and breadth dimensions are generally applicable to most mHealth programs, whereas the interaction and depth dimensions are usually applicable only to mHealth programs with relevant functions.
- The purpose of the study. For example, if the study’s purpose is to assess the effectiveness of the mHealth intervention, users’ deep engagement with the mHealth intervention is needed, meaning that the interaction and depth dimensions should be considered.
The final issue regarding adherence measurement is how adherence data can be collected. Automatic recording and self-reporting adherence are 2 common approaches, but they both have strengths and limitations. Automatic recording can be objective but may not detect some specific types of activity (eg, stationary movement or upper or lower body movement) [
]. Self-reporting is easy to use and applicable to all types of PA, but the validity of this approach can be affected by social desirability bias [ ]. The ideal way to measure adherence to mHealth PA interventions may be a combination of the objective measurement using mHealth and self-reporting.Factors That Influence Users’ Adherence
Some findings of this review are inconsistent with previous work: this review suggests that male sex, older age, and secondary education could be predictors of higher adherence. However, in a systematic review investigating the predictors of adherence to web-based psychological interventions, Beatty and Binnion [
] suggest that female sex and older age predict higher adherence. Another review finds that younger age is associated with higher adherence to internet interventions [ ]. These disagreements may be related to the differences in the types of interventions and target populations considered.This review found 12 technology-related factors that could affect users’ adherence, including 9 related to mHealth functions and 3 related to users’ experience. This is consistent with another systematic review in terms of technology-related factors (eg, PA tracking, PA goal setting, and customization of exercise) [
]. In the study by Attig et al [ ], the reasons for abandonment included data inaccuracy, privacy concerns, discomfort, loss of motivation, and loss of tracking feasibility. This finding suggested that the mHealth devices themselves are the key to users’ adherence, and in designing mHealth, the target users could be involved to improve device functions and thus adherence.Adherence and Intervention Outcomes
As this review and the adherence framework demonstrates, adherence could affect intervention outcomes, including health behavior outcomes such as PA level, psychological indicators such as PA motivation, and clinical indicators such as weight loss and blood pressure. The findings suggest that further studies should explore the association between adherence and health outcomes from the aforementioned 3 perspectives.
Hawley-Hague et al [
] and Donkin et al [ ] suggest that different types of clinical outcomes could be predicted using different types of adherence indicators (eg, log-ins were related to outcomes in physical health interventions, whereas module completion was most related to outcomes in psychological health interventions). Understanding how adherence influences the effectiveness of interventions could be crucial to understanding how adherence should be defined. However, this review identified only a limited number of studies that evaluate the association between adherence to mHealth and health outcomes. Further study is needed to explore in depth the association between each adherence dimension and each health outcome dimension.Limitations of This Review
This review includes limitations. First, given that this is a scoping review, the searches for this review were limited to publications in English; hence, evidence published in other languages could have been missed. Second, the proposed adherence framework was based on the findings of the scoping review alone; therefore, it may not include all possible factors because of the limited evidence in the literature. Further research could improve this framework by considering evidence from other sources such as expert group meetings. Third, this framework explores the adherence predictors and the association between adherence and health outcomes. However, we did not further explore how these predictors affect each adherence dimension and the associations between each adherence dimension and intervention outcomes. This is because the included studies have limited evidence regarding these issues. Further studies are needed to explore the adherence to mHealth programs by each adherence dimension.
Conclusions
This review suggests that adherence can be measured using the dimensions of length, breadth, depth, and interaction; that users’ characteristics, technology-related factors, and contextual factors can affect adherence; and that adherence is significantly associated with outcomes in terms of health behaviors, psychology, and clinical measures. These findings inform the development of a framework, linking together the adherence predictors, comprehensive adherence assessment, and clinical effectiveness. The framework could facilitate a comprehensive measurement of adherence as well as guide mHealth device development and further studies on adherence to mHealth PA interventions. Further research is needed to validate this framework; for example, by considering evidence from other sources such as expert group meetings or using Delphi approaches.
Acknowledgments
This study was funded by CT’s National Institute for Health Research (NIHR) Senior Investigator Award (NIHR200299). CT is the Director and, at the time this paper was written, EB was a Research Fellow at the NIHR Older People and Frailty Policy Research Unit. CT leads the Healthy Ageing Research Group, and YY is employed by the NIHR Applied Research Collaboration Greater Manchester. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.
Conflicts of Interest
None declared.
PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.
DOCX File , 85 KB
Example of search strategies for the MEDLINE database.
DOC File , 41 KB
Characteristics of the included studies.
DOC File , 198 KBReferences
- Biddle SJ, Bennie JA, Bauman AE, Chau JY, Dunstan D, Owen N, et al. Too much sitting and all-cause mortality: is there a causal link? BMC Public Health 2016 Jul 26;16:635 [FREE Full text] [CrossRef] [Medline]
- Haseler T, Haseler C. Lack of physical activity is a global problem. BMJ 2022 Feb 23;376:o348 [FREE Full text] [CrossRef] [Medline]
- Lucivero F, Jongsma KR. A mobile revolution for healthcare? Setting the agenda for bioethics. J Med Ethics 2018 Oct;44(10):685-689 [FREE Full text] [CrossRef] [Medline]
- Helbostad JL, Vereijken B, Becker C, Todd C, Taraldsen K, Pijnappels M, et al. Mobile health applications to promote active and healthy ageing. Sensors (Basel) 2017 Mar 18;17(3):622 [FREE Full text] [CrossRef] [Medline]
- Direito A, Carraça E, Rawstorn J, Whittaker R, Maddison R. mHealth technologies to influence physical activity and sedentary behaviors: behavior change techniques, systematic review and meta-analysis of randomized controlled trials. Ann Behav Med 2017 Apr;51(2):226-239. [CrossRef] [Medline]
- Yerrakalva D, Yerrakalva D, Hajna S, Griffin S. Effects of mobile health app interventions on sedentary time, physical activity, and fitness in older adults: systematic review and meta-analysis. J Med Internet Res 2019 Nov 28;21(11):e14343 [FREE Full text] [CrossRef] [Medline]
- Valenzuela T, Okubo Y, Woodbury A, Lord SR, Delbaere K. Adherence to technology-based exercise programs in older adults: a systematic review. J Geriatr Phys Ther 2018;41(1):49-61. [CrossRef] [Medline]
- Jonkman NH, Del Panta V, Hoekstra T, Colpo M, van Schoor NM, Bandinelli S, et al. Predicting trajectories of functional decline in 60- to 70-year-old people. Gerontology 2018;64(3):212-221 [FREE Full text] [CrossRef] [Medline]
- Gunnes M, Langhammer B, Aamot IL, Lydersen S, Ihle-Hansen H, Indredavik B, LAST Collaboration group. Adherence to a long-term physical activity and exercise program after stroke applied in a randomized controlled trial. Phys Ther 2019 Jan 01;99(1):74-85. [CrossRef] [Medline]
- Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug 05;13(3):e52 [FREE Full text] [CrossRef] [Medline]
- Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med 2017 Jun;7(2):254-267 [FREE Full text] [CrossRef] [Medline]
- Attig C, Franke T. Abandonment of personal quantification: a review and empirical study investigating reasons for wearable activity tracking attrition. Comput Human Behav 2020 Jan;102:223-237. [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018 Oct 02;169(7):467-473 [FREE Full text] [CrossRef] [Medline]
- Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol 2005 Jan 28;5(1):6 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. mHealth: new horizons for health through mobile technologies: second global survey on eHealth. World Health Organization. 2011. URL: https://apps.who.int/iris/bitstream/handle/10665/44607/9789241564250_eng.pdf?sequence=1&isAllowed=y [accessed 2020-09-28]
- Global strategy on diet, physical activity and health: a framework to monitor and evaluate implementation. World Health Organization. 2006. URL: https://apps.who.int/iris/handle/10665/43524 [accessed 2020-09-28]
- Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B, et al. Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. J Health Serv Res Policy 2007 Jan;12(1):42-47. [CrossRef] [Medline]
- Cai C, Bao Z, Wu N, Wu F, Sun G, Yang G, et al. A novel model of home-based, patient-tailored and mobile application-guided cardiac telerehabilitation in patients with atrial fibrillation: a randomised controlled trial. Clin Rehabil 2022 Jan;36(1):40-50. [CrossRef] [Medline]
- Jiwani R, Wang J, Li C, Dennis B, Patel D, Gelfond J, et al. A behavioral lifestyle intervention to improve frailty in overweight or obese older adults with type 2 diabetes: a feasibility study. J Frailty Aging 2022;11(1):74-82 [FREE Full text] [CrossRef] [Medline]
- Allicock M, Kendzor D, Sedory A, Gabriel KP, Swartz MD, Thomas P, et al. A pilot and feasibility mobile health intervention to support healthy behaviors in African American breast cancer survivors. J Racial Ethn Health Disparities 2021 Feb;8(1):157-165. [CrossRef] [Medline]
- Anan T, Kajiki S, Oka H, Fujii T, Kawamata K, Mori K, et al. Effects of an artificial intelligence-assisted health program on workers with neck/shoulder pain/stiffness and low back pain: randomized controlled trial. JMIR Mhealth Uhealth 2021 Sep 24;9(9):e27535 [FREE Full text] [CrossRef] [Medline]
- Bisson AN, Sorrentino V, Lachman ME. Walking and daily affect among sedentary older adults measured using the StepMATE app: pilot randomized controlled trial. JMIR Mhealth Uhealth 2021 Dec 01;9(12):e27208 [FREE Full text] [CrossRef] [Medline]
- Daly RM, Gianoudis J, Hall T, Mundell NL, Maddison R. Feasibility, usability, and enjoyment of a home-based exercise program delivered via an exercise app for musculoskeletal health in community-dwelling older adults: short-term prospective pilot study. JMIR Mhealth Uhealth 2021 Jan 13;9(1):e21094 [FREE Full text] [CrossRef] [Medline]
- Eisenhauer CM, Brito F, Kupzyk K, Yoder A, Almeida F, Beller RJ, et al. Mobile health assisted self-monitoring is acceptable for supporting weight loss in rural men: a pragmatic randomized controlled feasibility trial. BMC Public Health 2021 Aug 18;21(1):1568 [FREE Full text] [CrossRef] [Medline]
- Gruner MP, Hogaboom N, Hasley I, Hoffman J, Gonzalez-Carta K, Cheville AL, et al. Prospective, single-blind, randomized controlled trial to evaluate the effectiveness of a digital exercise therapy application compared with conventional physical therapy for the treatment of nonoperative knee conditions. Arch Rehabil Res Clin Transl 2021 Aug 1;3(4):100151 [FREE Full text] [CrossRef] [Medline]
- Han M, Rhee SY. Effect of adherence to smartphone app use on the long-term effectiveness of weight loss in developing and OECD countries: retrospective cohort study. JMIR Mhealth Uhealth 2021 Jul 12;9(7):e13496 [FREE Full text] [CrossRef] [Medline]
- Layton AM, Irwin AM, Mihalik EC, Fleisch E, Keating CL, DiMango EA, et al. Telerehabilitation using fitness application in patients with severe cystic fibrosis awaiting lung transplant: a pilot study. Int J Telemed Appl 2021 Feb 26;2021:6641853 [FREE Full text] [CrossRef] [Medline]
- Napolitano MA, Harrington CB, Patchen L, Ellis LP, Ma T, Chang K, et al. Feasibility of a digital intervention to promote healthy weight management among postpartum African American/Black women. Int J Environ Res Public Health 2021 Feb 23;18(4):2178 [FREE Full text] [CrossRef] [Medline]
- Shelton E, Barreto NB, Bidwell S, Folk-Tolbert M, Shelton A, Trickey AW, et al. Engagement and adherence with a Web-based prehabilitation program for patients awaiting abdominal colorectal surgery. J Gastrointest Surg 2021 Dec;25(12):3198-3207. [CrossRef] [Medline]
- Yudi MB, Clark DJ, Tsang D, Jelinek M, Kalten K, Joshi SB, et al. SMARTphone-based, early cardiac REHABilitation in patients with acute coronary syndromes: a randomized controlled trial. Coron Artery Dis 2021 Aug 01;32(5):432-440. [CrossRef] [Medline]
- Taraldsen K, Mikolaizak AS, Maier AB, Mellone S, Boulton E, Aminian K, et al. Digital technology to deliver a lifestyle-integrated exercise intervention in young seniors-the PreventIT feasibility randomized controlled trial. Front Digit Health 2020 Jul 31;2:10 [FREE Full text] [CrossRef] [Medline]
- Adu MD, Malabu UH, Malau-Aduli AE, Drovandi A, Malau-Aduli BS. User retention and engagement with a mobile app intervention to support self-management in Australians with type 1 or type 2 diabetes (My Care Hub): mixed methods study. JMIR Mhealth Uhealth 2020 Jun 11;8(6):e17802 [FREE Full text] [CrossRef] [Medline]
- Grau-Pellicer M, Lalanza JF, Jovell-Fernández E, Capdevila L. Impact of mHealth technology on adherence to healthy PA after stroke: a randomized study. Top Stroke Rehabil 2020 Jul;27(5):354-368. [CrossRef] [Medline]
- Jiménez-Reguera B, Maroto López E, Fitch S, Juarros L, Sánchez Cortés M, Rodríguez Hermosa JL, et al. Development and preliminary evaluation of the effects of an mHealth Web-based platform (HappyAir) on adherence to a maintenance program after pulmonary rehabilitation in patients with Chronic Obstructive Pulmonary Disease: randomized controlled trial. JMIR Mhealth Uhealth 2020 Jul 31;8(7):e18465 [FREE Full text] [CrossRef] [Medline]
- Liew SJ, Gorny AW, Tan CS, Müller-Riemenschneider F. A mobile health team challenge to promote stepping and stair climbing activities: exploratory feasibility study. JMIR Mhealth Uhealth 2020 Feb 04;8(2):e12665 [FREE Full text] [CrossRef] [Medline]
- Petersen JM, Kemps E, Lewis LK, Prichard I. Associations between commercial app use and physical activity: cross-sectional study. J Med Internet Res 2020 Jun 03;22(6):e17152 [FREE Full text] [CrossRef] [Medline]
- Tabak M, de Vette F, van Dijk H, Vollenbroek-Hutten M. A game-based, physical activity coaching application for older adults: design approach and user experience in daily life. Games Health J 2020 Jun;9(3):215-226. [CrossRef] [Medline]
- Toro-Ramos T, Michaelides A, Anton M, Karim Z, Kang-Oh L, Argyrou C, et al. Mobile delivery of the diabetes prevention program in people with prediabetes: randomized controlled trial. JMIR Mhealth Uhealth 2020 Jul 08;8(7):e17842 [FREE Full text] [CrossRef] [Medline]
- Edney S, Ryan JC, Olds T, Monroe C, Fraysse F, Vandelanotte C, et al. User engagement and attrition in an app-based physical activity intervention: secondary analysis of a randomized controlled trial. J Med Internet Res 2019 Nov 27;21(11):e14645 [FREE Full text] [CrossRef] [Medline]
- Ellis TD, Cavanaugh JT, DeAngelis T, Hendron K, Thomas CA, Saint-Hilaire M, et al. Comparative effectiveness of mHealth-supported exercise compared with exercise alone for people with Parkinson disease: randomized controlled pilot study. Phys Ther 2019 Feb 01;99(2):203-216. [CrossRef] [Medline]
- Fukuoka Y, Haskell W, Lin F, Vittinghoff E. Short- and long-term effects of a mobile phone app in conjunction with brief in-person counseling on physical activity among physically inactive women: the mPED randomized clinical trial. JAMA Netw Open 2019 May 03;2(5):e194281 [FREE Full text] [CrossRef] [Medline]
- Griauzde D, Kullgren JT, Liestenfeltz B, Ansari T, Johnson EH, Fedewa A, et al. A mobile phone-based program to promote healthy behaviors among adults with prediabetes who declined participation in free diabetes prevention programs: mixed-methods pilot randomized controlled trial. JMIR Mhealth Uhealth 2019 Jan 09;7(1):e11267 [FREE Full text] [CrossRef] [Medline]
- Höchsmann C, Infanger D, Klenk C, Königstein K, Walz SP, Schmidt-Trucksäss A. Effectiveness of a behavior change technique-based smartphone game to improve intrinsic motivation and physical activity adherence in patients with type 2 diabetes: randomized controlled trial. JMIR Serious Games 2019 Feb 13;7(1):e11444 [FREE Full text] [CrossRef] [Medline]
- Lee BJ, Park YH, Lee JY, Kim SJ, Jang Y, Lee JI. Smartphone application versus pedometer to promote physical activity in prostate cancer patients. Telemed J E Health 2019 Dec;25(12):1231-1236. [CrossRef] [Medline]
- Ni Mhurchu C, Te Morenga L, Tupai-Firestone R, Grey J, Jiang Y, Jull A, et al. A co-designed mHealth programme to support healthy lifestyles in Māori and Pasifika peoples in New Zealand (OL@-OR@): a cluster-randomised controlled trial. Lancet Digit Health 2019 Oct;1(6):e298-e307 [FREE Full text] [CrossRef] [Medline]
- Muralidharan S, Ranjani H, Mohan Anjana R, Jena S, Tandon N, Gupta Y, et al. Engagement and weight loss: results from the mobile health and diabetes trial. Diabetes Technol Ther 2019 Sep;21(9):507-513. [CrossRef] [Medline]
- Oftedal S, Burrows T, Fenton S, Murawski B, Rayward AB, Duncan MJ. Feasibility and preliminary efficacy of an m-health intervention targeting physical activity, diet, and sleep quality in shift-workers. Int J Environ Res Public Health 2019 Oct 10;16(20):3810 [FREE Full text] [CrossRef] [Medline]
- Rayward AT, Vandelanotte C, Corry K, Van Itallie A, Duncan MJ. Impact of a social media campaign on reach, uptake, and engagement with a free Web- and app-based physical activity intervention: the 10,000 steps Australia program. Int J Environ Res Public Health 2019 Dec 12;16(24):5076 [FREE Full text] [CrossRef] [Medline]
- Tong HL, Coiera E, Tong W, Wang Y, Quiroz JC, Martin P, et al. Efficacy of a mobile social networking intervention in promoting physical activity: quasi-experimental study. JMIR Mhealth Uhealth 2019 Mar 28;7(3):e12181 [FREE Full text] [CrossRef] [Medline]
- Valentiner LS, Thorsen IK, Kongstad MB, Brinkløv CF, Larsen RT, Karstoft K, et al. Effect of ecological momentary assessment, goal-setting and personalized phone-calls on adherence to interval walking training using the InterWalk application among patients with type 2 diabetes-a pilot randomized controlled trial. PLoS One 2019 Jan 10;14(1):e0208181 [FREE Full text] [CrossRef] [Medline]
- Dugas M, Crowley K, Gao GG, Xu T, Agarwal R, Kruglanski AW, et al. Individual differences in regulatory mode moderate the effectiveness of a pilot mHealth trial for diabetes management among older veterans. PLoS One 2018 Mar 7;13(3):e0192807 [FREE Full text] [CrossRef] [Medline]
- Mascarenhas MN, Chan JM, Vittinghoff E, Van Blarigan EL, Hecht F. Increasing physical activity in mothers using video exercise groups and exercise mobile apps: randomized controlled trial. J Med Internet Res 2018 May 18;20(5):e179 [FREE Full text] [CrossRef] [Medline]
- Spring B, Pellegrini C, McFadden HG, Pfammatter AF, Stump TK, Siddique J, et al. Multicomponent mHealth intervention for large, sustained change in multiple diet and activity risk behaviors: the make better choices 2 randomized controlled trial. J Med Internet Res 2018 Jun 19;20(6):e10528 [FREE Full text] [CrossRef] [Medline]
- Salvi D, Ottaviano M, Muuraiskangas S, Martínez-Romero A, Vera-Muñoz C, Triantafyllidis A, et al. An m-Health system for education and motivation in cardiac rehabilitation: the experience of HeartCycle guided exercise. J Telemed Telecare 2018 May;24(4):303-316. [CrossRef] [Medline]
- Torquati L, Kolbe-Alexander T, Pavey T, Leveritt M. Changing diet and physical activity in nurses: a pilot study and process evaluation highlighting challenges in workplace health promotion. J Nutr Educ Behav 2018;50(10):1015-1025. [CrossRef] [Medline]
- Trinh L, Arbour-Nicitopoulos KP, Sabiston CM, Berry SR, Loblaw A, Alibhai SM, et al. RiseTx: testing the feasibility of a Web application for reducing sedentary behavior among prostate cancer survivors receiving androgen deprivation therapy. Int J Behav Nutr Phys Act 2018 Jun 07;15(1):49 [FREE Full text] [CrossRef] [Medline]
- Levin ME, Pierce B, Schoendorff B. The acceptance and commitment therapy matrix mobile app: a pilot randomized trial on health behaviors. J Contextual Behav Sci 2017 Jul;6(3):268-275. [CrossRef]
- Lambert TE, Harvey LA, Avdalis C, Chen LW, Jeyalingam S, Pratt CA, et al. An app with remote support achieves better adherence to home exercise programs than paper handouts in people with musculoskeletal conditions: a randomised trial. J Physiother 2017 Jul;63(3):161-167 [FREE Full text] [CrossRef] [Medline]
- Ryan J, Edney S, Maher C. Engagement, compliance and retention with a gamified online social networking physical activity intervention. Transl Behav Med 2017 Dec;7(4):702-708 [FREE Full text] [CrossRef] [Medline]
- Willcox JC, Wilkinson SA, Lappas M, Ball K, Crawford D, McCarthy EA, et al. A mobile health intervention promoting healthy gestational weight gain for women entering pregnancy at a high body mass index: the txt4two pilot randomised controlled trial. BJOG 2017 Oct;124(11):1718-1728. [CrossRef] [Medline]
- Valle CG, Tate DF. Engagement of young adult cancer survivors within a Facebook-based physical activity intervention. Transl Behav Med 2017 Dec;7(4):667-679 [FREE Full text] [CrossRef] [Medline]
- Quintiliani LM, Mann DM, Puputti M, Quinn E, Bowen DJ. Pilot and feasibility test of a mobile health-supported behavioral counseling intervention for weight management among breast cancer survivors. JMIR Cancer 2016;2(1):e4 [FREE Full text] [CrossRef] [Medline]
- Block G, Azar KM, Romanelli RJ, Block TJ, Hopkins D, Carpenter HA, et al. Diabetes prevention and weight loss with a fully automated behavioral intervention by email, Web, and mobile phone: a randomized controlled trial among persons with prediabetes. J Med Internet Res 2015 Oct 23;17(10):e240 [FREE Full text] [CrossRef] [Medline]
- Guertler D, Vandelanotte C, Kirwan M, Duncan MJ. Engagement and nonusage attrition with a free physical activity promotion program: the case of 10,000 steps Australia. J Med Internet Res 2015 Jul 15;17(7):e176 [FREE Full text] [CrossRef] [Medline]
- Maher C, Ferguson M, Vandelanotte C, Plotnikoff R, De Bourdeaudhuij I, Thomas S, et al. A Web-based, social networking physical activity intervention for insufficiently active adults delivered via Facebook app: randomized controlled trial. J Med Internet Res 2015 Jul 13;17(7):e174 [FREE Full text] [CrossRef] [Medline]
- Laing BY, Mangione CM, Tseng CH, Leng M, Vaisberg E, Mahida M, et al. Effectiveness of a smartphone application for weight loss compared with usual care in overweight primary care patients: a randomized, controlled trial. Ann Intern Med 2014 Nov 18;161(10 Suppl):S5-12 [FREE Full text] [CrossRef] [Medline]
- Varnfield M, Karunanithi M, Lee CK, Honeyman E, Arnold D, Ding H, et al. Smartphone-based home care model improved use of cardiac rehabilitation in postmyocardial infarction patients: results from a randomised controlled trial. Heart 2014 Nov;100(22):1770-1779 [FREE Full text] [CrossRef] [Medline]
- Carter MC, Burley VJ, Nykjaer C, Cade JE. Adherence to a smartphone application for weight loss compared to website and paper diary: pilot randomized controlled trial. J Med Internet Res 2013 Apr 15;15(4):e32 [FREE Full text] [CrossRef] [Medline]
- Thomas JG, Wing RR. Health-e-call, a smartphone-assisted behavioral obesity treatment: pilot study. JMIR Mhealth Uhealth 2013 Apr 17;1(1):e3 [FREE Full text] [CrossRef] [Medline]
- Kirwan M, Duncan MJ, Vandelanotte C, Mummery WK. Using smartphone technology to monitor physical activity in the 10,000 steps program: a matched case-control trial. J Med Internet Res 2012 Apr 20;14(2):e55 [FREE Full text] [CrossRef] [Medline]
- Burke LE, Conroy MB, Sereika SM, Elci OU, Styn MA, Acharya SD, et al. The effect of electronic self-monitoring on weight loss and dietary intake: a randomized behavioral weight loss trial. Obesity (Silver Spring) 2011 Feb;19(2):338-344 [FREE Full text] [CrossRef] [Medline]
- Hawley-Hague H, Horne M, Skelton DA, Todd C. Review of how we should define (and measure) adherence in studies examining older adults' participation in exercise classes. BMJ Open 2016 Jun 23;6(6):e011560 [FREE Full text] [CrossRef] [Medline]
- Couper MP, Alexander GL, Zhang N, Little RJ, Maddy N, Nowak MA, et al. Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res 2010 Nov 18;12(4):e52 [FREE Full text] [CrossRef] [Medline]
- Sabia S, Cogranne P, van Hees VT, Bell JA, Elbaz A, Kivimaki M, et al. Physical activity and adiposity markers at older ages: accelerometer vs questionnaire data. J Am Med Dir Assoc 2015 May 01;16(5):438.e7-438.13 [FREE Full text] [CrossRef] [Medline]
- Mondal H, Mondal S. Social desirability bias: a confounding factor to consider in survey by self-administered questionnaire. Indian J Pharmacol 2018;50(3):143-144 [FREE Full text] [CrossRef] [Medline]
- Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med 2016 Dec;23(6):776-794. [CrossRef] [Medline]
- Christensen H, Griffiths KM, Farrer L. Adherence in Internet interventions for anxiety and depression. J Med Internet Res 2009 Apr 24;11(2):e13 [FREE Full text] [CrossRef] [Medline]
- Yang X, Ma L, Zhao X, Kankanhalli A. Factors influencing user's adherence to physical activity applications: a scoping literature review and future directions. Int J Med Inform 2020 Feb;134:104039. [CrossRef] [Medline]
Abbreviations
mHealth: mobile health |
PA: physical activity |
PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews |
Edited by A Mavragani; submitted 02.06.21; peer-reviewed by R Krukowski, L König; comments to author 04.08.21; revised version received 16.09.21; accepted 15.03.22; published 08.06.22
Copyright©Yang Yang, Elisabeth Boulton, Chris Todd. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 08.06.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.