Original Paper
Abstract
Background: The use of digital interventions can be accurately monitored via log files. However, monitoring engagement with intervention goals or enactment of the actual behaviors targeted by the intervention is more difficult and is usually evaluated based on pre-post measurements in a controlled trial.
Objective: The objective of this paper is to evaluate if engaging with 2 digital intervention modules focusing on (1) physical activity goals and action plans and (2) coping with barriers has immediate effects on the actual physical activity behavior.
Methods: The NoHoW Toolkit (TK), a digital intervention developed to support long-term weight loss maintenance, was evaluated in a 2 x 2 factorial randomized controlled trial. The TK contained various modules based on behavioral self-regulation and motivation theories, as well as contextual emotion regulation approaches, and involved continuous tracking of weight and physical activity through connected commercial devices (Fitbit Aria and Charge 2). Of the 4 trial arms, 2 had access to 2 modules directly targeting physical activity: a module for goal setting and action planning (Goal) and a module for identifying barriers and coping planning (Barriers). Module visits and completion were determined based on TK log files and time spent in the module web page. Seven physical activity metrics (steps; activity; energy expenditure; fairly active, very active and total active minutes; and distance) were compared before and after visiting and completing the modules to examine whether the modules had immediate or sustained effects on physical activity. Immediate effect was determined based on 7-day windows before and after the visit, and sustained effects were evaluated for 1 to 8 weeks after module completion.
Results: Out of the 811 participants, 498 (61.4%) visited the Goal module and 406 (50.1%) visited the Barriers module. The Barriers module had an immediate effect on very active and total active minutes (very active minutes: before median 24.2, IQR 10.4-43.0 vs after median 24.9, IQR 10.0-46.3; P=.047; total active minutes: before median 45.1, IQR 22.9-74.9 vs after median 46.9, IQR 22.4-78.4; P=.03). The differences were larger when only completed Barriers modules were considered. The Barriers module completion was also associated with sustained effects in fairly active and total active minutes for most of the 8 weeks following module completion and for 3 weeks in very active minutes.
Conclusions: The Barriers module had small, significant, immediate, and sustained effects on active minutes measured by a wrist-worn activity tracker. Future interventions should pay attention to assessing barriers and planning coping mechanisms to overcome them.
Trial Registration: ISRCTN Registry ISRCTN88405328; https://www.isrctn.com/ISRCTN88405328
doi:10.2196/35614
Keywords
Introduction
Digital behavior change interventions (DBCIs) hold the promise of providing personalized and adaptive treatments to improve health. It is already possible to track individual user interactions with DBCI components at a high level of detail and fidelity, such as when and for how long users are accessing them. The evidence linking use to intervention outcomes is mixed, and the relationships are not straightforward. Further, the type of engagement required for effects may also vary across different types of interventions [,].
User engagement with the real-world behaviors that a DBCI is attempting to influence, the macro-level engagement [], is more difficult to determine. Traditionally, behavioral changes are verified using questionnaires or laboratory measurements conducted intermittently, with measurement points several months apart and the risk of reporting bias. At that point, it is impossible to prove causation between specific intervention components and change. Monitoring the immediate effects of DBCI components would enable active process evaluation of interventions and contribute greatly to tailoring effective, adaptive, and personalized interventions. For example, if an intervention component has an immediate effect on the user’s behavior, the component could be repeated after a while to amplify the effect. If a component does not lead to an expected effect for a user, it could be switched off and an alternative intervention could be launched to create an adaptive intervention [].
Research on behavior change techniques has revealed the positive effects of self-regulatory techniques on physical activity behavior. Core self-regulatory techniques are self-monitoring of behavior and feedback; goal setting, that is, identification and formulation of a physical activity goal; action planning, that is, specification of the goal in a detailed plan for the performance of the behavior (context, frequency, duration, and the intensity of the activity); and coping planning, that is, identification of barriers to physical activity and planning ways to overcome them using, for example, an “if, then” approach [-].
This study focuses on a web-based toolkit for weight loss maintenance, consisting of various modules based on behavioral self-regulation and motivation theories, as well as contextual emotion regulation. The objective of this study was to investigate whether intervention modules aimed at increasing physical activity through goal setting and action planning, and coping planning had immediate and sustained effects on the physical activity behavior of users. This was a secondary analysis of the data from a European Commission Horizon 2020–funded NoHoW project.
Methods
Ethics Approval
The trial was registered with the ISRCTN registry (ISRCTN88405328). Ethical approval was granted by local institutional ethics committees at the Universities of Leeds (17-0082; 27 February 2017), Lisbon (17/2016; 20 February 2017) and the Capital Region of Denmark (H-16030495; 8 March 2017).
Study Procedures
The NoHoW trial (ISRCTN88405328) was an 18-month, 3-center, 2-by-2 factorial, single-blinded, randomized controlled trial, evaluating a digital weight loss maintenance intervention. The participants were required to be aged ≥18 years, have a verified ≥5% weight loss in the last 12 months with current weight at least 5% below their highest weight, and have had a BMI of ≥25 kg/m2 before weight loss. A total of 1627 participants were recruited and randomly assigned to one of the following four arms: (1) control or self-monitoring (n=400), (2) motivation and self-regulation (n=403), (3) emotion regulation (n=416), and (4) combined arm (n=408). All participants received activity trackers (Fitbit Charge 2) to be worn throughout the trial, weight scales (Fitbit Aria), and access to the web-based NoHoW Toolkit (TK) tailored to their respective arm. Participants in intervention arms were encouraged to complete 18 intervention modules in the TK during the first 6 months of the trial. The participants received weekly emails during the first 18 weeks as reminders recommending visiting a specific module. A detailed description of the trial is presented by Scott et al []. The TK design and content are presented in detail by Marques et al [].The motivation and self-regulation and combined arms had 2 modules focusing on physical activity: physical activity goal (Goal) and physical activity barriers (Barriers). The Goal module (see for screenshots) addressed goal setting and action planning, contained information on how to set goals, and had a form to set a goal and detailed plan for either the number of steps per day or other type of physical activity. The estimated duration to complete the Goal module was 10 minutes. The Barriers module (see for screenshots) was introduced later in the intervention and contained a testimonial on potential barriers of physical activity and an interactive exercise for identifying personal barriers and creating a coping plan to deal with them. The estimated duration to complete the Barriers module was 8 minutes.
Analysis
Visits to the Goal and Barriers modules were identified based on log files of the TK. The duration spent in the modules was calculated based on log events signifying entering and leaving the module. A module visit was considered complete if it lasted at least 33% of the estimated duration of the module (ie, 3.3/10 min for the Goal module and 2.6/8 min for the Barriers module) or if the duration of the visit could not be determined due to a missing end event. The threshold of 33% was determined by intervention designers as the minimum time required to become exposed to the behavior change mechanisms in these modules.
Daily summaries provided by Fitbit were used as the physical activity metrics and included daily steps; activity energy expenditure; active minutes categorized to fairly active, very active, and total active minutes; and distance. Activity metrics were averaged over the 7 days prior to visiting the modules and 7 days after visiting them. Days with less than 1000 steps were considered missing data and were not included. A similar threshold has been used in several previous studies (eg, [,]). It was also required that the 7-day periods contained at least 4 days of activity data. The immediate effects are presented both for all visits to the modules and for complete visits. If an immediate effect was found, maintenance of the effects was evaluated for 8 weeks following the module visits, considering only complete module visits. A comparison of overall changes in activity metrics between the first and sixth month of the study is presented in .
As most of the physical activity metrics had skewed distributions, nonparametric methods were used. Median and interquartile ranges were calculated, and nonparametric tests (Wilcoxon signed-rank test) were used for comparisons. All analyses were conducted with Matlab R2017a (Mathworks) and SPSS Statistics software (version 26; IBM Corp). Statistical significance was set at P<.05.
Results
User Statistics
The modules were available for 811 participants (ie, the participants randomized to the motivation and self-regulation and to the combined arms). Of the 811 participants, 498 (61.4%) visited the Goal module (252/403 motivation and self-regulation and 246/408 combined), and 406 (50.1%) participants visited the Barriers module (217/403 motivation and self-regulation and 189/408 combined). There were 628 visits to the Goal module, of which 309 were complete visits. The Barriers module had 514 visits, with 345 complete. The background characteristics of the visitors and nonvisitors of these modules are presented and compared in .
Immediate Effects of Modules
The Goal module was first visited a median of 55 days (IQR 48-64) after the first login to TK, and the Barriers module was first visited a median of 98 days (IQR 91-110) after the first login. and present the median and IQR for the activity metrics before and after visiting the Goal and Barriers modules, respectively. Visiting the Barriers module increased very active and total active minutes.
When only completed module visits were included, the results for the Goal module remained nonsignificant. For the Barriers module, the effects remained and became slightly stronger (for very active minutes: before median 22.4, IQR 10.1-41.0 vs after median 25.0, IQR 10.1-46.1; P=.007; and for total active minutes: before median 42.6, IQR 22.4-73.4 vs after median 46.6, IQR 21.8-79.4; P=.008).
| Before, median (IQR) | After, median (IQR) | P valuea | |
| Steps | 9637 (7383-12,485) | 9638 (7354-12,425) | .62 |
| Energy (kcal) | 1276 (997-1632) | 1274 (987-1640) | .74 |
| Fairly active (min) | 18.9 (10.2-33.3) | 18.3 (11.0-32.5) | .69 |
| Very active (min) | 23.9 (12.2-44.8) | 26.4 (12.6-44.8) | .39 |
| Total active (min) | 46.6 (25.1-75.4) | 48.3 (26.6-76.5) | .44 |
| Distance (km) | 6.8 (5.1-8.9) | 6.8 (5.1-8.7) | .60 |
aWilcoxon signed-rank test.
| Before, median (IQR) | After, median (IQR) | P valuea | |
| Steps | 9449 (7240-12,112) | 9119 (7202-12,382) | .70 |
| Energy (kcal) | 1211 (990-1585) | 1260 (943-1611) | .27 |
| Fairly active (min) | 16.7 (9.7-31.9) | 18.9 (9.6-33.8) | .15 |
| Very active (min) | 24.2 (10.4-43.0) | 24.9 (10.0-46.3) | .047 |
| Total active (min) | 45.1 (22.9-74.9) | 46.9 (22.4-78.4) | .03 |
| Distance (km) | 6.7 (4.9-8.5) | 6.4 (4.9-8.7) | .68 |
aWilcoxon signed-rank test.
presents the 8-week maintenance of the Barriers module effect for the 3 active minute metrics based on completed module visits only. Total active minutes and fairly active minutes were higher than before module completion for most of the 8-week period following the module. Also, very active minutes remained higher for 3 weeks after module completion. Values that significantly differed from the before value were denoted.
| Fairly active (min), median (IQR) | Very active (min), median (IQR) | Total active (min), median (IQR) | |
| Before | 15.9 (9.4-29.4) | 22.4 (10.1-41.0) | 42.6 (22.4-73.4) |
| Week 1 after | 18.4 (9.2-33.3) | 25.0 (10.1-46.1)a | 46.6 (21.8-79.4)a |
| Week 2 after | 19.1 (10.9-31.8)a | 26.6 (12.0-45.0)b | 45.6 (26.4-75.7)b |
| Week 3 after | 18.6 (10.2-34.6)b | 24.4 (11.9-45.6)a | 44.6 (25.3-78.2)a |
| Week 4 after | 18.1 (11.6-37.0)b | 23.1 (12.0-43.2) | 47.2 (25.2-76.8)a |
| Week 5 after | 19.6 (10.6-30.6)a | 24.4 (11.9-42.1) | 44.7 (25.7-72.3) |
| Week 6 after | 19.6 (11.4-33.3)b | 23.6 (12.4-41.2) | 45.3 (27.1-78.5)a |
| Week 7 after | 20.3 (11.2-32.7)b | 24.0 (12.0-40.9) | 44.4 (27.6-74.4)c |
| Week 8 after | 20.3 (11.2-31.7)a | 23.7 (11.2-42.3) | 46.1 (26.7-73.4) |
aP<.01, Wilcoxon signed-rank test.
bP<.001.
cP<.05.
Discussion
Principal Results
This paper investigates the immediate changes in measured physical activity after visiting 2 web-based intervention modules targeting physical activity goals and action planning (Goal), and coping planning (Barriers). The Barriers module had a significant but small, immediate effect on very active and total active minutes during the week after visiting the module. When only completed module visits were considered, increases were larger. Module completion was also associated with sustained increases in all categories of active minutes, of which fairly active minutes and total active minutes were sustained for most of the 8-week period following module completion, and very active minutes for the first 3 weeks. Coping planning, addressed in the Barriers module, has previously been identified as a core self-regulation technique that can directly impact behavior (eg, [,]), and our findings support this effect for physical activity. The Goal module did not show similar effects on physical activity. One explanation for the differences in the results is that the coping planning activity in the Barriers module asked participants to identify barriers and strategies to overcome them in the immediate future (following week), while in the Goal module participants were asked to set a goal to increase their activity moderately in an unspecified timeframe.
Strengths and Limitations
The strengths of this study include the large number of participants and the ability to monitor the exposure to specific intervention techniques as well as the subsequent behavior.
The reliability of the variables may be affected by differences in individual wear times of the trackers. This was mitigated by setting a threshold of 1000 steps to consider the day valid. There is also a known tendency for Fitbit (and other trackers) to overestimate the amount of moderate to vigorous activity [,]. Furthermore, we used multiple observations for some volunteers to maximize the sample size. Although the rank-based nonparametric tests could not accommodate a random effect to account for the multiple observations, we checked an analysis, which averaged the observations per participant and found that the results and conclusions were the same.
The timing of the modules may have impacted the results. Although the order of visiting the modules was not technically restricted, the Goal session appeared earlier in the intervention flow and was visited by more participants than the Barriers session. It is thus possible that participants visiting the Barriers session were more committed to the intervention and, therefore, more likely to adhere to behavioral guidance as well. Further, engagement was only assessed based on log files and not by the quality of the action and coping plans done by the user, or the enactment of these.
Future Work
As far as we know, no prior research to which we can directly compare these results exists. This shows the need for more research to examine the direct and immediate as well as longer-term effects of engaging with DBCI content on the enactment of the target behavior. Specifically, such work constitutes an important step toward identifying which behavior change techniques can have a differential impact on physical activity. This knowledge can, in turn, contribute to optimizing DBCIs in an adaptive and personalized way.
Conclusions
A self-regulation–based intervention module addressing physical activity barriers induced a significant increase in active minutes, and the effect was stronger when the module was completed. Module completion was further associated with sustained increases, especially in fairly active and total active minutes.
Acknowledgments
This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement 643309. The material presented and views expressed here are the responsibility of the author(s) only. The European Commission takes no responsibility for any use made of the information set out.
Authors' Contributions
EM and MMM conceived the study. EM conducted the analyses with consultation from GH. RJS was the principal investigator for the NoHoW trial, and BLH was the principal coordinator of the NoHoW project. EM, MMM, ALP and GH drafted the manuscript, and all authors revised the manuscript and approved the final version.
Conflicts of Interest
RJS consults for Slimming World through Consulting Leeds, which is a wholly owned subsidiary of the University of Leeds. Slimming World was a former partner in NoHoW. MMM and GH have previously consulted for Slimming World. Other coauthors have no conflicts of interest to declare.
Screenshots of the Goal module.
PDF File (Adobe PDF File), 346 KB
Screenshots of the Barriers module.
PDF File (Adobe PDF File), 460 KB
Additional analyses on background characteristics and activity metrics.
DOCX File , 17 KBReferences
- Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):833-842. [CrossRef] [Medline]
- Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JEWC. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012;14(6):e152 [FREE Full text] [CrossRef] [Medline]
- Almirall D, Nahum-Shani I, Sherwood NE, Murphy SA. Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research. Transl Behav Med 2014 Sep;4(3):260-274 [FREE Full text] [CrossRef] [Medline]
- Sniehotta FF, Scholz U, Schwarzer R. Bridging the intention-behaviour gap: planning, self-efficacy, and action control in the adoption and maintenance of physical exercise. Psychol Health 2005 Apr;20(2):143-160. [CrossRef]
- Dombrowski SU, Sniehotta FF, Avenell A, Johnston M, MacLennan G, Araújo-Soares V. Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: a systematic review. Health Psychol Rev 2012 Mar;6(1):7-32. [CrossRef]
- Hagger MS, Luszczynska A, de Wit J, Benyamini Y, Burkert S, Chamberland P, et al. Implementation intention and planning interventions in health psychology: recommendations from the synergy expert group for research and practice. Psychol Health 2016 Jul;31(7):814-839. [CrossRef] [Medline]
- Scott SE, Duarte C, Encantado J, Evans EH, Harjumaa M, Heitmann BL, et al. The NoHoW protocol: a multicentre 2×2 factorial randomised controlled trial investigating an evidence-based digital toolkit for weight loss maintenance in European adults. BMJ Open 2019 Sep 30;9(9):e029425 [FREE Full text] [CrossRef] [Medline]
- Marques MM, Matos M, Mattila E, Encantado J, Duarte C, Teixeira PJ, et al. A theory- and evidence-based digital intervention tool for weight loss maintenance (NoHoW Toolkit): systematic development and refinement study. J Med Internet Res 2021 Dec 03;23(12):e25305 [FREE Full text] [CrossRef] [Medline]
- Orstad SL, Gerchow L, Patel NR, Reddy M, Hernandez C, Wilson DK, et al. Defining valid activity monitor data: a multimethod analysis of weight-loss intervention participants’ barriers to wear and first 100 days of physical activity. Informatics 2021 Jun 06;8(2):39. [CrossRef]
- Hardcastle SJ, Jiménez-Castuera R, Maxwell-Smith C, Bulsara MK, Hince D. Fitbit wear-time and patterns of activity in cancer survivors throughout a physical activity intervention and follow-up: exploratory analysis from a randomised controlled trial. PLoS One 2020;15(10):e0240967 [FREE Full text] [CrossRef] [Medline]
- Kwasnicka D, Presseau J, White M, Sniehotta FF. Does planning how to cope with anticipated barriers facilitate health-related behaviour change? A systematic review. Health Psychol Rev 2013 Feb 19;7(2):129-145. [CrossRef]
- Schroé H, Van Dyck D, De Paepe A, Poppe L, Loh WW, Verloigne M, et al. Which behaviour change techniques are effective to promote physical activity and reduce sedentary behaviour in adults: a factorial randomized trial of an e- and m-health intervention. Int J Behav Nutr Phys Act 2020 Oct 07;17(1):127 [FREE Full text] [CrossRef] [Medline]
- Tedesco S, Sica M, Ancillao A, Timmons S, Barton J, O'Flynn B. Validity evaluation of the Fitbit Charge2 and the Garmin Vivosmart HR+ in free-living environments in an older adult cohort. JMIR Mhealth Uhealth 2019 Jun 19;7(6):e13084 [FREE Full text] [CrossRef] [Medline]
- Redenius N, Kim Y, Byun W. Concurrent validity of the Fitbit for assessing sedentary behavior and moderate-to-vigorous physical activity. BMC Med Res Methodol 2019 Feb 07;19(1):29 [FREE Full text] [CrossRef] [Medline]
Abbreviations
| DBCI: digital behavior change intervention |
| TK: NoHoW Toolkit |
Edited by A Mavragani; submitted 10.12.21; peer-reviewed by G Crombez, E Neter; comments to author 19.01.22; revised version received 23.02.22; accepted 15.03.22; published 14.04.22
Copyright©Elina Mattila, Graham Horgan, António L Palmeira, Ruairi O'Driscoll, R James Stubbs, Berit L Heitmann, Marta M Marques. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 14.04.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.