RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) Evaluation of the Use of Activity Trackers in the Clinical Care of Adults Diagnosed With a Chronic Disease: Integrative Systematic Review

Introduction

Noncommunicable diseases, also referred to as chronic diseases, are a major cause of premature death. They include cardiovascular disease, chronic respiratory disease, cancer, and diabetes mellitus. It is estimated that globally 41 million people die each year from chronic diseases, and this accounts for 71% of all deaths worldwide [1]. Major risk factors for the development of chronic diseases are low levels of physical activity and high levels of sedentary behavior [2]. It is estimated that globally 23% of adults are failing to achieve the recommended physical activity guidelines set by the World Health Organization [1]. The adult guidelines recommend 150 minutes of moderate to vigorous intensity physical activity per week, to undertake muscle exercises at least 2 days per week, and to reduce sedentary time [3].

Physical activity interventions have been developed in an effort to encourage individuals to be more active within both primary health care and community-based settings [4]. Activity trackers, such as Fitbit (Fitbit), Polar (Polar Electro), and Garmin (Garmin Ltd), which can monitor and provide feedback on steps, physical activity intensity, distance moved, and sedentary behavior, can support patients in leading a more active lifestyle and could be a cost-effective public health intervention [5].

During test and retest trials, activity trackers and pedometers were found to be valid and reliable methods of measuring steps (Misfit Shine, Withings Pulse, Fitbit Zip, and Digiwalker) [6]. Fitbit’s activity trackers (Ultra, One, Zip, Flex, Force, Charge, Charge HR, and Surge), for example, have been shown to have high interdevice reliability in respect of distance moved, steps, and energy expenditure [7]. A recent systematic review highlighted that the use of activity trackers by patients diagnosed with a chronic disease (chronic respiratory disease, type 2 diabetes, and cardiovascular disease) would significantly increase the number of daily steps taken (2123) and decrease a patient’s systolic blood pressure (–3.79 mm Hg), waist circumference (–0.99 cm), and low-density lipoprotein cholesterol concentration (–5.70 mg/dL) [8].

Interventions involving changes in lifestyle such as increased physical activity, reduction in sedentary time, and weight loss can significantly reduce the risk of adults developing chronic diseases and related health complications. Weight loss programs involving dietary and physical activity education coupled with the use of an activity tracker have been shown to significantly reduce the weight of participants [9].

While many studies involving public health interventions focus on effect and efficacy, these fail to fully evaluate the impact of such research when applied to real-world settings. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework was developed as both a planning and evaluation tool aimed at understanding the full impact of a public health intervention and how this could best be introduced into patient health care [10]. The framework comprises 5 main dimensions: reach, effectiveness, adoption, implementation, and maintenance. Reach seeks to measure the proportion of the targeted population who took part in the intervention. Effectiveness aims to understand the success of the intervention. Adoption covers the number of settings, practices, and plans that will engage with the intervention. Implementation looks at how the intervention was applied in practice. Maintenance seeks to understand the long-term viability of the project [10]. Each dimension should be considered when planning or evaluating a public health intervention. The main dimensions are subsequently subdivided into indicators that facilitate a detailed evaluation of each dimension [10]. Though each dimension should be considered at the planning stage of an intervention, it is not imperative to report in detail all components. Greater focus on those applicable to the individual study or clinical organization has been shown to improve the translation from research into practical health care, which has proved to be a barrier in the past [10].

The aim of this integrative systematic review was to comprehensively explore the reporting of the reach, effectiveness, adoption, implementation, and maintenance of using activity trackers in clinical care to support physical activity or reduce sedentary behavior in adults diagnosed with chronic diseases. The integrative methodology allows for the review of quantitative, qualitative, and mixed methods research while developing a comprehensive understanding of the RE-AIM elements reported and the particular health care problem under review [11].

Methods

Study Design

The protocol for this integrative systematic review was registered in the PROSPERO international prospective register of systematic reviews on March 23, 2022 (CRD42022319635). The review was undertaken in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 statement [12]. All published papers in English between 2015 and 2022 were included.

Eligibility Criteria

Textbox 1 provides a summary of the inclusion criteria using PICOS (population, intervention type, comparisons, outcomes of interest, and study type) for quantitative studies and SPIDER (sample, phenomenon of interest, study design, evaluation, and research type) for qualitative studies.

Search Strategy and Information Sources

A detailed search strategy was developed, with guidance from a qualified university librarian, around the core themes and RE-AIM dimensions. The key search words are displayed in Table 1. These were developed around the key original terms: “activity trackers,” “clinical care,” “physical activity,” “sedentary behaviour,” “adults,” “chronic diseases,” and “RE-AIM.”

The study searches were undertaken on the computer-based databases: Web of Science—Core Collection (all editions; Clarivate), MEDLINE (ProQuest; ProQuest), ACM (Digital Library; ACM Digital Library), APA PsycINFO (EBSCO), Cochrane Library (Cochrane), MEDLINE (Ovid, Embase, and Embase Classic; MEDLINE), and gray literature (GreyNet; GreyNet International), CORE (Core), OAIster and WorldCat.org (OCLC), and Open Access Thesis and Dissertations (OATD).

Screening Process

Papers identified during the search phase were initially uploaded onto EndNote (Clarivate). Duplicates were identified and removed. Papers were subsequently uploaded onto the Rayyan (Rayyan Systems Inc) collaborative systematic review screening system. All remaining papers were screened on the Rayyan system by 4 authors (WH, ER, CW, and DE) based on the topic, title, and abstract. Last, a full-text screening of the remaining papers was conducted by 2 authors (WH and ER). Conflicts were resolved at each stage through discussion between the screening authors without the need for an independent review.

Data Extraction

The final papers identified for review were uploaded onto a validated RE-AIM extraction Microsoft Excel spreadsheet (Microsoft Corporation; Multimedia Appendix 1). This spreadsheet was developed by combining 2 validated RE-AIM extraction tools, and it contains details of each paper, the 5 main RE-AIM dimensions (reach, effectiveness, adoption, implementation, and maintenance), and 28 RE-AIM indicators [11-14]. The identified papers were screened, and data were extracted by the main author (WH). This data extraction was quality-checked by 3 authors (AK, XJ, and ML), who reviewed 2 separate papers each. Conflicts were resolved through discussion between all data extraction authors. The PRISMA checklist was followed throughout [12] (Multimedia Appendix 2).

Data Items

Basic participant demographic details were extracted from each paper including number of participants, mean age, and gender. Each of the 5 main RE-AIM dimensions is supported by identifiable indicators (reach: n=7, effectiveness: n=6, adoption: n=6, implementation: n=5, and maintenance: n=4) [11-14]. The main RE-AIM dimensions and supporting indicators are displayed in Textbox 2.

Each identified journal paper was screened as previously described, and the reporting of each RE-AIM indicator was recorded on the RE-AIM data extraction sheet. The percentage of papers reporting the individual indicator was calculated.

Study Risk of Bias Assessment and Reporting Bias Assessment

As the RE-AIM framework is an evaluation tool designed to identify and focus on the reporting of recommended dimensions and indicators within interventions, no additional risk of bias assessment was conducted [15].

Outcome Measures

The total percentage of individual RE-AIM indicators reported from all journal papers was the main outcome measure.

Results

Overview

The search strategy identified 5619 potential studies for further analysis. After checking for duplicates, the number of eligible studies was reduced to 4585. After the topic, title, and abstract screening process, 63 studies were eligible. Full-text screening reduced this number to 15 studies for final analysis. Figure 1 [12] provides an overview of the screening process and the number of studies at each stage. The final 15 studies were published between 2016 and 2022 and were from Australia (n=1), Spain and the Netherlands (n=1), Spain (n=2), the United States (n=7), Canada (n=3), and Taiwan (n=1). The studies included involved the following chronic diseases: lower back pain (n=1), cardiovascular disease (n=2), hemophilia (n=1), colorectal cancer (n=1), breast cancer (n=1), stroke (n=2), type 2 diabetes (n=2), inflammatory arthritis (n=1), kidney disease (n=3), and serious mental illness (n=1).

The characteristics (author, country, topic area, number of participants, mean age of participants, chronic diseases studied, activity tracker, study duration, and study design) of the 15 studies are reported in Table 2. A context summary of the activity tracker use in each study is reported in Table 3.

‎

Participants Demographics

In total, 844 adult participants took part in the 15 studies. Of these, 412 (48.8%) were male, and 432 (51.2%) were female. The mean age of all participants was 58.3 (SD 8.6) years.

Percentage Reporting Across RE-AIM Dimensions

The RE-AIM dimensions are supported by 28 indicators. Table 4 shows the mean total number of indicators reported across the 5 main dimensions.

Percentage Reporting Across the RE-AIM Indicators

Reach

The reach dimension is supported by 7 indicators, and the overall numbers and percentages reported are shown in Table 5.

Effectiveness

The effectiveness dimension was supported by 6 indicators, and the overall numbers and percentages reported are shown in Table 6.

Adoption

The adoption dimension was supported by 6 indicators, and the overall numbers and percentages reported are shown in Table 7.

Implementation

The implementation dimension was supported by 5 indicators, and the overall numbers and percentages reported are shown in Table 8.

Maintenance

The maintenance dimension was supported by 4 indicators, and the overall numbers and percentages reported are shown in Table 9.

Discussion

Principal Findings

This integrative systematic review sets out to measure the level of reporting for each of the 28 RE-AIM indicators in studies identified during the search and screening phase. To the best of the lead author’s knowledge, no similar studies have been undertaken, and it is hoped these findings will be informative in the planning and evaluation of future public health interventions to support the management of chronic diseases. In summary, this review found that the RE-AIM dimensions of adoption and maintenance were underreported. The analysis of the indicators found that the use of qualitative methods to understand each dimension was underreported.

The use of commercial activity trackers within physical activity interventions either on their own or when combined with behavior change strategies has been shown to increase activity levels and reduce sedentary behavior in people diagnosed with a chronic disease [8]. In 2020, Franssen et al [8] in a systematic review identified 35 studies covering this area of research. The use of activity trackers specifically within clinical practice has been researched less. The search strategy undertaken for this review identified only 15 studies with all targeting physical activity and 1 targeting sedentary behavior. This indicates that their use in clinical practice could be increased and has a significant positive impact on the health of patients diagnosed with a chronic disease. Understanding of the reach, effectiveness, adoption, implementation, and maintenance of these interventions is vital for future programs. Using the RE-AIM framework allows for the analysis of individual studies and identifies specific areas of strength and weakness in interventions [13]. Each dimension of the RE-AIM framework in relation to the 15 identified studies is discussed in more detail in the following sections.

Reach

Reach is set at an individual level and measures participation factors such as number, proportion, and representativeness of those taking part in an intervention [13]. RE-AIM research has shown that the majority of health care studies report sample size and proportion of participants who are willing to take part [30]. On the other hand, few studies report if the participant sample is representative of a particular patient population [31]. Of the 15 studies identified in this review, 93% (n=14) to 100% (n=15) reported the method used to identify the target population (n=14, 93%), inclusion criteria (n=15, 100%), exclusion criteria (n=14, 93%), sample size (n=15, 100%), and participation rate (n=14, 93%). Representativeness and use of qualitative methods to understand reach were not reported in any of the studies. Understanding the needs and overall demographic makeup of a population is vital to the success of an intervention, especially when dealing with technology-based programs. Issues such as age, socioeconomic background, ethnicity (language), and employment status may act as barriers for patients to engage in eHealth interventions and as such should be considered in the planning stage of a study [32]. Qualitative methods to understand a program’s reach and in particular representativeness should be incorporated during the planning stage. This can aid the researcher’s understanding of why certain patients do or do not engage in a study at the recruitment stage.

Effectiveness

The effectiveness dimension aims to understand the impact of an intervention including outcomes, potential negative effects, wider issues such as quality of life, and variation outcomes between subgroups. These factors are vital when moving from a research environment into a real-world setting [13]. In the 15 studies under review, the effectiveness indicators assessment of effect at the shortest and longest time points and patient attrition rate were reported in all papers. Reported to a lower degree were the indicators’ imputation procedures (n=2, 12%), presence of quality-of-life measures (n=8, 53%), and use of qualitative methods to understand outcomes (n=2, 12%). The lack of reported imputation procedures increases the chance of introducing bias and impacts the internal reliability of the study. Statistical methods, such as listwise deletion, dealing with missing data should be reported especially if interventions are to be applied in economic real-world environments [30]. Positive or negative impacts of an intervention on a patient’s quality of life should also be considered in health-related studies. Focusing on just the physiological outcomes does not give a full indication of the effectiveness of an intervention. Factors such as mental and social well-being should also be considered. Qualitative methods to understand outcomes were also limited with only 2 studies reporting use. Qualitative methods can complement the quantitative data in health studies by exploring the psychological behavior change element of a patient’s participation as well as their experience of the intervention overall [33].

Adoption

The adoption dimension moves away from the individual participant level and focuses on the settings and intervention agents. In particular, this refers to where the intervention was delivered and by whom [13]. Studies identified within this review were all undertaken within outpatient clinical care. The interventions were either delivered by the research team or health care professionals. Adoption is seldom reported in health-related studies, and this creates issues when the impact of an intervention is assessed or applied in a real-world setting. When adoption is fully reported and considered at the planning stage of a study, there is evidence that these interventions are more successful [34]. Making comparisons between settings, sites, and the teams delivering an intervention (differential adoption) can help understand good practice [34]. The 15 studies identified during this review found low levels of reporting across the 6 indicators. In fact, only the indicators’ level of expertise of the delivery agent (n=2, 12%) and adoption rate (n=1, 7%) were reported. Without this information, it is difficult to apply the methods studied in full clinical practice.

Implementation

The implementation dimension is set at the settings level and measures how reliable and committed the delivery agent was in applying the intervention as intended along with reported monetary cost and staff time [13]. Implementation is important as it provides details of why an intervention was changed from that initially proposed. The cost factor is an important consideration for organizations intending on implementing any intervention, and this needs to be considered against the interventions’ benefits. This dimension is generally not well reported in health behavior studies [13]. This review found that implementation was overall well reported in 3 out of the 5 indicators recorded in the identified studies. The indicators’ measure of cost (n=0, 0%) and use of qualitative methods to understand the implementation of the study (n=1, 7%) were poorly reported. As these studies were delivered in outpatient clinical care, a measure of cost would be required by health care management in a real-world setting. The use of qualitative methods would also provide a greater understanding of the implementation process and how staff perceived the intervention. This would help understand the training needs of staff and any requirements to amend the intervention.

Maintenance

The maintenance dimension is set at both the setting and individual levels. At the setting level, it refers to how well an intervention becomes part of routine health care. At an individual level, it is a measure of the long-term impact on the patient’s health. As most interventions are tested over a relatively short timescale, this is seldom reported in studies [13]. The RE-AIM evaluation framework identifies studies at or beyond 6 months as a measure of maintenance [35]. This review found that 53% (n=8) of study interventions were undertaken over 6 months or less. None are still in existence as far as the research team is aware. Only 1 study reported a modification to the intervention. None used qualitative methods to understand the long-term effect of the intervention. Stronger reporting across all 5 dimensions would help to understand why these interventions are no longer in operation. The use of qualitative methods for the maintenance dimension would have assisted in this understanding. Seven of the reviewed studies were either pilot or exploratory in design. By their very nature, these would only be planned over a short timescale.

Study Strengths and Limitations

After the initial search, all papers were double-screened at the initial title and abstract stage and full-text stage. The review was undertaken in accordance with the PRISMA 2020 statement [12].

The initial study protocol sets out to identify specific adult age categories within each of the identified studies, for example, 18-20 and 21-30 years. As these were not reported, the protocol was amended with the focus being on adults aged 18 years and older. The lead author conducted all of the data extraction with 40% (n=6) of papers being quality-checked by 3 of the coauthors.

Conclusions

This integrated systematic review sets out to comprehensively explore the reach, effectiveness, adoption, implementation, and maintenance of using activity trackers in clinical care to support physical activity or reduce sedentary behavior in adults diagnosed with chronic diseases. Applying the RE-AIM framework over the 5 main dimensions and 28 indicators identified a number of areas that were both well and poorly reported in studies. For this area of research, improved measurement and reporting of the dimensions adoption and maintenance are required with a focus on the settings element within studies. At an indicator level, the main area of underreporting and use was that of qualitative methods. Such methods would allow for detailed exploration of the experiences of both staff and patients at each stage of an intervention and over all RE-AIM dimensions. This would provide a better understanding of the use of activity trackers within clinical care for adults diagnosed with a chronic disease. Qualitative thematic hematic analysis involving one-to-one interviews or focus groups conducted at the prestudy, during the study, and poststudy stages with health care staff and participants would provide a greater understanding of the strengths and weaknesses of an intervention. The RE-AIM framework and findings from this systematic review also provide an insight into how such interventions could be applied in a clinical setting focusing on the dimensions of reach, effectiveness, adoption, implementation, and maintenance. The limited reporting of adoption and maintenance in studies makes it difficult to progress from a research study setting into the real-world health environment. Better reporting of these dimensions in studies would help clinicians in using activity trackers during patient care.