Effectiveness of Digital Interventions for Reducing Behavioral Risks of Cardiovascular Disease in Nonclinical Adult Populations: Systematic Review of Reviews

Introduction

Background

The National Health Service (NHS) Long Term Plan sets out the UK government’s vision for preventing health problems and supporting the self-management of conditions [1]. A major target is cardiovascular disease (CVD), which causes 28% of all deaths in the United Kingdom and is the largest cause of premature death in deprived areas [2,3]. England’s primary large-scale intervention for CVD prevention is the NHS Health Check program [1,2], which was introduced in 2009 [4]. It is one of the largest public health prevention programs in the world, with over 6 million people in England having a check between 2013 and 2018 [5]. The NHS Health Check is a CVD risk assessment, which should be offered every 5 years to all adults aged between 40 and 74 years with no pre-existing vascular condition. As a result, people with previously undiagnosed conditions can be put on a clinical pathway and those who are at risk of developing a condition can be offered lifestyle support and advice to manage their risk. In particular, cardiovascular risk can be reduced by modifying 4 types of behavior: diet [6], physical activity [7], smoking [8], and alcohol consumption [9].

A key pillar of the Long Term Plan is predictive prevention—the use of technology and digital tools to identify health risks, make early diagnoses, and support positive health behaviors of those most at need through targeted treatments [3]. Predictive prevention, including the NHS Health Check, involves risk communication and behavior change. Evidence shows that risk communication alone does not lead to behavior change [10-13]. Therefore, we need to support behavior change. Digital tools are an increasingly important part of that landscape, and digital behavior change interventions may be offered to people after their NHS Health Check to manage their risk by helping them modify their diet, physical activity, smoking, or alcohol consumption.

Digital tools may be used either to supplement face-to-face services or to replace them. Replacement is particularly germane, since there is anecdotal evidence that face-to-face services are increasingly being defunded. In addition, services may need to shift from face-to-face to digital in response to the COVID-19 pandemic, which occurred after we had completed the review. Providers may hope that digital tools will offer a low-cost solution, with the potential to reach more people than traditional face-to-face services; however, the research base needs to be evaluated to see if there is sufficient evidence [14].

Aims and Objectives

The first step is to establish whether digital interventions are effective. We also need to know which modes of delivery are most effective in order to allocate resources to develop the most promising digital tools or to know where research is needed, if the evidence base is lacking. In this systematic review of reviews, we aim to summarize the evidence on the effectiveness of digital interventions in improving dietary, physical activity, smoking, and alcohol consumption behaviors in nonpatient adult populations in high-income countries.

Methods

Overview

The study protocol was registered with PROSPERO (registration number: CRD42019126074). All deviations from the protocol are explained in the Methods section. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for reporting (see Multimedia Appendix 1 for the checklist) [15].

Data Sources

Relevant reviews were obtained through an internet-based search and a manual search. First, 4 internet-based databases (MEDLINE, Embase, PsycINFO, and Cochrane Reviews) were searched for peer-reviewed review articles published in English between January 1, 2009, and February 25, 2019. We limited our search dates, only starting in January 2009, to make our study manageable and also because we expected reviews published in the last decade to capture earlier papers. Publications were restricted to English due to the absence of translation expertise. Gray literature searches were conducted in OpenGrey, ProQuest Dissertations and Theses, Google, and targeted websites (see Multimedia Appendix 2 for search terms and the gray literature search strategy). More articles were identified by manual searches of the reference lists from excluded reviews of reviews. We did not search in study registries, as we were looking for systematic reviews. We did not conduct full hand searches or consult experts to ascertain the literature for pragmatic and logistical reasons.

Review Selection

The reviews were screened using a three-stage process. A total of 2 reviewers (NG and AY) examined titles and discarded reviews that did not meet the inclusion criteria (Textbox 1). Each reviewer then independently screened the abstracts of 10% of the remaining reviews to identify studies that potentially met the inclusion criteria. Interreviewer agreement on inclusion was also assessed. Reviewers disagreed on 11 of 41 decisions. All disagreements were resolved through discussion. AY screened all the remaining abstracts. The relevant review articles were then obtained in full and screened independently for eligibility by NG and AY. Any disagreement over eligibility was resolved through discussion with a third reviewer (TC or BR).

Study Quality Assessment

Review confidence was critically appraised independently by 2 reviewers (NG and AY) for a 10% subsample of the included publications, using the AMSTAR-2 tool [16]. The findings were discussed to check for consistency. The remaining articles were divided and assessed by NG or AY. Any uncertainty was resolved through discussion with a third reviewer (BR).

Data Extraction

Data were extracted using a standardized form (Multimedia Appendix 3). We extracted data on the following predefined review components: objective, population, inclusion and exclusion criteria, search date, included studies (number, type, and countries), follow-up, method of synthesis, results and findings, and comparator. A total of 2 reviewers (NG and AY) extracted the data independently for 10% of the publications. Discrepancies were resolved through discussion. The remaining publications were divided among both reviewers. During data extraction, we also noted information about the control condition and any information on a comparison of effectiveness of no-intervention versus active controls. Data on adherence and attrition were also recorded, where available. The data extraction form has been presented in Multimedia Appendix 3.

Analysis

We conducted a systematic narrative synthesis using extracted data from included articles. No statistical analyses were conducted and meta-analysis was not possible with the included articles. We have presented results in the following categories: diet, physical activity, diet and physical activity combined, smoking, alcohol consumption, and multiple areas of behavior (all combinations other than diet and physical activity). Where there were enough reviews, we grouped by mode of delivery, especially internet (including email and interventions that require accessing a website) and mobile phone (including apps and SMS text messaging interventions); social media was categorized separately from internet, mainly because there are enough papers to make the subdivision worthwhile but also because the social aspect may differentiate social media interventions from other forms of internet interventions, so that it is appropriately considered a subclass [17]. When reporting effect sizes, for Cohen d, Hedges g, and other measures of standardized mean difference (SMD), we followed the convention that 0.2 is a small effect size, 0.5 is a medium effect size, and 0.8 is a large effect size [18]. For risk ratios (RRs), we classified 1.22 as small, 1.86 as medium, and 3.00 as large; for odds ratios (ORs), these were 1.32, 2.38, and 4.70, respectively [19].

Results

Overview

Searches identified 1739 potentially relevant records. After screening the titles and abstracts, 154 articles were retrieved in full. An additional 36 articles were identified through hand searches and gray literature searches. In total, 94 reviews met the inclusion criteria (Figure 1). A list of reviews excluded after full-text screening is provided in Multimedia Appendix 4. We were unable to retrieve one gray literature item by May 13, 2019, and it was therefore excluded.

‎View this figure

Review Characteristics

The included reviews examined the effectiveness of digital interventions on diet only [20-23], physical activity only [24-46], diet and physical activity combined [47-76] (for some it was possible to extract separate results about diet and physical activity behaviors, whereas some reported more general results on weight loss outcomes), alcohol consumption [77-82], and smoking cessation [83-98]. A further 15 reviews examined the effectiveness of digital interventions on a combination of our 4 target areas of behavior (not diet and physical activity; again, sometimes it was possible to extract separate information for each target area but other times, the results were only reported in combination) [99-113]. Some reviews covered a number of areas of behavior because their research questions focused on a health outcome (eg, CVD) or a mode of intervention delivery (eg, internet interventions) rather than a behavior. Where extracting information on an area of behavior from a combination review was possible, we included the relevant data in the results for that area. For a breakdown of the reviews, with the number found in each area, for each mode of intervention and type of control see Multimedia Appendix 5.

The populations reviewed were general (nonclinical) adult populations. However, the diet and physical activity reviews were often restricted to populations of individuals with overweight or obesity. The alcohol reviews were often restricted to problem drinkers, defined with reference to local guidelines [77,79,80], questionnaire scores [81], or reduced productivity at work [78]. There was a range of modes of delivery, including mobile app, SMS text messaging, social media, pedometer, wearable, and interactive computer program. The reviews included both active and nonactive or minimal intervention controls, many pooling both types, but where possible, we tried to extract separate information about effectiveness of active compared with nonactive controls. We regarded the provision of educational materials as a nonactive or minimal control. More reviews included behavioral outcomes (such as fat consumption, fruit and vegetable consumption, physical activity, alcohol consumption, smoking cessation, and smoking abstinence) than health outcomes (such as weight loss, BMI, and waist circumference). Multimedia Appendix 6 provides the key characteristics of all included studies.

Review Confidence Ratings

The confidence rating of each review is presented in the study characteristics table in Multimedia Appendix 6, and a summary of confidence ratings is provided in Table 1; 84% (79/93) of reviews were rated as critically low. During the completion of the AMSTAR-2 tool, reviewers noted that most reviews failed to satisfy items 4 (including and justifying a publication language inclusion criterion) and 7 (providing a list of excluded reviews and justifications). Both items are considered critical for systematic reviews but not for meta-analyses. A modified rating was produced alongside the original AMSTAR-2 rating, which did not classify either of these flaws as critical, to see whether a variation among reviews would be revealed. However, little change was observed, with only 5 reviews moving from a rating of critically low to low.

Effectiveness of Interventions

Table 2 summarizes the effectiveness of the different types of interventions.

Diet

Review Characteristics

A total of 20 reviews reported findings on diet behaviors or a weight loss outcome that resulted from an intervention that only targeted diet. The breakdown of their characteristics is shown in Table S1.

Effectiveness of Digital Interventions on Diet Compared With Mixed (Active and Nonactive) Controls for Diet Behaviors

The effectiveness of digital interventions in improving results related to diet was at best mixed, for both behavioral and health outcomes (Table 3); where improvements were reported, the effect sizes were typically small. This was the case across all modes of delivery.

Of the 6 reviews covering internet interventions, 2 meta-analyses found small, favorable effects of internet interventions on dietary behavior, Cohen d=0.223 [110] and Cohen d+=0.20 [113]. A total of 4 narrative syntheses found mixed results with heterogeneous target outcomes [63,99,100,108]. The most common dietary target across the studies was fruit intake, which increased by approximately 1 serving per day [99].

Across 6 reviews of mobile interventions, there was some evidence of positive effects on dietary behaviors, especially fruit and vegetable intake, but no statistically significant effects on weight loss [21,23,60,99,104,111]. One of these reviews [99] reported a previous systematic review (not covered in this paper) that found no significant change in calorie intake or the consumption of sugar-sweetened beverages in 4 trials evaluating diet.

A total of 3 social media reviews had mixed findings on dietary outcomes, such as fruit and vegetable intake and fat intake, but the 2 reviews with meta-analyses found no differences in weight [65,75].

The effects of interactive computer interventions were equivocal and not clinically significant [22,74].

Effect of Digital Interventions on Diet Compared With Nonactive Controls

There was little evidence about the effectiveness of digital interventions on diet compared with no intervention or minimal intervention controls, and the available evidence was mixed (Table 3). One meta-analysis found small effects on fruit and vegetable intake (g=0.16) and dietary fat reduction (g=0.22) [109], but another meta-analysis found no statistically significant effects on dietary behaviors [73].

Effect of Digital Interventions on Diet Compared With Active Controls

The relative effectiveness of digital interventions on diet compared with active comparators was mixed, with approximately half of the studies or less in narrative syntheses, showing that digital interventions were effective compared with active controls (Table 3). This was true across all modes of delivery: internet [99], mobile [99,104], and combined interventions [99].

Sustainability of Effects on Diet at Follow-Up

Reviews included studies that ranged from single-contact interventions to 5-year follow-ups (Table 3). Relatively few reviews reported on follow-ups; of those that did, about half reported follow-ups in the medium term (3-6 months) and half in the long term (≥12 months).

Where reported, digital interventions were generally found to be effective for over 3 to 6 months. Several reviews have found positive results at 6 months [20,21,74]. However, this finding is not universal [111].

Long-term findings were more mixed in the 5 reviews that investigated them. Two reviews suggested promising effects at 12 months [108] and 18 months [20]. However, 3 reviews found no significant effects at 12 months [22], 24 months [111], and 30 months [74].

Physical Activity

Review Characteristics

We included the findings on physical activity–related outcomes from 45 systematic reviews. The breakdown of their characteristics is shown in Multimedia Appendix 6.

Effectiveness of Digital Interventions on Physical Activity Compared With Mixed (Active and Nonactive) Controls

The effectiveness of digital interventions was mixed across all modes of delivery, apart from mobiles, for which the evidence was consistently positive (Table 4).

Evidence for the effectiveness of internet interventions on physical activity has been mixed. In total, 5 out of 10 reviews were positive [26,29,33,108,113], including 2 meta-analyses that found small but significant effects of internet interventions: Cohen d=0.14 [29] and Cohen d+=0.24 [113]. However, there was significant heterogeneity across studies [29]. In contrast, 5 studies were not positive [35,47,63,100,110], one of which was very unfavorable, with only 1 of 9 (11%) studies demonstrating an effect of the intervention [100]. Two meta-analyses found that the effect of internet interventions on physical activity was not significant [63,110].

A total of 4 reviews of social media interventions were mixed. In total, 2 meta-analyses of social media interventions found no significant difference in changes in physical activity [65,75], and a narrative synthesis reported mixed results [76]. However, one narrative synthesis found that 76% of studies using web-based social networks had positive results for physical activity [56].

The results of the mobile interventions to improve health were more positive. In total, 8 of 10 (80%) narrative syntheses reported a majority of positive results [25,27,31,39,41,42,60,99], with one reporting an increase of 800 to 1104 steps per day [25]. One review noted that effective interventions used SMS text messaging communication or self-monitoring [42]. A review that was specifically on SMS text messaging reported that effect sizes were all greater than 0.20, and the median was 0.50, a medium effect size [27]. In contrast, 2 narrative syntheses reported that most mobile trials did not show any benefits [61,111].

There was mixed evidence of active gaming across 3 reviews. One review found that gamification has a positive impact on physical activity and found evidence that gamification can increase motivation to exercise [37]. However, another review found a positive effect on attendance but not on physical activity or BMI [46]. A third review found that active gaming did not support increases in either physical activity or attendance [43].

Computer-delivered interventions in physical activity behaviors did not have consistent results in either weight loss or weight maintenance trials [74].

In total, 4 of the 5 reviews assessing a variety of interventions found favorable results [40,45,53,99]. One of the narrative syntheses reported a wide range of values for improvement, from 1.5 to 153 extra minutes of physical activity a week and 1000 to 2600 steps per day [99]. A meta-analysis found that computer, mobile, and wearable technology led to a mean change of −41.28 minutes per day of sitting time (a reduction in sitting time) [45]. However, one meta-analysis found no difference between the experimental and control groups [34].

Effectiveness of Digital Interventions on Physical Activity Compared With Nonactive Controls

Compared with minimal controls, evidence for the effectiveness of digital interventions has been mixed.

For internet interventions, one review found favorable evidence [39], another found unfavorable evidence [24], and a third found mixed evidence [36].

The 3 reviews of mobile interventions have also provided mixed evidence. One narrative synthesis found that interventions were effective, with 4 studies (3 pre-post and 1 comparative) reporting increases of 800 to 1104 steps per day [25]. However, another study found that mobile interventions were not effective in increasing physical activity of moderate-to-vigorous intensity or in decreasing sedentary behavior [30]. Wearables were also not very effective, with only 1 of 3 (33%) studies comparing a pedometer with a minimal control showing increased physical activity [32].

Social media–based interventions increased the daily number of steps taken by 1530 steps per day [49]. However, they were not associated with energy expenditure, total physical activity, or moderate-to-vigorous physical activity.

There were small effect sizes for both computer-delivered interventions (g=0.16) [109] and interactive voice response–based interventions (g=0.254) [73].

Effectiveness of Digital Interventions on Physical Activity Compared With Active Controls

There were mixed results compared with active controls.

A meta-analysis of internet interventions found an increase in physical activity with an SMD of 0.25 compared with active controls [102].

In total, 3 reviews of mobile phones had active controls, and there were mixed results. A meta-analysis found that the use of a mobile phone app was associated with significant changes in body weight (−1.04 kg) and BMI (−0.43 kg/m²); however, there was no significant difference in physical activity between the 2 groups [38]. A narrative synthesis app was also not favorable for assessing changes in physical activity, with less than half of the studies showing a significant increase in physical activity in the intervention group [104]. However, another narrative synthesis of general mobile interventions found that most studies had interventions that led to changes in body weight, increases in step count, and increases in frequency of physical activity [44].

There were also promising results from a review on wearables: when an activity tracker is combined with a comprehensive weight loss program, it may provide superior short-term (≤6 months) results than a standard weight loss program in middle-aged or older adults (>30 years) [28].

Sustainability of Effects on Physical Activity at Follow-Up

There was little evidence on sustainability, as many physical activity studies did not have follow-up assessment postintervention or only had follow-ups relatively soon after the intervention end point.

There is some evidence that digital interventions can have sustained effects. A meta-analysis assessing combinations of digital technologies found that the pooled effects showed mean changes (reductions) at short (≤3 months), medium (3 to 6 months), and long-term follow-up (>6 months) of −42.42 minutes per day, −37.23 minutes per day, and −1.65 minutes per day, respectively [45]. A meta-analysis of internet interventions also found a small but significant effect on physical activity for follow-ups at least six months postintervention (Cohen d=0.11) [29]. The sustainability of internet interventions was also supported by a narrative synthesis that found that only 12 of 35 (34%) studies had follow-up assessments, which ranged from 7 weeks to 15 months postprogram; 10 out of 12 (83%) studies demonstrated successful maintenance of physical activity and/or secondary measures indicative of positive changes in physical activity; however, follow-up durations were primarily shorter: in 9 studies, follow-up was conducted at less than 12 months [26]. However, for mobile interventions, 2 reviews found evidence that effects tended to decrease in the long term [31], with effects disappearing after as little as 12 weeks [44].

Diet and Physical Activity (Weight Management)

Review Characteristics

A total of 35 reviews reported on both diet and physical activity. The breakdown of their characteristics is shown in Multimedia Appendix 6.

Effectiveness of Digital Interventions on Diet and Physical Activity (Weight Loss) Compared With Mixed (Active and Nonactive) Controls

Overall, digital interventions were generally found to be effective, with mobile phone interventions in particular having consistently positive results (Table 5).

Internet interventions were found to be somewhat effective. A total of 5 narrative syntheses found them to be effective in approximately half of the studies [47,62,99,100,108]. In total, 3 meta-analyses quantified weight loss and other health effects. A total of 3 reviews found a significant reduction in weight: mean difference –1.34 kg [102], SMD 2.13 kg [70], and a 0.68-kg additional weight reduction over a period of 3 to 30 months [99]. A fourth review found that internet interventions significantly reduced waist circumference (mean change −2.99 cm) [69]. However, stratified analysis suggested that internet interventions were effective when used in combination with in-person counseling (−1.93 kg), rather than as a substitute [98]. This finding was supported by a narrative synthesis, which reported that internet interventions were more effective when they were enhanced to offer more than just educational resources (several studies found medium effect sizes) [62].

There were mixed results regarding the effectiveness of social media interventions. A total of 2 reviews were favorable [49,56], including a meta-analysis that found that social media–based interventions reduced body weight by 1.01 kg, BMI by 0.92 kg/m², and waist circumference by 2.65 cm but did not find significant changes in body fat or body fat percentage [49]. A total of 2 narrative syntheses found effects that were small and not meaningful [61,76]. Two meta-analyses found no significant effects on the diverse primary outcomes of the studies or on body weight [65,75].

Most reviews of mobile phone interventions found that they were effective in achieving weight loss via diet and physical activity. One meta-analysis found a statistically significant medium effect size (Cohen d=0.430) in favor of mobile phone interventions [60]. Two others quantified the change in terms of body weight: mobile phone interventions were associated with significant changes in body weight of −1.44 kg and in BMI of −0.24 kg/m² compared with controls [59]; the weighted mean body weight change in intervention participants was −2.56 kg compared with −0.37 kg in controls [71]. These results were supported by 3 narrative syntheses, which found evidence that mobile interventions led to weight loss [50,68,99]. However, one review of apps to promote healthy lifestyles found that most trials did not show significant differences [104], and one of the reviews that looked at the use of apps for diet, physical activity, and sedentary behavior, which concluded that apps led to weight loss, also concluded that the apps were more successful when used alongside other intervention components than when used alone [68].

In total, 4 of 5 narrative syntheses that ranged over a variety of digital interventions found that around half of the interventions were effective compared with controls [48,51,53,54], with only one finding that a large majority (81%) reported a significant reduction in adiposity [99]. One of these studies found that most trials reported within-group weight loss, even when there was no difference between digital interventions and controls [54]. Another study compared the success of different modes of delivery, finding the highest proportion of successful trials involving SMS text messaging or email (67%), followed by online chat rooms (50%), web-based (48%), and self-monitoring with technology (43%) [48].

Effectiveness of Digital Interventions on Diet and Physical Activity (Weight Loss) Compared With Nonactive Intervention Controls

A total of 6 reviews that reported results for digital interventions for diet and physical activity to no intervention or minimal intervention controls all agreed that digital interventions were more effective [66,67,69,70,72,74]. Table 5 provides a summary of the results.

Internet interventions were more effective than nonactive controls according to 3 meta-analyses and 1 narrative synthesis [67,69,70,72]. Internet interventions led to a greater reduction in waist circumference (mean change –2.99 cm, 95% CI −3.68 to −2.30, I2=93.3% vs 2.38 cm, 95% CI 1.61 to 3.25, I2=97.2%) [69]; internet-delivered personal feedback led to greater weight loss (mean difference 2.14 kg) [70]; and self-directed internet interventions led to significantly more weight loss (mean difference −1.56 kg) and showed a significantly greater reduction in BMI (mean difference −0.41 kg/m²) [72].

Similar effects were observed for computer-based interventions, which led to greater weight loss at 6 months (mean difference −1.5 kg) and were superior to limiting weight regain (mean difference −0.7 kg) [74].

A narrative review of tailored internet interventions found that effect sizes ranged from very small to moderate [67]. This was supported by a meta-analysis that covered various digital interventions, which found small effect sizes favoring digital interventions for weight (g=0.34) and behavioral outcomes (g=0.17) [66].

Effectiveness of Digital Interventions on Diet and Physical Activity (Weight Loss) Compared With Active Controls

The effects of digital interventions on diet and physical activity (with regard to weight loss) compared with active controls were mixed. A total of 2 meta-analyses found no differences between web-based interventions and active offline interventions for weight loss outcomes [52,69]. One meta-analysis of a range of interventions found that standard active treatment was more effective for weight loss (g=0.31) [66]. However, one meta-analysis found that using the internet had a modest but significant additional weight loss effect compared with offline control groups (−0.68 kg; P=.03), with a subgroup analysis showing that the internet was effective compared with controls for achieving weight loss (weight change=−1.01 kg) but not weight maintenance [58]. The same meta-analysis also found that it was effective to use the internet as an adjunct to face-to-face care (−1 kg; P<.001) but that adverse effects on weight loss were found when it was used as a substitute (+1.27 kg; P=.01) [58].

Computer-based interventions led to less weight loss than in-person treatment (mean difference 2.1 kg) and were not superior to infrequent in-person treatment in limiting weight regain at 6 months [74].

Sustainability of Effects on Diet and Physical Activity (Weight Loss) at Follow-Up

Although technology-related health interventions may be effective, the maintenance of behavior is challenging [64], and there is insufficient evidence to determine the effectiveness of digital interventions on weight maintenance [54]. One review of internet interventions reported that, in studies finding significant weight reduction, the magnitude of weight change ranged from 1 to 6 kg after 6 months of follow-up [99].

Many studies included in the reviews had short follow-ups. Where longer follow-ups were reported, effectiveness typically diminished over time. Examples of the diminishing effects are clear in the 2 reviews. One meta-analysis of internet-delivered personal feedback found a greater reduction in BMI (mean difference 0.54 kg/m²) and waist circumference (2.81 cm) at 0 to 4 months follow-up than at later times compared with undefined control groups [72]. A narrative synthesis supports these findings, concluding from multiple high-quality randomized controlled trials (RCTs) that weight loss occurs for a short term through mobile interventions, with moderate evidence for the medium term [50]. Two other meta-analyses reported significantly greater weight loss in favor of digital interventions in the medium term only (<6 months): 2.13 kg [52] and 1.55 kg [58]. One meta-analysis assessed differences between trials of different lengths, reporting no differences between shorter and longer trials (<6 or ≥6 months) [59].

Smoking

Review Characteristics

There were 28 reviews on smoking (see Multimedia Appendix 6 for summary characteristics).

Effectiveness of Digital Interventions on Smoking Compared With Mixed (Active and Nonactive) Controls

Digital interventions were generally effective across different modes of delivery (Table 6).

Internet interventions were generally found to be effective (Table 6). A total of 3 relevant meta-analyses found small or small-to-medium effects [85,89,91]: pooled RR 1.39 [85]; RR 1.40 [92], and RR 1.43 [88] and a short-term (≤6 months) OR 1.29 (95% CI 1.12 to 1.50; P=.001) [91]; the latter showed that internet interventions were also successful for the individual outcomes of prolonged abstinence, that is, not smoking since a quit date (OR 1.43) and 30-day point prevalence abstinence, that is, not smoking one or more days before the follow-up (OR 1.75) [91]. However, one narrative synthesis noted that although the 3 studies that measured smoking cessation all showed significant intervention effects, they were all assessed to be of low quality [100].

The 3 reviews of mobile interventions that focused on SMS text messaging found it to be effective, again with small effects: Cohen d=0.14 [94]; OR 1.36 [96]; and OR 1.36 [98]. A fourth meta-analysis differentiated between specific outcomes and found a medium effect (RR 2.19) on biochemically verified continuous abstinence, and a small-to-medium effect (RR 1.51) on verified 7-day point prevalence of smoking cessation was pooled [111]. However, 2 reviews (including 1 meta-analysis) with information on mobile interventions in general did not find that they were effective [89,99].

A meta-analysis of computer-delivered interventions also found a small but significant effect size associated with studies addressing tobacco use (Cohen d=0.14) [112].

There is mixed evidence from reviews that cover a variety of digital technologies. One narrative synthesis [99] and one meta-analysis found that they were effective, with interventions increasing the odds of smoking cessation at 1 month (OR 1.70) [83]. However, another narrative synthesis that included both phone and internet interventions found that there were mixed results [86].

A total of 2 reviews of social media interventions with narrative syntheses both found favorable effects of the interventions on outcomes related to smoking cessation [56,93].

As seen above, separate reviews of internet, mobile, and computer-delivered interventions found small effects for each. One meta-analysis looked for differences in effect sizes between different modes of intervention and found no statistically significant evidence of any differences in effect sizes of internet interventions, intensive advice, telephone support, individual counseling, or group behavior therapy [89].

Effectiveness of Digital Interventions on Smoking Compared With Nonactive Controls

There were small effects of digital interventions compared with nonactive controls for all modes of delivery (Table 6). Meta-analyses found small effects for pooled modes of digital interventions compared with controls: 1 month OR 1.70 (k=13) [83]; pooled estimate for prolonged abstinence, RR 1.32 (k=60) [84], and pooled estimate for point prevalence abstinence, RR 1.14 [84]. Internet interventions were more effective than nonactive controls [88-90,92,97], with 3 meta-analyses finding small effects compared with nonactive controls: RR 1.15 [97], RR 1.46 [89], and RR 1.60 [88]. Computer-tailored interventions had effects sizes of g=0.16 for point prevalence outcomes and g=0.24 for prolonged abstinence [109]. Mobile phone–based interventions appeared to have similar effectiveness to minimal controls (RR 1.18) and were more effective than no intervention (typically either waiting list control or no further contact until follow-up) [89].

Effectiveness of Digital Interventions on Smoking Compared With Active Controls

Most reviews concluded that internet interventions were not more effective than active controls, with 3 reviews (including 2 meta-analyses) not finding differences [88,90,97]. This included a meta-analysis that found no significant effects of internet interventions compared with face-to-face counseling or telephone counseling [88]. However, one narrative synthesis was more positive about the greater effect of internet interventions compared with conventional ones [99].

There was no information on mobile interventions compared with active controls, but one meta-analysis with mixed controls noted that the summary effect sizes favored the treatment groups even when 18 of the 20 (90%) controlled trials used an active control and 12 (66%) of these active controls included some smoking-related content, including smoke-free websites, self-help guidebooks, and smartphone apps [94].

Sustainability of Effects on Smoking at Follow-Up

There was agreement that the effects of internet interventions were sustained for up to 6 months [83,87,91,95]. This was quantified as interventions increasing the 6-month abstinence by 17% [95], increasing the odds of cessation at the 6-month follow-up (OR 1.29) [83], and increasing the likelihood of prolonged abstinence (ie, not smoking since a quit date; OR 1.19) [91]. However, there was disagreement about whether there were significant effects at 12-month follow-ups, with 2 reviews in favor [87,89], one quantifying the quit rate at 12 months as 8% [89] but another reporting that the positive results of internet interventions were not generally maintained at 1-year follow-up assessments [103].

There is mixed evidence on the sustainability of the effects of mobile interventions. A total of 2 meta-analyses found that mobile interventions were effective in the medium term, with a quit rate of 13% [89] and effects on biochemical measures of quitting at 6 months (RR 1.83) [98]. However, a study of SMS text messages suggested that any effect was not sustained [96].

Regarding computer-delivered interventions, one meta-analysis found that computer programs increased the odds of cessation at the 3-month (OR 2.04), 12-month (OR 1.68), and 18-month follow-up (OR 1.83) [83]. A meta-analysis of computer-delivered interventions did not find evidence of differential effects depending on the length of follow-up or number of sessions [112].

A total of 2 reviews that considered a range of different technologies found evidence of sustained effects. A narrative synthesis of internet and mobile interventions found that the OR for 7-day abstinence at 6 months ranged from 1.6 to 2.7 [99]. A meta-analysis found that digital interventions increased the odds of smoking cessation at 3 months (OR 1.30), 6 months (OR 1.29), and 18 months (OR 1.83) postintervention [83].

In a pooled analysis of web-based and computer-based interventions, the smoking cessation rate was 14.8% in the intervention group and 14.3% in the control group at the short-term 3-month follow-up (P=.42), 11.7% and 7.0% at the midterm 6- to 10-month follow-up ( P<.001), and 9.9% and 5.7% at the long-term 12-month follow-up (P<.001), respectively [92].

One meta-analysis that covered multiple intervention types found that SMS text messaging had the highest OR (2.81) at the 1-month follow-up, followed by the use of computer programs at the 3-month follow-up (OR 2.04), 12-month follow-up (OR 1.68), and 18-month follow-up (OR 1.83) and websites at 6 months (OR 1.37), while a DVD intervention and integrated videotelephony did not increase the odds of cessation compared with no intervention [83].

Alcohol

A total of 13 papers covered alcohol [73,77-82,99,103,104,111-113]. Multimedia Appendix 5 presents a breakdown of the study characteristics.

Effectiveness of Digital Interventions on Alcohol Consumption Compared With Mixed (Active and Nonactive) Controls

In total, 7 of 8 (88%) reviews with mixed controls reported favorable evidence that digital interventions were effective in decreasing alcohol consumption (Table 7), although one review noted that the controls were just as effective [103]. A total of 2 meta-analyses reported small effect sizes (Cohen d=0.26 [112] and Cohen d=0.14 [113]). A narrative synthesis of interventions using novel technologies found that, in studies finding benefits and reporting compliance with drinking recommendation as an outcome, the OR for drinking within the recommended limit ranged from 1.7 to 3.7 (small/medium to medium/large effects) [99]. In contrast, the sixth review, which surveyed mobile phone interventions, reported that the results were inconclusive for alcohol reduction, and the authors declined to perform a meta-analysis because the results were self-reported and therefore at risk of overstating the benefits [111].

Effectiveness of Digital Interventions on Alcohol Consumption Compared With Nonactive Controls

In total, 4 of 6 (67%) reviews with nonactive controls found an effect of the intervention (Table 7). Of the 4 meta-analyses, 2 found a medium effect on alcohol consumption (g=0.44 [80] and g=0.44 [79]). A third found a small-to-medium effect on total alcohol consumption (Cohen d+=0.31), small effects on 2 other consumption measures, and a measure of frequency of heavy drinking episodes (Cohen d+=0.16-0.19) and no effect on drinking frequency [82]. The fourth review found no effect [73].

Two studies quantified the effect on alcohol consumption, finding a reduction in weekly consumption of 22 g of alcohol [80] and 23 g of alcohol [77], approximately 3 UK units. However, when studies with a high risk of bias were excluded, this number decreased to 11 g of alcohol (or 1.5 UK units) per week [77]. With regard to frequency of consumption, participants who were given digital interventions had less than one drinking day per month, which is fewer compared with no-intervention controls (moderate-quality evidence); had approximately 1 binge-drinking session less per month in the intervention group compared with no-intervention controls (moderate-quality evidence); and drank 1 unit per occasion, which is less than the no-intervention control participants (moderate-quality evidence) [77]. Participants in internet interventions were significantly more likely to adhere to low-risk drinking guidelines at the immediate posttreatment follow-up, compared with the no-intervention controls [80].

Only one review was negative. A narrative synthesis of mobile apps reported that there was a single RCT on alcohol reduction where, contrary to expectation, the frequency of drinking occasions was higher in the intervention group [104].

Effectiveness of Digital Interventions on Alcohol Consumption Compared With Active Controls

In total, 2 reviews that separated active controls found that there were no significant differences between the intervention and control groups [77,99], one of which was specifically compared with face-to-face controls (Table 7) [77].

Sustainability of Effects on Alcohol Consumption at Follow-Up

The effectiveness of the interventions seemed to decrease over time (Table 7). One meta-analysis, which reported small-to-medium effects in the short term (Cohen d+=0.16-0.19), found that in the medium to long term, there were small (Cohen d+=0.07-0.12), significant effects on all outcomes [82]. A total of 2 reviews reported that, for internet-based intervention studies where there were 3-, 6-, or 12-month follow-up data, no significant differences in effect remained in later follow-up [80,103]. In contrast, one review of computer- or mobile-delivered interventions found that positive differences in measures of drinking were seen at 1, 6, and 12 months [77], and one review of internet interventions found a medium effect size (g=0.39), lasting up to 6 or 9 months posttreatment, as compared with no intervention; the effects of the interventions beyond 9 months could not be assessed, but 2 studies in the review suggested that they had faded out by 12 months [79].

Other Combinations

Review Characteristics

A total of 11 reviews covered a number of areas of behavior [101-107,109,110,112,113]. The breakdown of their characteristics is shown in Multimedia Appendix 5.

Effectiveness of Other Digital Combination Interventions Compared With Mixed (Active and Nonactive) Controls

All 5 reviews of internet interventions concluded that they were effective in changing behavior (Table 8). A total of 3 meta-analyses found small effects: Cohen d=0.19 [105], Cohen d=0.139 [110], and Cohen d=0.16 [113]. However, one study reported that the effect sizes were heterogeneous [113]. The fourth meta-analysis quantified the effects on health outcomes, finding statistically significant reductions in systolic blood pressure (mean difference –2.66 mm Hg), diastolic blood pressure (mean difference –1.26 mm Hg), glycated hemoglobin level (mean difference –0.13%), and low-density lipoprotein cholesterol level (mean difference –2.18 mg/dL) [102].

Of the 3 reviews of mobile interventions, a review of SMS text messaging concluded that they were effective, but the narrative synthesis of apps was the only review in the other combinations category that did not find a preponderance of positive results [104]. The effect sizes for the SMS text messaging interventions were small (Cohen d=0.24 [101] and Cohen d=0.329 [107]), and the effect sizes in the meta-analysis were heterogeneous [107].

Other modes of intervention continued in the pattern of positive, albeit small, effects. Computer-delivered interventions had a small effect (Cohen d=0.20) [112]. Prompts delivered by SMS text messaging or email were effective in changing diet, physical activity, and smoking behaviors [106].

Relative Effect of Interventions by Number of Behaviors Targeted

A total of 3 meta-analyses that compared interventions targeting single versus multiple areas of behaviors found that both types of interventions were effective but they differed in whether they found interventions with single or multiple targets to be more effective. A total of 2 meta-analyses of internet-based interventions found that the effect size of interventions targeting a single area of health behavior was not significantly different to the effect size of those targeting multiple areas of health behaviors, but the numerical difference was in favor of single-area studies (Cohen d=0.146 vs Cohen d=0.121 for multiple-behavior studies [110]; Cohen d+=0.17 vs Cohen d+=0.12 [113]). However, the computer feedback–based meta-analysis found a trend for increased effect sizes across studies that intervened in 1 (g=0.15), 2 (g=0.21), and 3 (g=0.24) areas of behavior, but this trend did not continue with the one study that intervened in 4 areas (g=0.12) [109]. The review covered diet, physical activity, smoking cessation, and mammography.

Relative Effect of Intervention by Area of Behaviors Targeted

The review of SMS text messaging interventions found that interventions targeting smoking cessation and physical activity were more successful than interventions targeting other areas of health behavior, including alcohol and weight loss [107]. In contrast, a review of computer-delivered interventions found that interventions targeting smoking had statistically significantly lower effect sizes than those targeting alcohol (Cohen d=0.26 and 0.12, respectively) [112].

Effect of Other Combination Interventions Compared With Nonactive Controls

Digital interventions had their largest effect sizes when compared with nonactive controls; however, the effect sizes were generally small. Internet interventions had the largest effect size when compared with waitlists and placebos, but the effect sizes were small: Cohen d=0.28 [105] and Cohen d=0.22 [107]. Tailored websites had extremely small effect sizes when compared with no-treatment control conditions (Cohen d=0.07), and the effects were not statistically significant compared with nontailored print materials [110]. There were medium effect sizes for SMS text messaging interventions that employed no-treatment control groups (Cohen d=0.369) [107].

Effect of Other Combination Interventions Compared With Active Controls

The effects of digital interventions compared with active controls were very small or nonexistent.

A total of 2 meta-analyses found that there was an extremely small effect size (Cohen d=0.08) [110] or no significant difference [105] when internet-based interventions were compared with sophisticated print interventions. There were larger effects in internet-based interventions that combined internet application with human support (blended care) [102]. Computer-delivered interventions had very small effect sizes when the comparison condition was an active comparison (Cohen d=0.10); studies employing active treatments as the comparison condition mainly produced effect sizes close to zero [112]. SMS text messaging interventions had small effect sizes compared with active comparisons (Cohen d=0.226), which was statistically significantly smaller than their effect size compared with the no-treatment control groups [107].

Although digital feedback was effective, it seems that it was no more effective than feedback delivered by nondigital means, and 2 reviews that examined feedback interventions found that the medium did not affect behavior change (SMS text messaging, print, email, telephone, newspaper articles [106] and print, computer, telephone, etc [109]).

Sustainability of Effects at Follow-Up

There was mixed evidence on the sustainability of interventions. A meta-analysis of internet-based interventions found the largest effect size at 1 month to 4 months [105], but a meta-analysis of computer-tailored interventions found that effects peaked from 4 months to 12 months postbaseline [109]. A total of 3 meta-analyses found that the correlation between effect size and follow-up was not significant for internet-based [110], SMS text messaging [107], or computer-delivered interventions [112].

The effects did seem to decline in the long run, and the 2 meta-analyses that explicitly examined effectiveness 1 year postintervention (for internet-based and computer-delivered interventions) found that the effect size declined, although it remained statistically significant [102,109].

Adherence (Considered for All Behavioral Areas)

Typically, adherence data were not reported. Where reported, there were generally decreases in program usage over the intervention period [75]. Rates of attrition were variable and sometimes very high, for instance, in one diet review, it was 0% to 84% [61] and in 2 smoking reviews it was ≥60% [86] and 35% to 84% (median 70%) [93]. Bucking the trend, a couple of reviews reported high adherence to digital physical activity interventions [46,60]. Many reviews found a dose-response relationship, whereby the effectiveness of the intervention was positively associated with dietary usage [20,21,61], weight loss [47], smoking [87,90,93,103], and alcohol [79]. However, there was no unanimity, for instance, one combination review found that the attrition rate was lower in internet-based interventions than in face-to-face settings [103] and 2 others found no evidence that retention influenced outcomes [107,110].

Discussion

Principal Findings

We reviewed 94 systematic reviews and meta-analyses that examined the effectiveness of digital interventions in changing health-related behavior and improving health outcomes in the areas of diet, physical activity, diet and physical activity combined, alcohol consumption, and smoking cessation, alone or in any other combination. The effectiveness of digital interventions differed according to the area of health behavior reviewed. Small positive effects were evident in smoking- and alcohol-related interventions. Similar findings were observed in the combined diet and physical activity interventions, as well as in other outcome combinations. However, there was little evidence of the effectiveness of stand-alone diet interventions, and evidence of the effectiveness of physical activity interventions was mixed, with some consistently positive evidence for mobile interventions and some promising evidence for exergaming. Digital interventions were most effective in the short-to-mid term (approximately 3 to 6 months), but there was insufficient evidence about their long-term effect. Typically, they were more effective than no intervention. There is mixed evidence on their effectiveness compared with nondigital interventions.

Our secondary objective was to identify differences in effectiveness between the modes of delivery of digital interventions. We identified internet-based interventions to be one of the more effective interventions for each area studied, except for physical activity alone. Mobile interventions were particularly effective for diet and physical activity combined (medium effects), but they were also effective for alcohol and smoking (small effects) and physical activity alone. Social media interventions were not effective for diet and physical activity combined (weight loss interventions), they had mixed effects for diet, and there was limited evidence for other areas. Computer-delivered technologies had small effects for smoking and alcohol consumption, but the effects for diet and physical activity were mixed.

The effect sizes reported in the reviews were generally below the National Institute for Health Care Excellence (NICE) guidelines for effectiveness of interventions, although it was often difficult to compare results with NICE guidelines, as different measures were used.

For weight management, NICE guideline PH53 (Recommendation 13) states that commissioned lifestyle weight management programs should have at least a 60% completion rate and should be likely to lead to an average weight loss of at least 3%, with at least 30% of participants losing at least 5% of their weight [114]. It should be noted that this is based on all participants, that is, those who attend at least one session. For those completing the service, that is, attending at least 75% of sessions, the key performance indicator of 50% losing at least 5% has been set [115]. In contrast, most reviews reported changes in weight in kilograms or changes in BMI, rather than percentage weight loss. The highest effect sizes for BMI in our review were −0.92 kg/m² [49] and −0.43 kg/m² [38], which are extremely unlikely to represent a 3% or 5% weight loss in individuals with overweight or obesity. The largest changes in weight found in the reviews of digital interventions were −2.71 kg in one study [72] and −2.56 kg in one meta-analysis [71]. In comparison, the Hartman-Boyce evidence review that supports NICE guideline PH53 found an average effect size of −2.59 kg for face-to-face services at 12 months (intention-to-treat analysis) [116]. Most effect sizes from digital interventions did not reach the effect size of face-to-face services.

For smoking, the national outcome measure for stop smoking services is 4-week quits. Smokers attempting to stop without additional support generally have a success rate of 25% at 4 weeks for carbon monoxide–validated quits and a success rate of about 35% at 4 weeks for self-reported quits [117]. Therefore, to show an impact, services must achieve success rates equivalent to or in excess of these rates that smokers achieve without support. Patients who receive stop smoking service support (behavioral support and pharmacotherapy) are 3 times more likely to quit than those with no support [118], and there is a cessation rate of 35% for brief intervention services but only at the 4-week point [117]. It is difficult to compare the results of our review with these services because the outcomes of digital smoking interventions are often expressed as ORs for smoking cessation rather than cessation rates. The only available review that used cessation rates demonstrated a cessation rate of 14.8% [92]. This is lower than the observed rate from stop smoking services or brief advice, but the follow-up point was considerably later and cessation rates decreased over time.

For alcohol, the average weekly reduction in drinking from brief advice interventions is 20 g of alcohol (about 2.5 UK units) [119], so the reductions in weekly alcohol consumption achieved by digital alcohol interventions are comparable when including all interventions (22 g of alcohol or 3 UK units) but lower if one restricts attention to high-quality evidence (11 g of alcohol or 1.5 UK units) [77].

To the extent that the effectiveness of digital interventions is below the NICE guidelines, doctors and organizations should be cautious about recommending them to patients who would benefit from behavior change in the 4 areas of our review. However, it may be valuable to recommend them to people who refuse a face-to-face intervention. On the basis of the evidence that we found, digital interventions for weight loss should combine diet and exercise unless they are mobile interventions targeting physical activity.

For digital interventions for both smoking and combined diet and physical activity, there were studies showing significant health effects but not significant behavioral effects. This seems paradoxical. However, it is possible that even a small increase in physical activity or a small improvement in diet, which may not be statistically significant, can improve health markers over an intervention period, particularly in the most inactive and less fit individuals. Light physical activity is beneficial for health outcomes, including cardiometabolic risk factors [120]. We also noted that, although statistically significant, the health effects are small and possibly not meaningful.

Limitations

Owing to the rapid nature of our review, we did not perform full hand searches or consult experts. This may mean that we overlooked some reviews. At the other end of the spectrum, by reviewing reviews, there is the possibility that some studies were double counted if they were covered in more than one review.

Our ability to make inferences from the literature reviewed is limited for the following reasons.

Heterogeneity was consistently high across reviews. Heterogeneity of effects probably reflects heterogeneity of interventions, which could be a consequence of rapid advances in digital devices and systems. There were also heterogeneous outcome measures. As the reviews covered different types of digital interventions and outcome measures, it was difficult to make comparisons. Differing outcome measures may have differentially impacted the effectiveness of modes of intervention or the general effectiveness of interventions in the areas we investigated. For instance, smaller effect sizes were reported in studies addressing smoking use than in studies on alcohol consumption, possibly because studies addressing alcohol use tended to use reductions in drinking behavior as their outcome variable, while studies addressing smoking use tended to apply the more stringent standard of abstinence [112].

Follow-up times were relatively short, so we cannot know if behavior change would be sustained in the long term. Some trials only provided behavioral data, so we cannot be sure of health outcomes. A review of physical activity found that the average rate of sustained use of digital health interventions over 10 weeks was 50% [64]. This is consistent with the findings of another systematic review on physical activity apps, which concluded that apps are effective in the short term (up to 3 months) but not longer [121].

Most trials reported intent-to-treat analysis, and typically, adherence data were not reported. This makes it difficult to assess nonsignificant effects to determine whether they resulted from ineffectiveness of treatment or from nonadherence. Where attrition rates were reported, they were often high.

Anecdotally, digital interventions are being used both to supplement and replace face-to-face services. However, most reviews did not discriminate between these functions. In the domain of weight loss, 5 reviews reported enhanced effects on weight loss in interventions that incorporated personal contact or counseling [51,54,57,62,74]. One meta-analysis showed that infrequent in-person treatment was superior in limiting weight gain to computer-based interventions (mean difference 0.5 kg). Digital interventions that particularly benefit from involving people alongside are thought to include sensitive tailoring of feedback [57,62] and social support [54].

Confidence ratings were critically low in 79 of 93 reviews (85%). However, when isolated, those reviews that were rated critically low presented findings that were consistent with the overall findings: equivocal evidence on effectiveness for diet or physical activity outcomes but consistent findings of short-term effects for alcohol, smoking, and other combined outcomes.

Even when the AMSTAR-2 [16] ratings were moderated (so that justifying any publication language inclusion criterion and providing a list of justifications for excluding reviews were no longer considered critical flaws), 74 of 93 (80%) reviews were rated critically low. During the generation of confidence ratings, it was noted that many reviews failed to satisfy items 2 and 13. These are considered critical items for all review types. Item 2 specifies that, as a minimum, reviews state that a protocol containing research questions, search strategy, inclusion/exclusion criteria, and a risk of bias assessment was completed before conducting the review. Item 13 dictates that reviews should account for the risk of bias in individual studies when interpreting/discussing the results of the review. The inclusion of these items in the AMSTAR-2 rating may represent an aspiration to improve standards. Our AMSTAR-2 quality ratings are consistent with other evidence that suggests that it is possible to satisfy PRISMA standards yet still have poor methodological quality [122]. However, judging reviews according to such high standards, such that they are virtually all rated as being very low, masks the differences in quality.

Future Research Work

This is the first review of reviews on the effectiveness of digital interventions with such a large scope. It summarizes the state of our knowledge of digital interventions for health improvement behaviors in nonclinical populations. However, the literature could be developed to be more helpful for professionals and organizations who need to decide whether to promote digital interventions, which ones to promote, which areas of behaviors to promote them for, and who to promote them to.

For policy purposes, reviews with mixed controls are of limited use. It matters whether a digital intervention is being compared with no intervention or an active nondigital control. It also matters whether the intervention is a stand-alone digital intervention or whether digital is being used as an adjunct to face-to-face services. We cannot assume that a digital intervention that is successful as an adjunct will also be successful as a stand-alone intervention. Therefore, reviews are needed to separately summarize the evidence base for these different ways of using digital interventions. It could be helpful to have well-structured and coordinated reviews that collate a high-level picture for each area of behavior, which can be updated on a regular basis. We need comparisons with national measures of effectiveness, such as the NICE guidelines, to more easily influence policy.

We also need reviews that can help us determine which interventions are most effective. In the future, it would be helpful to conduct comparative research on the mode of delivery of digital interventions (including comparisons of effect sizes), so that we can determine the most promising interventions for further development. There was also a lack of evidence about the long-term effects of interventions, and more studies on the sustainability of behavior change after digital interventions are needed. It would be especially useful to have this information in comparison with active controls.

Professionals also need to know who should be recommended digital interventions. Therefore, it would also be useful to know whether effect sizes are consistent across various subgroups of the population or whether digital interventions have different effects in different subgroups. We were not specifically looking for this information, but the reviews that were surveyed had mixed findings about whether the effectiveness of digital interventions varied with sex and age. Three meta-analyses found no significant association between sex and effect size or age and effect size [102,109,110]. However, one study found that the effect of interventions declined as age increased [105]. There may also be sex-based differences in adherence, and it is plausible, though not proven, that adherence moderates effect size. One narrative synthesis found that women and middle-aged participants were more likely to use web-based intervention services than men and younger participants, and women were more adherent to the overall intervention [103]. There is even less information about differential effects according to socioeconomic status (SES). In the domain of smoking, one review found that the relative effectiveness of technology-based interventions appeared to be comparable between low- and high-SES groups [83].

The acceptability of digital interventions to their target users also warrants further study. In one review of digital interventions of addictive behaviors, participants expressed a preference for internet-based services because of the convenience and increased confidentiality, and individuals who might not otherwise seek treatment said they would consider an internet-based intervention [103].

Providers may be drawn to digital tools in the hope that they are cost-effective. While not the purpose of our review of reviews, we noted insufficient evidence in the reviews to draw any preliminary inferences about the cost-effectiveness of digital interventions. The evidence from the reviews was mixed. There was evidence in favor (internet-based health interventions [41]), evidence against (adaptive e-learning interventions [22]), and mixed evidence: 1 of 3 (33%) web-based interventions was cost-effective compared with in-person interventions at 6 months [52]. Cost-effectiveness may also depend on whether digital interventions supplement or replace face-to-face interventions. Cost-threshold analyses indicated that some form of electronic intervention is likely to be cost-effective when added to nonelectronic behavioral support, but there is substantial uncertainty with regard to determining the most effective (thus most cost-effective) type of electronic intervention, which warrants further research [84]. Future work will need to investigate cost-effectiveness to allocate resources to developing the most promising digital tools.

Conclusions

Our review of reviews summarizes the state of our knowledge of digital interventions for health improvement behaviors in nonclinical populations. We found positive but small effects for digital interventions that targeted diet and physical activity combined, greater effects—but still small—for smoking and alcohol consumption, and positive, medium-sized effects for mobile interventions for physical activity alone. More high-quality research is needed to assess the sustainability of the effects of digital interventions in the long term, the differences between modes of delivery for digital interventions, their effect on different population subgroups, their cost-effectiveness compared with existing behavior change approaches, and in particular whether they are better used as an adjunct to or replacement for face-to-face treatment. We need the answers to these questions to be able to make an informed decision about whether digital behavior change tools should be integrated into the NHS Health Check program.