This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (491) Web-based and Mobile Health Interventions (3297) Smoking Cessation (556)

Published on in Vol 22, No 10 (2020): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/16255, first published .
Tailored Web-Based Smoking Interventions and Reduced Attrition: Systematic Review and Meta-Analysis

Tailored Web-Based Smoking Interventions and Reduced Attrition: Systematic Review and Meta-Analysis

Tailored Web-Based Smoking Interventions and Reduced Attrition: Systematic Review and Meta-Analysis

Authors of this article:

Amika Shah1 Author Orcid Image ;   Michael Chaiton1, 2 Author Orcid Image ;   Dolly Baliunas1, 3 Author Orcid Image ;   Robert Schwartz1, 2 Author Orcid Image
Article Authors Cited by (3) Tweetations (4) Metrics

    Review

    1Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

    2Ontario Tobacco Research Unit, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

    3Office of Education, Centre for Addiction and Mental Health, Toronto, ON, Canada

    Corresponding Author:

    Michael Chaiton, PhD

    Dalla Lana School of Public Health

    University of Toronto

    155 College St

    Toronto, ON, M5S 2S1

    Canada

    Phone: 1 416 978 7096

    Email: michael.chaiton@camh.ca


    Background: The increasing number of internet users presents an opportunity to deliver health interventions to large populations. Despite their potential, many web-based interventions, including those for smoking cessation, face high rates of attrition. Further consideration of how intervention features impact attrition is needed.

    Objective: The aim of this systematic review is to investigate whether tailored web-based smoking cessation interventions for smokers are associated with reduced rates of attrition compared with active or passive untailored web-based interventions. The outcomes of interest were dropout attrition at 1-, 3-, 6-, and 12-month follow-ups.

    Methods: Literature searches were conducted in May 2018 and updated in May 2020 on MEDLINE (Medical Literature Analysis and Retrieval System Online), PsycINFO (Psychological Information), EMBASE (Excerpta Medica dataBASE), CINAHL (Cumulated Index to Nursing and Allied Health Literature), Scopus, and the Cochrane Tobacco Addiction Group Specialized Register with the following search terms: smoking cessation, tailored, or web- or internet-based. Included studies were published in English before or in May 2020 using a randomized controlled trial design. Studies were restricted to those with web-based delivery, a tailored intervention group, an untailored control group, and a reported outcome of smoking cessation. Studies were assessed for methodological quality using the Cochrane Risk of Bias tool. Two reviewers independently extracted the study characteristics and the number of participants lost to follow-up for each treatment group.

    Results: A total of 13 studies were included in the systematic review, of which 11 (85%) were included in the meta-analysis. Tailoring had no statistically significant effect on dropout attrition at 1-month (risk ratio [RR]=1.02, 95% CI 0.95-1.09; P=.58; I2=78%), 3-month (RR=0.99, 95% CI 0.95-1.04; P=.80; I2=73%), 6-month (RR=1.00, 95% CI 0.95-1.05; P=.90; I2=43%), or 12-month (RR=0.97, 95% CI 0.92-1.02; P=.26; I2=28%) follow-ups. Subgroup analyses suggested that there was a statistically significant effect of tailoring between the active and passive subgroups at 1-month (P=.03), 3-month (P<.001), and 6-month (P=.02) follow-ups but not at 12-month follow-up (P=.25).

    Conclusions: The results suggest that tailoring of web-based smoking cessation interventions may not be associated with reduced rates of dropout attrition at 1-, 3-, 6-, or 12-month follow-ups. Significant differences between studies that include untailored active and passive control groups suggest that the role of tailoring may be more prominent when studies include a passive control group. These findings may be because of variability in the presence of additional features, the definition of smokers used, and the duration of smoking abstinence measured. Future studies should incorporate active web-based controls, compare the impact of different tailoring strategies, and include populations outside of the Western countries.

    J Med Internet Res 2020;22(10):e16255

    doi:10.2196/16255

    Keywords

    internet; world wide web; smoking cessation; web-based intervention 


    Background

    Smoking is the leading cause of preventable death [1]. Annually, over 7 million deaths worldwide are attributed to cigarette smoking [2], leading to a global loss of 150 million disability-adjusted life years [3]. The majority of smokers want to quit [4]; however, it can take 30 or more attempts to successfully quit [5]. Although smoking cessation is difficult, the price of not quitting is high—over 50% of long-term smokers die from smoking [6,7]. Fortunately, evidence-based smoking cessation interventions can double or triple the chances that a quit attempt will result in long-term cessation [8]. However, most effective clinical interventions have been found to be more costly and to have lower reach compared with public health interventions [9].

    Web-Based Interventions

    Interventions delivered over the internet, termed web-based interventions, have the potential to increase the public health impact of smoking cessation interventions because of their widespread use and scalability [10]. Despite their potential, few web-based interventions succeed in delivering the full treatment intended because of high rates of attrition. In a seminal paper, Eysenbach [11] delineated 2 types of attrition in web-based interventions: nonusage and dropout. The former refers to users who do not use the web-based intervention, whereas the latter refers to users who do not complete the follow-up study procedures. Eysenbach [11] hypothesized that both types of attrition are related and may be explained by a common experience of the user losing interest. As such, research that contributes to the science of attrition is recommended to better understand the phenomenon of attrition in the context of trials of web-based interventions [11].

    Like many web-based interventions [11], web-based smoking cessation interventions often report high rates of dropout attrition [12], posing threats to the validity of evidence surrounding these interventions. Although studies on smoking cessation often employ intention-to-treat analyses, where those lost to follow-up are assumed to be smokers, high rates of dropout attrition increase the risk of attrition bias, which may lead to the underestimation of the effectiveness of web-based smoking cessation interventions. Such findings have been reported by systematic reviews of web-based smoking cessation interventions that have identified attrition bias as a critical challenge in assessing the effectiveness of these interventions [12-14]. In addition, bias caused by attrition makes it difficult to identify features of web-based smoking cessation interventions that are most effective in promoting smoking cessation. Thus, further consideration of the features that influence dropout attrition is needed to fully understand the impact of web-based smoking cessation interventions, the features that characterize effective interventions, and their mechanisms of action.

    Tailoring

    Tailoring is one feature that has received significant interest [13]. Tailored print smoking cessation materials have been found to be more effective than untailored materials [15,16], suggesting that this feature may be important for their web-based equivalents. Likewise, systematic reviews of web-based smoking cessation interventions have highlighted the importance of tailoring in promoting smoking cessation [12-14]. For instance, Taylor [12] investigated the effectiveness of interactive, tailored, and combined (tailored and interactive) web-based interventions compared with nonactive and active control interventions. Combined interactive and tailored interventions were found to be moderately more effective than nonactive controls (eg, printed self-help guide) but not more effective than active controls (eg, counseling sessions) [12]. The authors noted that many of the studies had high rates of attrition, making it difficult to assess the effectiveness of tailored web-based smoking cessation interventions [12]. Thus, the ongoing challenge of high dropout attrition presents a need to explore the effect of intervention design features, such as tailoring, on attrition [15,17].

    Objectives

    This study aims to investigate across randomized controlled trials (RCTs), whether tailored active web-based smoking cessation interventions reduce dropout attrition at 1-, 3-, 6-, and 12-month follow-ups compared with an untailored active or passive control. In the context of this review, we define the terms tailoring, active, and passive as follows.

    Tailoring

    The term tailoring is often used interchangeably with personalization and targeting; however, important differences exist among these 3 terms [17]. Personalization refers to materials that have been customized using an individual’s name, whereas targeting involves designing materials for a particular subgroup or population with one or more shared characteristics (eg, youth) [17]. In contrast, tailored materials refer to materials that are designed for the characteristics of a particular individual based on individual assessment (eg, quit date, level of motivation, and self-efficacy) [17]. These terms may be viewed as existing on a continuum ranging from generic to individually tailored [17], with each term varying in the level in which they consider and incorporate characteristics of the user into the intervention. Although we recognize the differences among these terms, for simplicity, we operationalize the term tailored as an umbrella term that encompasses personalization, targeting, and tailoring to the individual.

    Active

    The term active is defined as interventions or control groups that require more than one engagement by the user. The multiple engagements must be part of the intervention and not simply part of the study procedures. For instance, a study with a single tailored email that requires multiple follow-up procedures would not be considered active in the context of this review. The authors chose to focus on active interventions, as attrition is more likely to be an issue for interventions and studies that require multiple engagements over time, rather than a single engagement.

    Passive

    The term passive is defined as control groups that require a single engagement by the user. The authors chose to compare studies with either an active or passive control group to assess whether the effect of tailoring differs between studies that compare active tailored interventions to control groups that require similar (active) or different (passive) levels of engagement.


    Protocol and Registration

    This review is designed and reported in line with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) reporting guidelines (for checklist, see Multimedia Appendix 1) [18]. A protocol was developed to guide this review; however, the protocol was not registered. Details outlined in the protocol included the study rationale, research question, eligibility criteria, study selection process, outcomes of interest, and the processes followed in conducting the meta-analysis. No deviations were made from the protocol from start to the final review [18].

    Data Source and Search Strategy

    Searches were conducted using the following electronic databases: MEDLINE (Medical Literature Analysis and Retrieval System Online), PsycINFO (Psychological Information), EMBASE (Excerpta Medica dataBASE), CINAHL (Cumulated Index to Nursing and Allied Health Literature), Scopus, and the Cochrane Tobacco Addiction Group Specialized Register. Search strategies for each database were created using synonyms of the 4 main search concepts: (1) smoking cessation (outcome measure), (2) tailoring (intervention feature), (3) web- or internet-based (technology), and (4) randomized control trial (study design). A sample search strategy for MEDLINE is provided in Multimedia Appendix 2. An initial search was conducted in May 2018 for studies published in or before May 2018. The review was later updated in May 2020 to identify any studies published between May 2018 and May 2020. Search results were exported and managed on EndNote Web.

    Study Selection

    Two authors (AS and MC) independently reviewed papers retrieved from the search strategy by title and abstract. Rayyan, a web-based application for systematic reviews, was used to facilitate the screening process between the 2 reviewers. The inclusion criteria were that studies (1) employed any type of randomized controlled trial (RCT) design (eg, crossover trials, parallel trials, and factorial trials), (2) included a tailored web-based smoking cessation intervention, (3) had an untailored control intervention (both web-based or nonweb-based were accepted), and (4) assessed smoking cessation through any method (ie, point prevalence estimate, self-report, and biochemical validation) at least 1 month after the start of the intervention. Papers could be published in or before May 2020. Only peer-reviewed articles published in English with a study population of smokers were included in the analysis. Dissertations, poster abstracts, and studies that described lifestyle interventions (interventions that targeted multiple health behaviors) were excluded (Figure 1).

    Studies were selected based on the tailored intervention component. If the tailored component was not delivered online (ie, a computer-tailored letter that was printed and mailed to the participant), the study was excluded. Multicomponent interventions were included; however, if the tailored component was not delivered on the web (eg, tailored text messages accompanied by a web page), the study was deemed ineligible. These criteria were used to isolate the impact of tailoring and increase the comparability of the interventions included in the meta-analysis.

    Studies that met the exclusion criteria when reviewed by title and abstract were excluded. The remaining studies were moved on to the next stage, where they were assessed using the full text. Full-text assessments of studies were later conducted for eligibility, with reasons for exclusion recorded. Disagreements in study eligibility were resolved through discussions between the two reviewers (AS and MC).

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) diagram.
    View this figure

    Study Quality

    Studies were assessed for methodological quality using the Cochrane Risk of Bias Tool [19], with reasons for risk level recorded (Figure 2). Risk of bias assessment was conducted at the study level. Risk of bias assessments were considered during the data interpretation stage.

    ‎
    Figure 2. Risk of bias graph.
    View this figure

    Data Extraction and Analysis

    The following details were extracted from the included studies: author, year of publication, study location, sample, population characteristics, intervention, theoretical framework used, tailored components, control group used, follow-up period, and key findings. In addition, the reported number of individuals lost to follow-up for the intervention and control arms were extracted from each study and compared against the original sample of individuals allocated to the intervention and control arms. If missing data were related to the outcome of interest (dropout attrition), the authors were contacted for additional information. Risk ratios (RRs) and 95% CIs were used to measure the difference in attrition between tailored interventions and untailored control groups. All significance tests were 2-tailed, with P<.05 considered statistically significant. A meta-analysis was conducted with the software RevMan 5 (Cochrane Organization) using the Mantel–Haenszel fixed effects model to pool the RRs. A fixed effects model was chosen as the most conservative model with the expectation that there would be some heterogeneity in design. The I2 statistic was used to assess statistical heterogeneity at 3 upper limits (low, I2<25%; moderate, I2 between 25% and 75%; high, I2>75%) [20]. A sensitivity analysis was conducted to examine the effect of potential outliers on the meta-analysis results.


    Search Results

    The results of the search strategy are summarized in the PRISMA flowchart shown in Figure 1. When the first search was conducted in May 2018, the search strategy yielded a total of 1292 studies, of which 801 (62.0%) were duplicates. Upon screening the remaining 491 studies by title and abstract, 274 (55.8%) studies were excluded. The remaining 217 studies were later screened by the full text, and 204 (94.0%) studies were excluded for not meeting the inclusion criteria. From this initial search, 13 studies were included in the systematic review [21-33]. The search update conducted in May 2020 identified a total of 150 studies. After removing duplicates, 56.7% (85/150) of the studies were screened by title and abstract against the inclusion criteria, and 79% (67/85) studies were subsequently excluded. The remaining 21% (18/85) of the studies were screened by their full texts. Within these 18 studies, 2 (11%) appeared to meet the inclusion criteria, although neither study could be included in the analysis. At the time of the search update, the full text of Kahler [34] was not available, and thus, the details required for the meta-analysis could not be extracted. Additionally, due to unclear details regarding the control intervention, we excluded the study by Altendorf et al [35].

    Study Characteristics

    Details of the included studies, such as the author, year of publication, study location, sample, population characteristics, intervention, theoretical framework used, tailored components, control, and key findings, are outlined in Multimedia Appendix 3. The included studies were published from 2008 to 2018 in the following countries: the United States [21,24,26,27,32,33], Switzerland [22], Norway [23], Australia [25], France [28], Spain [29], Denmark [30], and the Netherlands [31]. Of the 13 included studies, 4 (31%) studies used an active control intervention, whereas 9 (69%) studies used a passive control intervention.

    The 14 trials included a total of 12,661 participants: 6538 (51.64%) randomized to a tailored intervention and 6123 (48.63%) to an untailored control intervention. Participants were recruited from a variety of settings, including postsecondary institutions [24,29], hospitals or clinics [26,27,31,32], a Quitline [24], and from a website [21,22,28,33]. Follow-up periods ranged from 1 to 18 months. Five studies reported follow-up at 1 month [23,25,30,32,33], 6 studies at 3 months [21-23,26,28,33], 9 studies at 6 months [21,22,26-31,33], 6 studies at 12 months [21,22,26-31,33], and 1 study at 18 months [21]. One study reported outcomes at 8, 20, and 30 weeks [26] and another at 7 months [27]. Of the 13 studies included in the systematic review, 7 (54%) studies [23-26,28,29,33] reported significant differences in smoking cessation between the tailored intervention and untailored control at one or more follow-up periods.

    Study Quality

    Most studies included in the analysis were found to have an unclear or high risk of performance bias (Figure 2). Moreover, 4 studies were found to be at high risk of attrition bias as characterized by studies with more than 50% of the sample randomized to a condition lost to attrition.

    Attrition

    All the 13 included studies reported dropout attrition, whereas only 1 (8%) study [25] reported nonusage attrition. Dropout attrition across the 13 studies ranged from 5% to 67% for tailored interventions and from 3% to 64% for untailored interventions (Table 1). Among the 13 studies that met the inclusion criteria, 2 (15%) studies [24,25] were excluded from the meta-analysis, as they reported dropout attrition at timepoints that could not be compared with any other study included in the analysis. Although Borland [25] included follow-ups at 1 and 7 months, attrition was not reported for the 1-month follow-up, and thus, this study was excluded from the meta-analysis. The exclusion of these 2 studies resulted in a total of 11 studies that were included in the meta-analysis. The results of the meta-analysis with respect to the follow-up periods of interest (1, 3, 6, and 12 months) are described in the following section.

    Table 1. Summary of attrition (n=13).
    1st author, referenceAttrition type reportedFollow-up periodIntervention sample, nIntervention loss to follow-up, n (%)Control sample, nControl loss to follow-up, n (%)
    An [24]

    		Dropout8 weeks25713 (5.1)2608 (3.1)

    		Dropout20 weeks25724 (9.3)26020 (7.7)

    		Dropout30 weeks25723 (8.9)26030 (11.5)
    Borland [25]

    		Dropout and nonusage1 month809Not reported422Not reported

    		Dropout and nonusage7 months809104 (12.9)42266 (15.6)
    Das [26]

    		Dropout3 months10516 (15.2)11120 (18.0)

    		Dropout6 months10513 (12.4)11115 (13.5)

    		Dropout12 months10513 (12.4)11114 (12.6)
    Graham [21]

    		Dropout3 months651151 (23.2)679142 (20.9)

    		Dropout6 months651168 (25.8)679154 (22.7)

    		Dropout12 months651180 (27.7)679187 (27.5)

    		Dropout18 months651201 (30.9)679213 (31.4)
    Harrington [27]Dropout6 months74898 (13.1)74089 (12.0)
    Mavrot [22]

    		Dropout3 months580290 (50.0)580251 (43.3)

    		Dropout6 months580353 (60.9)580331 (57.1)
    Nyguyen [28]

    		Dropout3 months1242639 (51.4)1236720 (58.3)

    		Dropout6 months1242667 (53.7)1236729 (59.0)

    		Dropout12 months1242732 (58.9)1236793 (64.2)
    Pardavila-Belio [29]Dropout6 months13319 (14.3)12211 (9.0)
    Skov-Ettrup [30]

    		Dropout1 month453106 (23.4)45264 (14.2)

    		Dropout6 months45367 (14.8)45275 (16.6)

    		Dropout12 months45384 (18.5)45266 (14.6)
    Smit [31]

    		Dropout6 months13289 (67.4)11974 (62.2)

    		Dropout12 months13257 (43.2)11955 (46.2)
    Tsoh [32]

    		Dropout1 month235 (21.7)195 (26.3)

    		Dropout2 months239 (39.1)196 (31.6)
    Wangberg [23]

    		Dropout1 month1029613 (59.6)1043640 (64.1)

    		Dropout3 months1029648 (63.0)1043644 (61.7)

    		Dropout12 months1029303 (29.4)1043300 (28.8)
    Westmaas [33]

    		Dropout1 month37698 (26.1)34095 (27.9)

    		Dropout3 months376134 (35.6)340112 (32.9)

    		Dropout6 months376156 (41.5)340136 (40.0)
    One-Month Follow-Up

    Of the 11 studies, 6 (55%) studies were included in the meta-analysis and 4 (36%) studies [23,30,32,33] reported attrition at 1-month follow-up. Only 1 (9%) study [23] compared a tailored active intervention with an untailored active control. There were no significant differences in the risk of attrition between tailored interventions and untailored controls at 1-month follow-up (RR=0.97, 95% CI 0.91-1.04; P=.41). Across the 3 studies [30,32,33] that compared a tailored active intervention with an untailored passive control, the risk of attrition was higher among tailored interventions at 1-month follow-up (RR=1.20, 95% CI 1.00-1.44; P=.04). This estimate was statistically significant, although it had high heterogeneity (I2=79%). The test for subgroup differences indicated significant differences between studies that included an active control and those that included a passive control at 1-month follow-up (χ21=4.7; P=.03).

    When the 4 studies [23,30,32,33] were pooled, no significant differences in the risk of attrition between tailored interventions and untailored controls were found at 1-month follow-up (RR=1.02, 95% CI 0.95-1.09; P=.58) with high heterogeneity (I2=78%) in the estimate (Figure 3). These findings suggest that tailoring had no effect on dropout attrition at 1-month follow-up.

    ‎
    Figure 3. Comparison of attrition between tailored and untailored conditions at 1-month follow-up by control type.
    View this figure
    Three-Month Follow-Up

    Of the 11 studies included in the meta-analysis, 6 (55%) reported attrition at 3-month follow-up [21-23,26,28,33]. A total of 27% (3/11) of the studies [21-23] compared a tailored active intervention with an untailored active control. When these 3 studies were pooled, tailored interventions were at a significantly higher risk of attrition compared with untailored controls (RR=1.06, 95% CI 1.00-1.13; P=.03) with moderate heterogeneity in the estimate (I2=42%). The remaining 3 studies [26,28,33] compared a tailored active intervention with an untailored passive control. Among these studies, the risk of attrition was lower for the tailored intervention than for the untailored intervention (RR=0.91, 95% CI 0.85-0.97; P=.006). This estimate was statistically significant with moderate heterogeneity (I2=43%). Significant differences between the active and passive subgroups were found at 3-month follow-up (χ21 =11.9; P<.001).

    When the 6 studies [23-25,28,30,35] that reported attrition at 3-month follow-up were pooled (Figure 4), no significant differences were found in the risk of attrition between the tailored interventions and their untailored controls (RR=0.99, 95% CI 0.95-1.04; P=.80) with high heterogeneity in the estimate (I2=73%). This finding suggests that tailoring had no effect on dropout attrition at 3-month follow-up.

    ‎
    Figure 4. Comparison of attrition between tailored and untailored conditions at 3-month follow-up by control type.
    View this figure
    Six-Month Follow-Up

    Of the 11 studies included in the meta-analysis, 9 (82%) reported attrition at 6-month follow-up [21,22,26-31,33]. Of these 9 studies, 2 compared a tailored active intervention with an untailored active control [21,22]. Differences in the risk of attrition between the tailored and untailored groups for these 2 studies were not significant (RR=1.09, 95% CI 1.00-1.19; P=.06) with no heterogeneity in the estimate (I2=0%). In addition, 7 of the 9 studies (78%) [26-31,33] compared a tailored active intervention with an untailored passive control. Among these 7 studies, there were no significant differences in the risk of attrition between the tailored intervention relative to the untailored control (RR=0.96, 95% CI 0.90-1.02; P=.16) with low heterogeneity in the estimate (I2=22%). The test for subgroup differences suggested that significant differences existed between the active and passive subgroups at 6-month follow-up (χ21=5.4; P=.02).

    When the 9 studies [21,22,26-31,33] that reported attrition at 6-month follow-up were pooled, no differences were found in dropout attrition between the tailored interventions and untailored controls (RR=1.00, 95% CI 0.95-1.05; P=.90) with moderate heterogeneity (I2=43%, Figure 5). This suggests that tailoring had no effect on the risk of attrition at 6-month follow-up.

    ‎
    Figure 5. Comparison of attrition between tailored and untailored conditions at 6-month follow-up by control type.
    View this figure
    Twelve-Month Follow-Up

    Of the 11 studies included in the meta-analysis, 6 (55%) reported attrition at 12-month follow-up [21,23,26,28,30,31]. Of these 6 studies, 2 studies [21,23] compared a tailored active intervention with an untailored active control. The pooled effect of these 2 studies found that there were no significant differences in the risk of attrition between tailored interventions and untailored controls (RR=1.02, 95% CI 0.91-1.13; P=.77) with no heterogeneity in the estimate (I2=0%).

    Four studies [26,28,30,31] reported attrition at 12-month follow-up that compared a tailored active intervention with an untailored passive control. The pooled effect of these 4 studies found a lower risk of attrition among the tailored intervention (RR=0.95, 95% CI 0.89-1.00; P=.07). This estimate was not statistically significant and had moderate heterogeneity (I2=36%). The test for subgroup differences found no significant differences between the active and passive subgroups at 12-month follow-up (χ21=1.3; P=.25).

    At 12-month follow-up, the pooled effect of the 6 studies [21,23,26,28,30,31] (Figure 6) demonstrated a lower risk of attrition for the tailored intervention compared with the untailored control (RR=0.97, 95% CI 0.92-1.02; P=.26). This estimate was not statistically significant and had moderate heterogeneity (I2=28%), suggesting that tailoring had no effect on dropout attrition at 12 months.

    ‎
    Figure 6. Comparison of attrition between tailored and untailored conditions at 12-month follow-up by control type.
    View this figure

    Sensitivity Analysis

    Given the high heterogeneity of the results found for attrition at 1-, 3-, and 6-month follow-ups, a post hoc sensitivity analysis was conducted. Through visual inspection of the forest plots, the study by Westmaas [33] was suspected to be an outlier, and thus, this study was removed for the sensitivity analysis. Without the study by Westmaas [33] included in the meta-analysis, tailoring was found to have a higher risk of dropout attrition at 1-month follow-up (RR=1.03, 95% CI 0.96-1.11; P=.38) with high heterogeneity (I2=86%). Tailoring was found to have a lower risk of attrition both at 3-month follow-up (RR=0.99, 95% CI 0.94-1.03; P=.62) with high heterogeneity (I2=78%) and at 6-month follow-up (RR=0.99, 95% CI 0.94-1.05; P=.78) with moderate heterogeneity (I2=49%, Multimedia Appendix 4). As none of these estimates were statistically significant, the sensitivity analysis suggests that tailoring did not have a statistically significant effect on attrition at 1-, 3-, and 6-month follow-ups.


    Principal Findings

    This study sought to understand the impact of tailoring web-based interventions on attrition among studies using active and passive control groups. Although several reviews have investigated the efficacy of web-based smoking cessation interventions [13-15,36], to our knowledge, this is the first systematic review that compared the rate of attrition of tailored and untailored web-based smoking cessation interventions. Our review found no differences in the likelihood of attrition between tailored web-based interventions and untailored controls at 1, 3, 6, or 12-month follow-ups.

    The results of this systematic review align with the findings of previous studies. Strecher [37] conducted a path analysis of tailoring in web-based smoking cessation interventions and found that the relationship between tailoring depth and smoking cessation was weakly mediated by longitudinal engagement. Moreover, in a trial of a web-based smoking cessation program, the number of web pages opened (a measure of engagement) did not predict 12-week cessation [38]. Thus, the results of this review, as well as previous studies, suggest that retention may not mediate the impact of tailoring on smoking cessation in the long term.

    When considering the engagement level (active vs passive) of the control intervention, tailoring was found to have no effect early on at 1-month follow-up but increased attrition by the 3-month follow-up for studies with an active control. This increase in attrition, however, was diminished at 12-month follow-up. These results suggest that for studies with an active control group, tailoring had no effect on attrition in the long term at 12-month follow-up. The opposite was found to be true among studies that compared a tailored active intervention with an untailored passive control group, where tailoring was associated with increased attrition at 1-month follow-up and decreased attrition at 3-month follow-up. This effect diminished over time, with no effect of tailoring on attrition at 12-month follow-up. When stratified by the type of control intervention, significant differences in the effect of tailoring were found between the active and passive subgroups at 1-, 3-, and 6-month follow-ups.

    A potential explanation for the increased attrition for studies with an active control is that participants in the active tailored intervention group may derive benefits from the intervention early on and, thus, seize their participation before the end of the study. For studies with a passive untailored control, the opposite effect may have been observed, as there may be fewer other components to maintain interest in the intervention and in the study. Additional research is needed, however, to investigate this effect.

    An important finding of this review was that only a few studies used an active control group (n=4), which suggests a lack of RCTs that compare web-based interventions with active control groups. The lack of active control groups may stem from the application of traditional RCT designs, well suited for investigating drug efficacy, onto digital health interventions [39]. Although robust methods such as the RCT are important, web-based interventions often have multiple active ingredients, including content and technology features. As such, there is a need to consider what the active ingredients of the technology are hypothesized to be to isolate them between the intervention and the control.

    Another notable result was that all studies included in the review took place in the Western countries—a finding that has been reported by a previous review [36]. Given that developing countries continue to be targeted by tobacco companies [40], face difficulties in adopting tobacco strategies [41], and are projected to account for 80% of smoking-related deaths in the next century [42], studies are needed to evaluate the effectiveness of web-based smoking cessation interventions in populations outside of the Western countries. Tailoring interventions for these populations may require consideration of different variables (ie, cultural factors) and have unique impacts on attrition and effectiveness.

    Implications for Future Work

    The findings of this review highlight the importance of selecting an appropriate control condition when evaluating the effectiveness of web-based smoking cessation interventions and identifying the important design components of these interventions. Where possible, web-based interventions should be compared with interventions that require the same amount of engagement to isolate the hypothesized active ingredients of the intervention. Without appropriate control interventions, studies may risk misrepresenting the benefits and mechanisms of intervention features, such as tailoring.

    In this study, tailoring was intentionally operationalized as a unitary construct [43] to investigate the impact of tailoring on attrition. Although beyond the scope of this review, future research is needed to explore more specific questions related to tailoring, such as the impact of various tailoring strategies on the effectiveness of web-based smoking cessation interventions. The findings of this review suggest, however, that an investigation of specific tailoring strategies may be difficult because of unclear reporting of tailoring among the included studies. Future studies on tailored web-based smoking cessation interventions should more clearly outline the tailored components using the tailoring reporting standards proposed by Harrington [44]. Reporting in accordance with these standards will facilitate more nuanced analyses of the differential impact of various tailoring strategies.

    Limitations

    The findings of this review should be interpreted with caution because of several limitations. First, multiple studies included in the review were found to be of unclear or high risk of performance bias. Second, the high heterogeneity in the intervention type and components, as well as the study design, made it difficult to pool the data on attrition and generalize the results of the study. Indeed, some studies included in this review had interventions that were both tailored and interactive [21-24,27,32]. This is an important limitation of this study, as interactivity can include personalization but not in all cases. Although previous reviews have grouped tailoring and interactivity [14], we sought to isolate the impact of tailoring and, thus, did not include interactivity. Some studies also had features in addition to tailoring and interactivity, such as social support or coaching, which could have impacted attrition. Third, in this review, we focused on dropout attrition and thus our findings may not capture the impact of tailoring on nonusage attrition. As only 1 study included in this review reported nonusage attrition, we anticipate that such analyses may be difficult to perform without improvements in reporting. Fourth, this review did not account for user acceptance and experience of technologies, which may have affected attrition, particularly in the short term. Finally, given that technologies are rapidly being developed both for research and in the consumer market, this review may have missed web-based interventions that were publicly available, especially outside of the Western countries. The restriction on published RCT studies in the English language may have contributed to the lack of interventions delivered internationally.

    Conclusions

    This study aimed to investigate the impact of tailoring on attrition in web-based smoking cessation interventions. A systematic review of the literature yielded 14 RCTs that compared tailored web-based interventions with untailored control interventions, with 4 (29%) of the studies using an active control intervention and 10 (71%) studies using a passive control intervention. A meta-analysis of attrition reported by 86% (12/14) of the included studies found no effects of tailoring on attrition at 1, 3, 6, and 12-month follow-ups. The findings of this review suggest that tailoring may not be associated with reduced rates of attrition in web-based smoking cessation studies in the long term, although the high heterogeneity of effects indicates that the findings should be interpreted with caution. Future studies that incorporate active web-based controls, compare the impact of different tailoring strategies, and include populations outside of the Western countries are needed. Moreover, although beyond the scope of this review, future reviews may consider the impact of other technology features on attrition. It is with a greater understanding of how intervention features impact attrition that technologies may be better designed to retain users who may benefit from web-based smoking cessation interventions.

    Acknowledgments

    This review was conducted as part of the Research on Advancing Cessation Treatment project, which was funded by the Ontario Ministry of Health and Long-Term Care’s Health System Research Fund (Canada).

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) checklist.

    PDF File (Adobe PDF File), 111 KB
    ‎
    Multimedia Appendix 2

    Sample search strategy (MEDLINE: Medical Literature Analysis and Retrieval System Online).

    PDF File (Adobe PDF File), 43 KB
    ‎
    Multimedia Appendix 3

    Characteristics of the included studies.

    DOCX File , 32 KB
    ‎
    Multimedia Appendix 4

    Sensitivity analysis of tailoring at 1-, 3-, and 6-month follow-ups.

    PDF File (Adobe PDF File), 630 KB
    1. WHO Report on the Global Tobacco Epidemic, 2013: Enforcing Bans on Tobacco Advertising, Promotion and Sponsorship. World Health Organization. 2013.   URL: https://apps.who.int/iris/bitstream/handle/10665/85380/9789241505871_eng.pdf?sequence=1 [accessed 2019-08-05]
    2. Tobacco. World Health Organization.   URL: http://www.who.int/news-room/fact-sheets/detail/tobacco [accessed 2019-08-05]
    3. GBD 2015 Tobacco Collaborators. Smoking prevalence and attributable disease burden in 195 countries and territories, 1990-2015: a systematic analysis from the Global Burden of Disease Study 2015. Lancet 2017 May 13;389(10082):1885-1906 [FREE Full text] [CrossRef] [Medline]
    4. Fong GT, Hammond D, Laux FL, Zanna MP, Cummings KM, Borland R, et al. The near-universal experience of regret among smokers in four countries: findings from the International Tobacco Control Policy Evaluation Survey. Nicotine Tob Res 2004 Dec;6 Suppl 3:S341-S351. [CrossRef] [Medline]
    5. Chaiton M, Diemert L, Cohen JE, Bondy SJ, Selby P, Philipneri A, et al. Estimating the number of quit attempts it takes to quit smoking successfully in a longitudinal cohort of smokers. BMJ Open 2016 Jun 9;6(6):e011045 [FREE Full text] [CrossRef] [Medline]
    6. Doll R, Peto R, Boreham J, Sutherland I. Mortality from cancer in relation to smoking: 50 years observations on British doctors. Br J Cancer 2005 Feb 14;92(3):426-429 [FREE Full text] [CrossRef] [Medline]
    7. Lam T. Absolute risk of tobacco deaths: one in two smokers will be killed by smoking: comment on "Smoking and all-cause mortality in older people". Arch Intern Med 2012 Jun 11;172(11):845-846. [CrossRef] [Medline]
    8. Cummings KM, Hyland A. Impact of nicotine replacement therapy on smoking behavior. Annu Rev Public Health 2005;26:583-599. [CrossRef] [Medline]
    9. Abrams DB, Orleans CT, Niaura RS, Goldstein MG, Prochaska JO, Velicer W. Integrating individual and public health perspectives for treatment of tobacco dependence under managed health care: a combined stepped-care and matching model. Ann Behav Med 1996;18(4):290-304. [CrossRef] [Medline]
    10. Norman CD, McIntosh S, Selby P, Eysenbach G. Web-assisted tobacco interventions: empowering change in the global fight for the public's (e)Health. J Med Internet Res 2008 Nov 25;10(5):e48 [FREE Full text] [CrossRef] [Medline]
    11. Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    12. Taylor GM, Dalili MN, Semwal M, Civljak M, Sheikh A, Car J. Internet-based interventions for smoking cessation. Cochrane Database Syst Rev 2017 Sep 4;9:CD007078 [FREE Full text] [CrossRef] [Medline]
    13. Shahab L, McEwen A. Online support for smoking cessation: a systematic review of the literature. Addiction 2009 Nov;104(11):1792-1804. [CrossRef] [Medline]
    14. Graham AL, Carpenter KM, Cha S, Cole S, Jacobs MA, Raskob M, et al. Systematic review and meta-analysis of internet interventions for smoking cessation among adults. Subst Abuse Rehabil 2016;7:55-69 [FREE Full text] [CrossRef] [Medline]
    15. Kreuter MW, Strecher VJ, Glassman B. One size does not fit all: the case for tailoring print materials. Ann Behav Med 1999;21(4):276-283. [CrossRef] [Medline]
    16. Strecher VJ. Computer-tailored smoking cessation materials: a review and discussion. Patient Educ Couns 1999 Feb;36(2):107-117. [CrossRef] [Medline]
    17. Geraghty AW, Torres LD, Leykin Y, Pérez-Stable EJ, Muñoz RF. Understanding attrition from international Internet health interventions: a step towards global eHealth. Health Promot Int 2013 Sep;28(3):442-452 [FREE Full text] [CrossRef] [Medline]
    18. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    19. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane collaboration's tool for assessing risk of bias in randomised trials. Br Med J 2011 Oct 18;343:d5928 [FREE Full text] [CrossRef] [Medline]
    20. Fletcher J. What is heterogeneity and is it important? Br Med J 2007 Jan 13;334(7584):94-96 [FREE Full text] [CrossRef] [Medline]
    21. Graham AL, Cobb NK, Papandonatos GD, Moreno JL, Kang H, Tinkelman DG, et al. A randomized trial of internet and telephone treatment for smoking cessation. Arch Intern Med 2011 Jan 10;171(1):46-53 [FREE Full text] [CrossRef] [Medline]
    22. Mavrot C, Stucki I, Sager F, Etter J. Efficacy of an internet-based, individually tailored smoking cessation program: a randomized-controlled trial. J Telemed Telecare 2017 Jun;23(5):521-528. [CrossRef] [Medline]
    23. Wangberg SC, Nilsen O, Antypas K, Gram IT. Effect of tailoring in an internet-based intervention for smoking cessation: randomized controlled trial. J Med Internet Res 2011 Dec 15;13(4):e121 [FREE Full text] [CrossRef] [Medline]
    24. An LC, Klatt C, Perry CL, Lein EB, Hennrikus DJ, Pallonen UE, et al. The RealU online cessation intervention for college smokers: a randomized controlled trial. Prev Med 2008 Aug;47(2):194-199. [CrossRef] [Medline]
    25. Borland R, Balmford J, Benda P. Population-level effects of automated smoking cessation help programs: a randomized controlled trial. Addiction 2013 Mar;108(3):618-628. [CrossRef] [Medline]
    26. Das S, Hickman NJ, Prochaska JJ. Treating smoking in adults with co-occurring acute psychiatric and addictive disorders. J Addict Med 2017;11(4):273-279 [FREE Full text] [CrossRef] [Medline]
    27. Kathleen FH, Young-Il K, Meifang C, Rekha R, Maria P, Rajani S, et al. Web-based intervention for transitioning smokers from inpatient to outpatient care: an RCT. Am J Prev Med 2016 Oct;51(4):620-629. [CrossRef] [Medline]
    28. Thanh VN, Guignard R, Lancrenon S, Bertrand C, Delva C, Berlin I, et al. Effectiveness of a fully automated internet-based smoking cessation program: a randomized controlled trial (STAMP). Nicotine Tob Res 2019 Jan 4;21(2):163-172. [CrossRef] [Medline]
    29. Pardavila-Belio MI, García-Vivar C, Pimenta AM, Canga-Armayor A, Pueyo-Garrigues S, Canga-Armayor N. Intervention study for smoking cessation in Spanish college students: pragmatic randomized controlled trial. Addiction 2015 Oct;110(10):1676-1683. [CrossRef] [Medline]
    30. Skov-Ettrup L, Dalum P, Bech M, Tolstrup JS. The effectiveness of telephone counselling and internet- and text-message-based support for smoking cessation: results from a randomized controlled trial. Addiction 2016 Jul;111(7):1257-1266. [CrossRef] [Medline]
    31. Smit ES, Hoving C, Cox VC, de Vries H. Influence of recruitment strategy on the reach and effect of a web-based multiple tailored smoking cessation intervention among Dutch adult smokers. Health Educ Res 2012 Apr;27(2):191-199. [CrossRef] [Medline]
    32. Tsoh JY, Kohn MA, Gerbert B. Promoting smoking cessation in pregnancy with video doctor plus provider cueing: a randomized trial. Acta Obstet Gynecol Scand 2010;89(4):515-523 [FREE Full text] [CrossRef] [Medline]
    33. Westmaas JL, Bontemps-Jones J, Hendricks PS, Kim J, Abroms LC. Randomised controlled trial of stand-alone tailored emails for smoking cessation. Tob Control 2018 Mar;27(2):136-146. [CrossRef] [Medline]
    34. Kahler CW, Cohn AM, Costantino C, Toll BA, Spillane NS, Graham AL. A digital smoking cessation program for heavy drinkers: pilot randomized controlled trial. JMIR Form Res 2020 Jun 8;4(6):e7570 [FREE Full text] [CrossRef] [Medline]
    35. Altendorf M, Hoving C, Van Weert JC, Smit ES. Effectiveness of message frame-tailoring in a web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2020 Apr 3;22(4):e17251 [FREE Full text] [CrossRef] [Medline]
    36. Do HP, Tran BX, Le Pham Q, Nguyen LH, Tran TT, Latkin CA, et al. Which eHealth interventions are most effective for smoking cessation? A systematic review. Patient Prefer Adherence 2018;12:2065-2084 [FREE Full text] [CrossRef] [Medline]
    37. Strecher VJ, McClure J, Alexander G, Chakraborty B, Nair V, Konkel J, et al. The role of engagement in a tailored web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2008 Nov 4;10(5):e36 [FREE Full text] [CrossRef] [Medline]
    38. Strecher VJ, Shiffman S, West R. Randomized controlled trial of a web-based computer-tailored smoking cessation program as a supplement to nicotine patch therapy. Addiction 2005 May;100(5):682-688. [CrossRef] [Medline]
    39. Catwell L, Sheikh A. Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Med 2009 Aug;6(8):e1000126 [FREE Full text] [CrossRef] [Medline]
    40. Sebrié E, Glantz SA. The tobacco industry in developing countries. Br Med J 2006 Feb 11;332(7537):313-314 [FREE Full text] [CrossRef] [Medline]
    41. World Health Organization. Economics of Tobacco Toolkit. Assessment of the Economic Costs of Smoking Geneva, Switzerland: WHO 2011. World Health Organization.: World Health; 2020.   URL: https://apps.who.int/iris/bitstream/handle/10665/44596/9789241501576_eng.pdf
    42. WHO Report on the Global Tobacco Epidemic 2008: The MPOWER Package. World Health Organization. 2008.   URL: https:/​/apps.​who.int/​iris/​bitstream/​handle/​10665/​43818/​9789241596282_eng.​pdf;jsessionid=DC45BB004C41FD035C9C0E3B85AB10C7?sequence=1 [accessed 2019-08-05]
    43. Hawkins RP, Kreuter M, Resnicow K, Fishbein M, Dijkstra A. Understanding tailoring in communicating about health. Health Educ Res 2008 Jun;23(3):454-466 [FREE Full text] [CrossRef] [Medline]
    44. Harrington NG, Noar SM. Reporting standards for studies of tailored interventions. Health Educ Res 2012 Apr;27(2):331-342. [CrossRef] [Medline]

    ‎
    MEDLINE: Medical Literature Analysis and Retrieval System Online
    PRISMA: Preferred Reporting Items for Systematic reviews and Meta-Analyses
    RCT: randomized controlled trial
    RR: risk ratio

    Edited by G Eysenbach; submitted 14.09.19; peer-reviewed by R Bajpai, A Lazard, M Kwon, S Cha; comments to author 27.11.19; revised version received 24.04.20; accepted 03.06.20; published 19.10.20

    Copyright

    ©Amika Shah, Michael Chaiton, Dolly Baliunas, Robert Schwartz. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 19.10.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.