The Effectiveness of Smartphone App–Based Interventions for Assisting Smoking Cessation: Systematic Review and Meta-analysis

Introduction

Smoking is a leading cause of premature death globally. Among tobacco-related deaths, cigarette smoking accounts for half of the people who regularly smoke [1-3]. Quitting smoking reduces the risk of all-cause mortality by 11%-34% [4]. However, an estimated 1.3 billion individuals worldwide continue to use tobacco products [5]. Many interventions for promoting smoking cessation have been implemented worldwide [6-11]. Behavioral change therapies and pharmacotherapeutics are the most recommended approaches to assist smoking cessation, both individually and in combination, and are effective in increasing smoking cessation in adults [12].

Health information technology–based interventions (such as a computer, tablet, or smartphone) [13] are associated with increased smoking cessation in the general population [9,13]. The number of smartphone users is increasing [14], and smartphone app–based interventions have been developed and are widely used. The advantages of using a smartphone app–based intervention include the ease of use, as they can be used anywhere at any time, the ability to provide visual information via video, and the ability to deliver interventions on a large scale. Moreover, they also have the ability to tailor messages in terms using specifically provided information (such as age, mood, time, and nicotine independence scores) by using algorithms. However, the evidence for the effectiveness of smartphone-based interventions for smoking cessation is currently controversial. To evaluate the cumulative effects of smartphone app–based smoking cessation (SASC) interventions on smoking cessation, researchers conducted a Cochrane systematic review in 2019. However, the level of evidence was insufficient because the review only included 5 randomized controlled trials (RCTs) with small sample sizes, which might have affected the pooled effect of the meta-analysis [9].

Recently, more RCTs of SASC interventions with larger sample sizes have been conducted [6,15-18], and have provided opportunities to explore the effectiveness of SASC interventions with greater confidence. However, the results of those RCTs were conflicting. Five studies found that the SASC interventions did not lead to a higher smoking abstinence rate [19-23]; in contrast, other studies reported that the SASC interventions increased smoking abstinence [18,24-26]. Hence, our study aimed to explore the pooled effects of SASC interventions in promoting smoking abstinence.

Methods

This study was conducted according to the Preferred Reporting Items for Systematic Review and Meta-Analyses (ie, PRISMA) guidelines [27]. The protocol of this study was registered in the International Prospective Register of Systematic Reviews (CRD42021267615).

Search Strategy

We performed a comprehensive literature search to identify RCTs related to the SASC intervention for smoking cessation. Databases, including the Cochrane Library, PubMed, Embase, PsycINFO, and Web of Science, were searched for published papers in English. For Chinese publications, we searched the China National Knowledge Infrastructure and Wanfang databases, which are the primary databases in China. In addition, we manually searched through the reference lists of study reports to identify additional eligible studies and contacted the authors as needed for more information.

The search terms included a combination of keywords related to mobile health and smoking cessation. The search strategies used in PubMed, Embase, PsycINFO, Web of Science, and the Cochrane Library are listed in the Multimedia Appendix 1. The key terms included telemedicine, mobile health, mHealth, eHealth, telehealth, digital health, mobile phone, mobile device, cellphone, smartphone, smartphone app*, cellphone app*, mobile app*, smartphone-based, cellphone-based, mobile phone-based, portable electronic app*, portable software app* or smoking, smoking abstinence, smoking cessation, tobacco abstinence, tobacco use cessation, tobacco quitting, quit* smok*, give up smok*, and stop* smok*. In addition, search filters were used to extract only RCTs. The searches were conducted independently by 2 investigators and they used EndNote (version X9; Clarivate Analytics) to import and manage selected articles.

Study Selection Criteria

Overview

Studies were selected if they evaluated the effect of SASC interventions, were guided by health behavior change theory, and used an RCT design, including publications, dissertations, and conference papers based on the Population–Intervention–Comparison–Outcome (ie, PICO) method. Although we did not specify the exclusion of pilot studies in the research proposal, according to the National Institutes of Health, the goal of pilot work is not to test hypotheses about the effects of an intervention, but rather to assess the feasibility and acceptability of an approach to be used in a larger-scale study [28]. Thus, we calculated the sample size based on the smoking abstinence rate [24]. As a result, each group of 113 provides 80% power to detect a 15% reduction in smoking abstinence rate with 2-sided testing and set at .05 and the following equivalence tests for the difference of 2 proportions in a cluster-randomized design. For this review, pilot studies (each group with fewer than 113 participants) were excluded.

PICO Elements

Population

Eligible participants included (1) anyone who smokes ≥5 cigarettes per day and (2) active users of a smartphone. Exclusion criteria included studies with participants younger than 18 years.

Intervention

Smartphone app–assisted interventions to support smoking cessation based on health behavior change theory. According to Gitlin [29], without a theory, behavioral interventions will have limited success. The theory addresses 3 broad essential questions: why the intervention should work (the development phase), how the intervention works (the evaluation phase), and how the intervention works in real settings (the implementation phase). Most of the studies reported that the intervention was designed based on the behavioral change theory. For the reports that did not cite a guiding theory, the corresponding authors were contacted to confirm the use of the health behavior change theory prior to selecting the study for inclusion in the review.

Comparison

The comparison group could be any other type of supportive smoking cessation, including standard care, SMS text messaging intervention, web-based intervention, or smoking cessation counseling.

Outcomes

The outcome was the smoking abstinence rate, reported as either a 7-day point prevalence abstinence rate or a continuous abstinence rate, assessed at least 3 months after baseline using objective or self-report measures. According to the Nicotine and Tobacco Subcommittee on Biochemical Verification [30], biochemical validation is not always necessary in smoking cessation studies since levels of misrepresentation are generally low (0%-8.8%), thus reducing the likelihood of bias in reporting. All studies measuring smoking abstinence, including solely self-reported outcomes, were included.

Data Extraction

Titles and abstracts were retrieved using the search strategy by an author (YQG), and the duplicates were removed. Based on the selection criteria, 2 reviewers (YQG and YW) independently screened the titles and abstracts. The authors (YQG and YLC) reviewed the full texts, and any disagreements were resolved through discussion until consensus was reached or by consulting the third author (YW). The full texts of the selected papers were reviewed, and the reasons for exclusion were documented.

Two reviewers (YQG and YLC) then independently extracted the data from each eligible study. The extracted information included: (1) study characteristics, including authors’ names, year of publication, country setting, study design, sample characteristics, name of the smartphone app–assisted interventions to support smoking cessation, comparator, details of the intervention, type of outcome measurements, retention rate, funding, and conflict of interest, follow-up period, whether or not the RCT protocol was published or registered; and (2) the outcomes of interest included biochemically verified smoking abstinence rate or self-reported smoking abstinence rates from different smoking cessation interventions after 3, 6, and 12 months.

Risk-of-Bias Assessment

Two reviewers (YQG and YLC) independently assessed the quality of studies according to the Cochrane Collaboration risk of the bias assessment tool, RoB 2 [31,32], which includes 5 domains corresponding to specific types of bias: (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in the measurement of the outcome, and (5) bias in the selection of the reported result. A judgment of “low risk,” “high risk,” or “some concern” of bias was assigned to each domain. Disagreements between the 2 reviewers were resolved by consulting with the third reviewer (YW). A final decision was made after the 3 reviewers reached a consensus.

Data Synthesis and Analysis

We analyzed the pooled effects of continuous or point-prevalence tobacco abstinence rates as the primary outcomes. Subgroup analyses were performed based on different follow-up points, interventions (solely SASC interventions or SASC combined with pharmacotherapies), levels of adherence, and outcome measure methods.

The meta-analysis was conducted using RevMan (version 5.4; Cochrane Training), a desktop version of Review Manager software. The funnel plot, a simple scatter plot of the intervention effect estimates from individual studies against some measure of each study’s size or precision, and the Begg and Egger tests were conducted with Stata (version 17 SE; StataCorp) software to determine the presence of publication bias. The number of people randomized to the intervention and control groups was extracted to calculate the odds ratio (OR). The pooled effect sizes are presented in forest plots.

Heterogeneity among studies was assessed using the chi-square test, and I² values were used to determine heterogeneity across studies, attributing to the proportion of total variation, in which an I²>50% indicated substantial heterogeneity of effects, and random-effects models were applied. If the I² was <50%, fixed-effects models were selected [33]. A leave-one-out sensitivity analysis was done by removing each trial from the analysis to explore whether an outlier trial had an undue influence on the meta-analysis. Subgroup analyses were performed based on different follow-up time points. Additionally, we separated the RCTs by whether the SASC interventions were used alone or with adjuvants of clinical smoking cessation therapy. All statistical tests were 2-tailed, and a P value of less than .05 was considered statistically significant.

Results

Results of the Literature Search

Our initial search yielded 2659 publications based on the defined search terms (Figure 1). After removing 1073 duplicates and irrelevant publications, a total of 1506 abstracts were screened, of which 1468 articles were excluded. The full-text reports for 38 potential studies were retrieved, of which 29 studies were excluded. According to the selection criteria, a final sample of 9 studies was included in this review. A flowchart of the study selection process is presented in Figure 1.

Nine RCTs were included in the meta-analysis, with a total of 12,967 participants. Sample sizes ranged from 240 to 5293 participants. The finally included trials were all published between 2018 and 2022, even though there was no year limit used as a search strategy. Among the included studies, 4 were conducted in European countries (the United Kingdom, Spain, France, or Switzerland) [21,22,24,26], 3 in the United States [20,23,25], 1 in Canada [19], and 1 in Japan [18]. Five studies reported biochemically verified measurements of smoking abstinence, and 4 studies relied on self-reported smoking abstinence. The mean age of the included participants ranged from 36 to 54 years old. The reported smoking abstinence rate among SASC groups varied from 7.8% to 63.9%. The comparators consisted of self-help booklets, web-based behavioral therapy, brief information or calculators (such as days without cigarettes or money saved), or scheduling interactions between patients and smoking cessation consultants. The details of compensation paid to the participants were reported in 5 studies [18,19,23,25,26], but not in the other 4 studies [20-22,24]. The characteristics of participants, intervention details, and outcome measures are presented in Tables 1 and 2.

‎

Risk of Bias

Bias was assessed based on the Cochrane risk of the bias assessment tool, RoB 2 [31,32]. Three studies were double-blinded [19,22,25], 4 studies used the single-blinded approach of either the participants [18,24] or study staff [23,26], and 2 studies did not report the blinding method [20,21]. In fact, given the nature of the SASC interventions, researchers could not prevent the participants from knowing their group assignment.

Several sources of bias were found. The rate of loss to follow-up was high, and the representativeness of the research results was biased. Four studies reported that the retention rate was less than 50%, reflecting moderate risk. More details are shown in Figure 2. Even though 4 included studies used self-report to measure smoking abstinence, we did not consider it a high source of bias in the measurement of the outcome since biochemical validation is not always necessary in smoking cessation studies because levels of misrepresentation are generally low (0%-8.8%) [30].

‎

As shown in Figure 3, the funnel plot was conducted to identify whether publication bias existed. The funnel plot was generally symmetrical (Egger test, P=.10; Begg test, P=.12), which implied no publication bias existed in the included studies.

‎

Results of the Meta-analysis

The smoking abstinence rate among the 9 enrolled studies is displayed in Figure 4. There was no significant difference in the smoking abstinence rate between the experimental and control groups (OR 1.25, 95% CI 0.99-1.56, P=.06, I²=73.6%). A leave-one-out sensitivity analysis was performed to examine the results. As displayed in Table 3, the sensitivity analysis shows that our results did not change in the heterogeneity outcomes, with the I² ranging from 66% to 76%.

‎

Subgroup Analysis of Smartphone App–Based Intervention at Different Follow-Up Time Points

Referring to Figure 5, subgroup analysis was performed to further explore the differences in effectiveness by the duration of follow-up. The 4 studies that reported 3-month results (3 studies, N=3442) found no significant differences between the SASC group and the control group (OR 1.47, 95% CI 0.96-2.24, P=.08, I²=81.9%). Moreover, at the 6-month follow-up (8 studies, N=12,727), no statistically significant differences were observed between groups (OR 1.20, 95% CI 0.95-1.50, P=.12, I²=74.2%). Studies that reported the smoking abstinence rate at 12 months (3 studies, N=1342) found statistically significant differences between groups (OR 1.79, 95% CI 1.38-2.33, P<.001, I²=7%) [18,24,26]. However, among all 3 studies, both the intervention and control groups received pharmacotherapy. On top of that, the intervention groups also received smartphone app–based interventions, while the comparator group only received pharmacotherapy. In this way, the effect of smartphone app–based intervention was observed.

‎

Subgroup Analysis of Solely SASC Interventions or SASC Combined With Pharmacotherapies

Referring back to the elements of the interventions (Multimedia Appendix 2), it shows 6 studies delivered the health behavior change theories driving smoking cessation intervention by smartphone apps alone; however, 3 studies used the apps to augment pharmacological cessation therapies. As shown in Figure 6, 6 studies with smartphone app–based interventions alone reported no significant differences in effectiveness between the 2 groups (OR 1.03, 95% CI 0.85-1.26, P=.74, I²=57.1%). However, intervention groups combining smartphone-based app use and pharmacologic therapies for smoking cessation (3 studies, N=1342) reduced the chance of quitting smoking. Researchers conducted the interventions via smartphone apps, coupled with pharmacotherapies and consultations for the participants. On the other hand, comparators were pharmacotherapies and consultations for the participants. The smartphone app combined with pharmacotherapies and consultation groups increased the smoking abstinence rate compared with the comparators (OR 1.79, 95% CI 1.38-2.33, P<.001, I²=7%).

‎

Subgroup Analysis of Different Levels of Adherence

As shown in Figure 7 (5 studies, N=3029), 5 studies with higher adherence rates (more than 50%) showed better effectiveness of SASC (OR 1.48, 95% CI 1.20-1.84, P<.001, I²=24.5%). The other 4 studies (N=9938) with poor adherence rates (less than 50%) showed limited effectiveness of SASC (OR 1.02, 95% CI 0.78-1.33, P=.88, I²=68.9%).

‎

Subgroup Analysis of Different Recruitment Strategies

As shown in Figure 8, the 3 studies where participant recruitment occurred in clinics showed better effectiveness of SASC (OR 1.76, 95% CI 1.33-2.34, P<.001, I²=4.3%) compared to the 6 studies that recruited participants through the web (OR 1.07, 95% CI 0.86-1.33, P=.55, I²=67.5%).

‎

Subgroup Analysis of Different Outcome Measure Methods

As shown in Figure 9, five studies used self-report as the measure of smoking abstinence outcome. The pool effects of these studies showed limited effectiveness of SASC intervention (OR 1.00, 95% CI 0.85-1.18, P=.99, I²=59%). However, 5 studies used the bio-verified method (exhaled carbon monoxide or cotinine urine test) to measure smoking abstinence. The pool effects of the 5 studies showed the effectiveness of SASC (OR 1.46, 95% CI 1.11-1.92, P=.001, I²=53%). All 3 SASC-assisted pharmacotherapy intervention studies used the bio-verified method to measure smoking abstinence. To minimize the intervention factors, we explored the pooled effect of bio-verified outcomes studies by removing those 3 studies. There were no significant differences in effectiveness between the 2 groups (OR 1.21, 95% CI 0.57-2.59, P=.62, I²=69%).

‎

Discussion

Principal Findings

We conducted a systematic review and meta-analysis of 9 relatively high-quality RCTs through a comprehensive database search to determine the impact of SASC interventions. A total of 12,967 adult participants were included in this study. Although our findings failed to prove that SASC interventions alone were effective for smoking abstinence, improved smoking abstinence was achieved when SASC interventions were used to augment smoking cessation pharmacotherapy. In addition, improved smoking abstinence was shown for studies with participants recruited from the clinic rather than through the web. The study also found that the method for measuring the outcome of smoking cessation (self-report or verified biological measures) did not necessarily impact outcome assessment, which is consistent with previous reports [30]. Compared to previous meta-analyses of SASC interventions, this study identified more reports of RCTs with larger sample sizes, allowing us to draw more accurate and valid inferences. In addition, bias was assessed in 5 distinct domains via RoB 2. Within each domain, we answered the signaling questions. These answers lead to judgments of “low risk of bias,” “some concerns,” or “high risk of bias.” Major biases in the included studies were the lack of blinding; given the nature of the intervention and the control, some researchers could not blind participants to treatments. Thus, the results may reflect participant expectation bias. Furthermore, not all research programs could capture consistent participant-level data on intervention adherence, which may increase the nonrespondent bias.

Based on our meta-analysis, there are several possible reasons why SASC interventions alone showed lower smoking abstinence rates. First, adherence to smartphone app–based interventions may be the main reason for the failure of smoking cessation. As shown in our results, lower abstinence rates resulted from poor adherence to the intervention. On the contrary, Masaki and Webb’s studies [18,26] reported that SASC-assisted pharmacotherapy groups had higher smoking abstinence rates, possibly because SASC intervention could increase the participants’ understanding of behavioral intervention methods and provide smoking cessation information at any time. In addition, as shown in our results, improved smoking abstinence was achieved if the study participants were recruited from clinics rather than through the web. Smokers who took smoking cessation medications had face-to-face communication and clinical follow-up with clinic smoking cessation consultants, which in turn may have increased smokers’ motivation and frequency of using smoking cessation apps.

Poor adherence may result from the ubiquitous low engagement of the smartphone app–based intervention, which is not uncommon and consistent with previous health behavior change studies [34-36]. Smartphone-based interventions may have little effect if individuals do not adhere to the intervention protocol [29]. We did not do the meta-analysis based on engagement because the measures of engagement with the intervention were inconsistent and inadequate [37,38], as they did not monitor the number of log-ins or time spent using each module to verify the engagement. Although there is no consensus on the benchmarks for reporting eHealth engagement, engagement could be increased through various mechanisms that depend heavily on the characteristics of current technology, users’ characteristics (such as literacy and habits), and their environmental factors. Therefore, to raise adherence to SASC interventions, it is highly suggested to consider improving the app to support smokers’ engagement and setting benchmarks for reporting smartphone-based interventions. In addition, the financial compensation paid may also increase the motivation to complete the intervention.

Secondly, ineffective intervention may result from not fully exploiting features. The features of smartphone apps that have the potential to support behavior change and customization, such as real-time feedback, individualized messaging, or interaction, may have increased the effectiveness and individualization of app-based interventions. Etter’s study [22] provided automated, individually tailored counseling messages based on the user’s profile, regularly sent for 6 months, immediate feedback during episodes of craving and tobacco withdrawal symptoms, and a module on nicotine therapy that includes personalized feedback and follow-up. Webb’s study [26] provided personalized CBT-based support via the in-app chat and phone, with a 30-min phone call at baseline, discussing their individualized quit plan and methods of using NRT. Affret et al’s [21] study provided personalized messages relating to the answers to the different questionnaires. The details of the customized features were various. There was a lack of representativeness and a precise definition of the individualized in the real sense. We did not check the effectiveness based on the extent of the individualization feature. In fact, with the current level of artificial intelligence technology development, computer engineers have been able to provide instantaneous ecological momentary assessment methods, predict smoking risks based on user data, and push timely information based on algorithms. However, researchers may face numerous barriers when offering SASC intervention; for instance, research participants may worry about their privacy or consider cumbersome data entry a burden. To date, only a few SASC intervention studies have achieved individualization and timely information response.

Although smartphone app–based intervention alone did not achieve superior efficacy, higher effect sizes were found in studies that combined smartphone app–based intervention with pharmacotherapy. There might be 2 reasons for this, which include the fact that SASC intervention is not limited by time or region, as app users can receive the intervention according to their schedule. Second, the smokers who received pharmacotherapy are regularly followed up by physicians, which may increase the stickiness of app use. In addition, we found that the SASC is more effective at 12 months than shorter follow-ups. This could be because all 12-month follow-up studies are the same studies that intervene with the participants with the SASC combined with pharmacotherapy. To assist smokers in quitting smoking in the future, it is recommended to use SASC intervention in smokers who receive smoking cessation drug treatment. Future studies can reflect on how to balance the frequency of the combination of outpatient follow-up and SASC to maximize the smoking cessation rate with the minimized cost of human medical resources.

Strengths and Limitations

The strengths of our meta-analysis are that all of the included studies had large sample sizes. Although smoking cessation research based on smartphone apps is still emerging, our research team included many high-quality RCTs, which allowed us to draw more accurate and valid inferences. Moreover, all the included studies conducted behavioral theory-driven interventions, which could enhance the delivery of the intervention and boost its effects on participants. However, our study also has some limitations. First, heterogeneity among the included studies still exists even after we applied the subgroup and sensitivity analyses. Second, subgroup analysis based on different follow-up times had been conducted, but the number of included studies was even lower. Third, although all smoking cessation interventions were applied by smartphone apps, the specific interventions may vary widely between the studies. Fourth, low adherence to the SASC intervention brought increased threats to the research’s reliability and validity. Lastly, the abstinence outcome criteria used in the included studies varied, such as continuous smoking abstinence and 7-day point prevalence smoking abstinence indicators, which may also increase the heterogeneity of the pool studies. Future work on improving adherence to SASC intervention is warranted to evaluate the future effect of SASC intervention. It is recommended to use SASC intervention in the future for smokers who receive smoking cessation drug treatment to assist smokers to quit smoking and to continue to refine the frequency of the combination of outpatient follow-up and SASC to maximize the smoking cessation rate and optimize the cost of human medical resources. Finally, although the promise of the engagement report of the SASC intervention has not yet been fully realized, future research may provide insight into how to target the engagement of smartphone app–based interventions, which may affect intervention adherence and smoking cessation abstinence.

Conclusions

In conclusion, our study is confined to determining the effectiveness of smartphone-based apps in smoking cessation intervention within the boundaries of existing evidence. The current meta-analysis did not reveal superior effects for stand-alone smartphone-based apps. However, smartphone app–based interventions may be the most effective tool affiliated with pharmacotherapeutics. Furthermore, smartphone app–based interventions with good adherence showed improved smoking abstinence. More rigorously designed RCTs with robust engagement reporting should be encouraged to test the effectiveness of smartphone-based app interventions for smoking abstinence.