Comparing the Efficacy of Two Internet-Based, Computer-Tailored Smoking Cessation Programs: A Randomized Trial

Introduction

Self-help smoking cessation booklets and leaflets can reach large numbers of smokers, but they may not be very effective [1]. Computer technology and psychological theory can be combined to produce effective individualized self-help smoking cessation materials and to disseminate them at the population level, in particular on the Internet [2]. Because individually tailored materials take into account the relevant characteristics of each participant, smokers may be more interested in reading these materials than untailored booklets, and may be more likely to apply the advice included therein [3,4]. Consequently, tailored materials may be more effective than those intended for all smokers [1,5]. Several studies have tested the effectiveness of computer-tailored smoking cessation programs, with positive and negative results [1,6]. These programs were evaluated either on personal computers or when feedback materials were printed and sent by mail. Few studies tested the effect of smoking cessation programs administered on the Internet [7,8,9]. An early randomized trial conducted in 1998 on CompuServe showed that after 3 months smoking abstinence rates were higher in smokers who took part in an online discussion group and received e-mail messages compared to a control group, but this effect was not maintained at 6-month follow-up [7]. More recently, the only other randomized trial on this topic showed that in nicotine patch users, an online computer-tailored program was more effective in the short-term (10 weeks) than a non-tailored program [9]. A non-randomized trial showed that the effect of an online interactive program could be improved by tailored follow-up by email [8]. We know of no randomized trial comparing two online, computer-tailored smoking cessation programs. Such comparisons are nevertheless necessary, given the large variability in the effect of these programs [1].

In a previous study, we tested a paper version of a computer-tailored program [10,11]. In this version, questionnaires, computer-tailored counseling letters and stage-matched booklets were sent by mail to smokers. This study showed that 7 months after entry into the program, 7-day smoking abstinence rates were 2.4 times greater (8.0% vs 3.3%, P<.001) in the intervention group than in a control group that received no treatment. The same program is also available online, but the efficacy of the online version is unknown. We compared the online version of this program with another online smoking cessation program intended for users of nicotine replacement products.

Several websites offer interactive, computer-tailored smoking cessation programs, but these programs have never been compared directly in a randomized trial. Our aim was to compare the efficacy of two online, computer-tailored smoking cessation programs.

Methods

Setting and Participants

Participants were visitors of Stop-tabac.ch, a French-language website that provides information, advice, and support to smokers and ex-smokers. This website was listed among the 5 best websites on smoking cessation in a recent study [12], and it is listed first in Google.fr when searching with the words arrêter de fumer, fumer, or tabac (quit smoking, smoke, or tobacco) (tested February 21, 2005).

Interventions

Various services are available to visitors of Stop-tabac.ch, including fact sheets, booklets, answers to frequently asked questions, personal stories written by current and former smokers, discussion forums and chat rooms, tests, games, and two interactive, computer-tailored smoking cessation programs [10,11]. Each month, about 2% of the 50000 monthly visitors of the website take part in these interactive computer-tailored programs [13]. After reading an information page that briefly describes the programs, participants are informed that they will have to answer a questionnaire, that their answers will be retained on file, and that the data will be used only to organize a follow-up and for statistical analyses conducted in an anonymous format. They have the option of refusing to have their answers retained on file. The next step consists of answering the tailoring questionnaire. Enrollment of participants in this study took place between April 2003 and July 2004. In this period, two different questionnaire forms, referring participants either to the original or the modified program, appeared alternatively in random order. Thus participants were randomly assigned to either program.

Both programs consisted of tailored personal counseling letters compiled by a computer according to the answers made by participants. The counseling letters appeared on the screen immediately (<5 seconds) after the answers were submitted. Participants were advised to print their counseling letter and to read it again later. Participants in the original program were also advised to print stage-matched booklets available on the website.

The Original Program

The original program was based on the Transtheoretical Model of Change [14,15], on the Theory of Planned Behavior [16], on theories of relapse prevention [17] and tobacco dependence [18], on the Agency for Health Care Policy and Research recommendations [19], and on relevant literature [20,21]. The questionnaire, counseling letters, and brochures were also based on extensive research conducted on Swiss smokers and ex-smokers [22,23,24]. The tailoring questionnaire (Figure 1) assessed demographic characteristics, smoking status, stage of change (precontemplation, no intention of quitting smoking in the next 6 months; contemplation, seriously considering quitting in the next 6 months; preparation, has decided to quit in the next 30 days; action, has quit smoking for 6 months or less; and maintenance, has quit smoking for more than 6 months)[15], level of tobacco dependence, attitudes towards smoking, self-efficacy, use of self-change strategies and coping methods, and intention to use nicotine replacement therapy (NRT). We used validated multi-item scales to measure these variables [22,23,24,25]. Former smokers indicated the date that they had quit smoking. After answering the 62-item questionnaire, participants received a personal counseling letter of 6 to 9 pages (3000-4000 words) illustrated with cartoons and graphs that were also tailored to each participant's answers (Figure 2). The counseling letters consisted of about 20 paragraphs of text, chosen by the computer in a library of 350 paragraphs according to pre-established decision rules. This program was launched online in French in 1997 and was later expanded to include English, Danish, Italian and Chinese versions [13]. The interactive program was updated to include innovations (eg, new NRT products and bupropion), pictures, and a few additional questions and feedback paragraphs. Overall, the online version of the program tested in the present study is nevertheless largely similar to the paper version tested in our previous studies [10,11].

‎View this figure

‎View this figure

The Modified Program

The modified program was developed by us for Novartis, a producer of nicotine replacement products, when these products became available over-the-counter (OTC, ie, without a medical prescription) in Switzerland in 2000. This program was intended to provide some smoking cessation counseling to smokers who bought OTC NRT products and thus did not receive medical supervision. Compared with the original program, the modified program used a shorter questionnaire (38 questions) that included ad hoc questions instead of validated multi-item scales (Figure 3). The counseling letter was of similar length (3000-4000 words), but contained more information on NRT and nicotine dependence, and less information on health risks and coping strategies (Figure 4).

‎View this figure

‎View this figure

Additional Program Interactions

In both programs, 1 month and 2 months after entering the study, participants were invited by email to answer the same tailoring questionnaire again in order to receive a second counseling letter. To write the second letter, the computer compared each participant's new answers with the answers given on their previous visit. Participants were congratulated for any progress they had made since their last visit or encouraged, if they had relapsed. In both groups, participants received on average 1.2 counseling letters; 84% of participants received only 1 counseling letter and 16% 2 or more letters.

Outcome Measures

To assess smoking abstinence, an email message was sent out 11 weeks after receipt of the baseline questionnaire; those who failed to respond received up to 3 email reminders. Participants answered the following question by clicking on Yes or No directly in the email message: "Did you smoke any tobacco in the past 7 days (even one puff of cigarette, cigar, pipe, etc)?" The criterion of 7 days' abstinence was used in a recent guideline to assess smoking cessation in randomized trials [26]. We used an intention-to-treat analysis in which all non-respondents were counted as smokers.

Sample Size Calculations

Sample size calculations indicated that a sample of 5300 was necessary to detect a between-program difference of 2 percentage points in abstinence rates in current smokers (8% vs 6%, confidence level 95%, power 80%). The expected difference of 2 percentage points was estimated on a basis of a synthesis of previous studies of computer-tailored programs [1], and taking into account an expected follow-up rate of about one third [28] and an intention-to-treat analysis. With its final sample size of 11969 participants, the study was powered to detect differences in subgroups of participants, in particular current and former smokers.

Statistical Analyses

We used chi-square tests to compare proportions (eg. abstinence rates) and t tests to compare means. We used odds ratios (OR's) with 95% confidence intervals (CI's) to express the proportion of non-smokers (abstinence rate) in the original program compared to the proportion of non-smokers in the modified program. We tested the effectiveness of the program in subgroups, stratifying by age, sex, number of cigarettes per day, and stage of change.

Because participation rates in the follow-up survey differed in the two groups, we report both intention-to-treat data, where all baseline participants were included in the denominator and non-respondents were counted as smokers, and an analysis including only those who took part in the follow-up survey. We also conducted a sensitivity analysis, extrapolating results under a hypothetical situation where response rates to the follow-up survey were assumed to be the same in the two study arms.

Results

Participation

The raw database included 12434 records. We excluded 245 participants who had taken part in both programs and deleted 220 entries of people registered twice in the same program. Thus 11969 participants were included in the study. Figure 5 illustrates the flow of participants through the trial.

‎View this figure

At baseline, the two study groups were similar in terms of age and sex distributions, smoking status (current or former smokers), stage of change, cigarette consumption, and, among former smokers, the number of days since smoking cessation (Table 1). As in a previous study [13], the sample included a substantial proportion of former smokers (n=3095, 26%), and relatively few smokers in the precontemplation stage of change (n=385, 3%). Smokers in this study were more motivated to quit smoking than a representative sample of smokers in Geneva (distribution of smokers by stage of change in Geneva: 74%, precontemplation; 22%, contemplation; 4%, preparation) [27].

The response rate to the follow-up survey was 35.4% (4237 of 11969). However, more participants in the original program (n=2341, 39.2%) than in the modified program (n=1896, 31.6%) answered the follow-up survey (χ²=76.7, P<.001). In both groups, the median interval between the baseline and follow-up surveys was 2.5 months (25th, 50th, and 75th percentiles in both groups; 75, 77 and 80 days respectively).

Smoking Abstinence Rates

At follow-up, when all baseline participants were included in the denominator and non-respondents were counted as smokers, the 7-day abstinence rate was higher for the original program than for the modified program (14.6% vs 10.7%, P<.001, OR=1.43, 95% CI 1.28 - 1.59) (Table 2). Thus compared with the modified program, the original program produced 1 additional quitter for every 26 participants. The original program was more effective than the modified program in baseline current smokers (abstinence rates: 10.9% vs 8.9%, OR=1.24, CI 1.08-1.63, P=.003) and in baseline former smokers (25.2% vs 15.7%, OR=1.81, CI 1.51-2.17, P<.001).

Among smokers in the precontemplation and preparation stages of change, there was no statistically significant difference in quit rates between programs; but the original program produced more quitters than the modified program among smokers in the contemplation stage (Table 2). Among light smokers (1-10 cigarettes/day), there was no difference in quit rates between programs; but the original program was more effective than the modified program in smokers of 11 to 24 cigarettes/day (OR=1.28) and in heavy smokers (25 or more cigarettes/day) (OR=1.54). The relative effect of the two programs was the same in men and women and across age groups. Interestingly, younger smokers (≤19 years old) were the least likely to quit smoking.

Secondary Analysis

When we included only the 4237 participants who answered the follow-up survey, abstinence rates were significantly (χ²=5.0, P=.03) higher in the original program (873 out of 2341, 37.3%) than in the modified program (644 out of 1896, 34.0%).

Sensitivity Analysis

In a sensitivity analysis, we extrapolated data assuming a hypothetical situation where the same proportion of participants in both groups (39.2%) answered the follow-up survey. Under this assumption, 2356 out of 6003 (39.2%) participants in the modified program (instead of 1896) would have answered the follow-up survey, and 800 out of 2356 (34.0%) would have quit smoking. Under this assumption, and including all baseline participants in the denominator, 13.3% (800 out of 6003) would have been abstinent in the modified program versus 14.6% (873 out of 5966) in the original program (χ²=4.3, P=.04). Thus the original program was still more effective than the modified program, even after taking into account the difference between groups in response rates.

Discussion

Efficacy

We compared two Internet-based computer-tailored smoking cessation programs: a program based on theory and preliminary research conducted in the study population; and a modified and simplified version of the same program designed for NRT users. The original program was more effective than the modified program in helping current smokers in the contemplation stage of change quit smoking and in helping former smokers avoid relapse. In a previous study, we showed that when implemented on paper (ie, when counseling letters and booklets were sent by mail), the original program was more effective than no intervention [10,11]. This study showed that the efficacy of this program was apparently maintained when it was implemented over the Internet. Among baseline smokers, 7-day abstinence rates were quite comparable in the Internet version (10.9%) and in the paper version of the original program (8%) [10]. In this previous study, we tested the original program on current smokers only. The present study suggests that this program was also effective in preventing relapse in former smokers.

Because the present study did not include a no-treatment control group, we are unable to say whether both programs were more effective than no intervention. However, quit rates in smokers in the modified program (8.9%) were higher than quit rates in smokers in the no-treatment control group in our previous study (3.3%) [10], which suggests that even the modified program might be more effective than no intervention. Tests of the Internet versions of both programs against a no-treatment control group are nevertheless warranted, but such tests are made difficult by the risk of selective drop-out in the no-treatment group, and by the potential for contamination from external programs, as disappointed participants in the no-treatment group could obtain counseling from other websites able to be found in just a few clicks.

The follow-up in both programs consisted of short, monthly email messages inviting participants to answer the same questionnaire again, in order to receive a second counseling letter that was largely similar to the first one. This follow-up procedure may not have been intensive enough, which may explain why so few participants obtained additional counseling letters. The follow-up in the program could be improved by using individually tailored email messages, sent more frequently just before and after the quit date, as was done by Lenert and colleagues [10].

Cost

The total cost of implementing the website where the two programs are available is currently 70000 Swiss francs a year (US$ 60000), for a reach of over 8000 participants per year in the computer-tailored programs, and for 600000 visitors per year to the website (where other features, such as discussion forums and personal stories, are more popular than the computer-tailored program). The average duration of a visit to the website is 7 minutes, with an average of 8 pages viewed per visit. This is comparable to the cost of running a small smoking cessation clinic which would treat about 50 smokers a month.

Strengths and Limitations of This Study

A strength of this study is that it was powered to detect small differences in quit rates in subgroups of participants (eg, current and former smokers). The response rate at follow-up was low (35%), but it was close to the average response rate of 39.6% reported in a meta-analysis of 68 Internet-based surveys [28], and it was in the range of 30-40% in response rates obtained in follow-up surveys of the three other efficacy trials of online smoking cessation programs [7,8,9]. Follow-up rates in Internet studies are lower than those usually found in smoking cessation studies. Several steps could be taken to increase follow-up rates in Internet surveys such as: asking participants to indicate a second email address or the email address of a relative; asking them to keep their email address active for the duration of the study; requiring participants to commit to taking part in the follow-up survey; and asking for a phone or fax number, or a postal address. Paying participants could introduce bias and is not a very cost-effective option, given the large samples obtained in Internet studies.

There were more non-respondents in the modified program group than in the original program. This could produce an artificial advantage for the original program in intention-to-treat analyses where non-respondents are counted as smokers. However, even when data were analyzed in respondents only, quit rates were higher in the original program. In addition, a sensitivity analysis showed that if participation rates had been similar in both study arms, the original program would still have been more effective than the modified program. Under this assumption however, the between-group difference would have been smaller.

Fewer participants in the modified program than in the original program took part in the follow-up. The modified program was developed in collaboration with a pharmaceutical company and emphasized NRT use. Participants were informed of this collaboration and may have been less keen to take part in the follow-up of a program associated with the industry than in the original program, which was university based.

Because the study did not include a no-treatment control group, it remains possible that the natural quit rates in this sample (ie, the quit rate outside any intervention) lies somewhere between the quit rates measured in the 2 study arms. In this case, the programs would have no effect. However, the original program produced similar quit rates whether it was implemented on the Internet or on paper, and these quit rates were higher that in the no-treatment control group in our previous study [10]. This suggests that the original program is more effective than no intervention. Nevertheless tests of the online versions of both programs against no-intervention control groups are warranted.

The difference in program efficacy between the original and modified versions was observed only in smokers in the contemplation stage of change, but not in those in the precontemplation and preparation stages. Similarly, the paper version of the original program had no effect in smokers in the preparation stage [10]. The paper version had however a significant impact in smokers in the precontemplation (3 percentage points) and contemplation stages (4 percentage points) [10]. These results suggest that this program may be effective mainly in motivating contemplators to make a quit attempt. A new version of the program should be developed in order to better take into account the needs of smokers in the preparation stage.

We measured point prevalence of abstinence after 2.5 months, but this approach may not reflect long-term continuous abstinence rates. In our previous study of the paper version of the original program, we showed that the effect measured 7 months after entry into the program was not maintained, one-and-a-half years after the intervention was stopped [11]. Previous research showed that one half of the people who succeed in abstaining from smoking for 6 months will relapse within 5 years [29]. Thus long-term follow-up studies are needed to assess whether Internet-based programs have sustained effects. The only existing studies are short-term (<=6 months) [7, 8,9]. Long-term studies are however limited by the difficulty of obtaining high response rates in Internet surveys [28].

We conducted no biochemical verification of smoking status for several reasons. First, collecting saliva samples for the determination of cotinine (a metabolite of nicotine) or collecting expired carbon monoxide would have decreased participation rates [30]. Second, biochemical verification will not change the results of most smoking cessation studies because self-report is generally accurate in adults, and because large between-group differences in misreporting are unlikely [31]. Third, biochemical verification is not recommended in large scale population-based studies with limited face-to-face contact, and in studies where data collection is done over the Internet [32]. In a study conducted in a similar population, we showed that for the association between saliva cotinine and self-report of smoking, the area under the receiver operating characteristic curve was 0.95, and that most cases of disagreement were due to occasional smokers rather than to misreporting [33]. Furthermore, at least two studies indicated that in intervention trials, self-report of smoking was not, or only minimally, biased in intervention groups, compared with controls [34,35]; therefore, such bias would not explain away our results.

Conclusion

The original program was more effective than a modified version of the same program intended for NRT users. Given the already documented large variability in the effect of computer-tailored programs [1], other available online smoking cessation programs should be compared directly, in randomized trials [36].