Utilization of a Web-Based vs Integrated Phone/Web Cessation Program Among 140,000 Tobacco Users: An Evaluation Across 10 Free State Quitlines

Introduction

Fifty years after the release of the first Surgeon General’s Report on Smoking and Health, tobacco use is still the leading preventable cause of death in the United States [1]. Although cigarette use in particular has declined among American adults in the past 20 years, this shift is driven by a small proportion of relatively higher-income counties [2], indicating a widespread need for accessible and affordable cessation services. Over the past 2 decades, state governments throughout the United States have provided phone-based tobacco cessation services, called quitlines, to help tobacco users quit by providing evidence-based counseling. These services are offered statewide at no charge and often include nicotine replacement therapy (NRT). More recently, quitlines have responded to the aforementioned need for more easily accessible services by offering Web programs for use not only alongside traditional phone-based programs, but also as a stand-alone service without phone-based counseling. According to North American Quitline Consortium (NAQC) data [3], 44 state quitlines offered a self-directed Web-based intervention in 2012 [4], representing a notable increase from 27 in 2010 [5]. NAQC does not specify whether these Web services are stand-alone or integrated with standard quitline phone services, highlighting the need for more research into the structure of these services.

Tobacco users want to access cessation help via the Internet for a number of reasons, including convenience and a desire to remain anonymous [6]. Users also have reported a desire to access personalized, interactive websites, which are less common than sites simply containing educational content related to tobacco cessation [7]. State health departments have an interest in implementing Web-based services because they have been associated with the lowest cost per quit when compared to treatment delivered via phone or in person at a health care clinic or workplace [7]. Web-based services also have the potential to combine the wide reach of Internet-based health promotion with aspects of face-to-face counseling; those Web-based interventions that mirror other counseling modalities with a tailored, interactive approach have been shown to be more effective [8,9].

Despite the proliferation of Web-based services and the study of reasons for use, relatively little is known about who uses Web-only services, especially when users are given the choice between different programs. Randomized controlled trials have provided support for the efficacy of some Web-based cessation services [10], but more research is needed to understand how tobacco users select a treatment modality, their demographics, and how they engage with treatments in a real-world setting of tobacco users seeking help with quitting. The purpose of this paper is to examine the characteristics, utilization patterns, and return rates of tobacco users self-selecting into 1 of 2 free state tobacco cessation programs: stand-alone Web services versus Web services offered in combination with phone-based counseling. This information could inform outreach strategies, content tailoring, and future research evaluating outcomes for different program types.

Methods

Study Design

In this real-world observational study, participants selected 1 of 2 tobacco cessation programs offered through their state quitlines: (1) an integrated phone/Web program (Phone/Web) or (2) a stand-alone Web program (Web-Only). The Western Institutional Review Board reviewed the study and determined that it met the requirements for a waiver of consent under 45 CFR 46.116(d) on March 20, 2014.

Participants and Sample Selection

Ten state quitlines that offered both (1) a phone-based 1-call or multiple-call cessation program integrated with the Web Coach website (Phone/Web) and (2) a stand-alone Web Coach website program (without coaching calls; Web-Only) for the majority of the study timeframe were invited and agreed to participate in the study: Connecticut, Delaware, Florida, Idaho, Indiana, Kansas, New Mexico, North Carolina, Oklahoma, and Oregon. English-speaking participants aged 18 years and older who enrolled in a Phone/Web or Web-Only program through 1 of the 10 state quitlines from August 2012 through July 2013 were included in this analysis; for each state, registrants were only included from the timeframe in which both programs were offered to all enrollees.

Based on these inclusion criteria, 149,362 registration records were identified, of which 6698 individuals (4.48%) had 2 or more program enrollments during the study timeframe. To represent individuals only once in the analysis groups, the following steps were taken to remove duplicate registrations:

1. For participants with multiple Phone/Web program enrollments (4620/149,362, 3.09%), the first enrollment was retained to include the less biased case.
2. Because each individual who enrolled in Web-Only was intended to have only 1 Web Coach website account for life, those participants with multiple Web-Only program enrollments (623/149,362, 0.42%) were duplicated in error. In light of this, the enrollment with the greatest number of log-in days was retained to include the most accurate and complete data.
3. For participants who enrolled in the Web-Only program and switched to a Phone/Web program soon after (1394/149,362, 0.93%), the phone program enrollment was retained.
4. The most appropriate record could not be determined for the remaining participants (380/149,362, 0.25%) who had enrolled in both the Phone/Web and Web-Only programs. Because this group amounted to a very small percentage of the final sample, these participants were excluded from all analyses.

Analyses focused on the 141,429 unique adult English-speaking participants who enrolled in an integrated Phone/Web (113,019/141,429, 79.91%) or Web-Only (28,410/141,429, 20.09%) program offered by 1 of the 10 participating state quitlines. All participating states had contracted with Alere Wellbeing for the services offered through the quitline (including phone counseling, mailed materials, and Web-based services).

Phone/Web and Web-Only Program Descriptions

Enrollment

Quitline participants started their enrollment (“method of entry”) in the cessation program online, over the phone by calling the quitline, or via fax referral, a process in which health care providers fax-referred their patients who were then proactively called by the quitline. Participants who started enrollment online could also request a callback to complete their enrollment with a registration specialist by phone. During both phone and Web enrollment procedures, participants were presented with the program options available to them and then selected their preferred program.

Participants in both the Phone/Web and Web-Only programs enrolled in the Web Coach website by providing their email address and consenting to be contacted via email. Participants then had to authenticate their account by using the log-in information provided in the initial email sent to them by the program.

Phone Program

The phone-based coaching program (Phone/Web) operated by Alere Wellbeing was offered as a 1-call or multiple-call program. The 1-call program included an initial assessment and planning call with Quit Coach staff to identify the participant’s strengths and challenges, and to develop a quit plan. The multiple-call program included all aspects of the 1-call program plus either 3 or 4 outbound calls from the quitline. Participants in both phone programs were encouraged to call their quitline for support as needed. Both phone programs also included written educational materials for the participant (Quit Guide), referrals to community resources (when requested), health plan information (when appropriate), and access to the Web Coach website.

Web Coach Website

The Web Coach 2.0 website is the second version of the online participant application for the tobacco cessation coaching program operated by Alere Wellbeing (1.0 launched in 2006; 2.0 launched July 2011). It is grounded in social cognitive theory and designed to guide tobacco users through an evidence-based process of quitting tobacco. The website was offered as a stand-alone program (Web-Only) or integrated with the phone-based coaching program described previously (Phone/Web) and was tailored to each participant’s tobacco status and needs (ie, different content was recommended and enabled based on the participant’s quit status and activities completed). The Web Coach website also allowed participants to reach out to Quit Coach staff through phone call requests (for those in the Phone/Web program) and through chat and email in both Phone/Web and Web-Only; Quit Coach staff also moderated and participated in community forum discussions on the site. However, the focus of this paper is on the utilization of Web-based features and not the counseling options with a coach.

The Web Coach website home page (Figure 1) included links to recommended site content for each participant. The 4 key groups of features included:

1. The Quitting Plan (Figure 2): an interactive tool that enabled participants to build a personalized plan to quit using tobacco. The Quitting Plan guided participants through choosing a quit medication, setting a quit date, conquering urges and cravings to use tobacco, controlling their environment, and getting social support.
2. Progress Trackers: tools that helped participants who had not quit record and track their smoking patterns (Tobacco Tracker) and the potential financial savings of quitting (Cost Savings Calculator) (Figure 3). Participants who quit could use the Urge Tracker to record the strength of their urges or cravings to use tobacco (Figure 4), and could use the Cost Savings Calculator to review time quit, money saved, smoke-free breaths taken, and free time gained by quitting (Figure 5).
3. Interactive Practice Content (Figure 6): the Practices page introduced the 4 Essential Practices of Quitting, where participants could access e-lessons, articles, videos, and worksheets based on the Practices.
4. Community (Figure 7): the Community area was a place for participants to connect with one another to discuss their successes and challenges with quitting and staying quit; Quit Coach staff moderated the forums and actively participated in the discussions.

Web Coach website participants in both programs were sent the same tailored emails (up to approximately 25 messages) to remind and encourage them to log in to the site. First-time participants were sent a reminder email to visit if they had not logged in to the site within several days of enrolling in the program; additional reminder emails were sent if the participant still had not logged in at later time points. Participants who logged in but did not return within a certain time period received a reminder email to visit the site. Participants also received reminders to set a quit date, complete their quit plan, or update their tobacco status after their quit date if they had not done so. When a participant ordered NRT through the Web Coach website, summary information and links to use instructions were emailed. Emails also provided motivation through encouraging messages around the quit date, and through congratulatory emails when the participant reduced their tobacco use. The email schedule was designed to anticipate typical withdrawal symptoms and send encouragement to track urges and work on coping skills after the participant quit. Finally, regular check-in emails were sent to encourage participants to stay quit and follow their stay-quit plan.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

State Offerings

All 10 states included in the study offered both a Web-Only program and an integrated Phone/Web program to all participants. Eight of 10 states offered the multiple-call program described previously to all adults (dependent on readiness to quit for some states). In 2 state quitlines, the multiple-call program was only available to select groups of registrants (eg, uninsured, Medicaid); other registrants in these quitlines were eligible for the 1-call program. Eight states also offered a 10-call program to pregnant tobacco users. During the study timeframe, 6 states offered NRT through both their Phone/Web and Web-Only programs, 2 states offered NRT to Phone/Web enrollees only, and 2 states did not offer NRT through either program.

Measures

Demographic characteristics (gender, age, race/ethnicity, education, health insurance, chronic condition status) and tobacco use (type, frequency, amount, years of tobacco use, time to first use after waking, other users at home and/or work) data were collected during standard program registration that is compliant with the NAQC Minimal Data Set [11]. Participants who enrolled online were asked the same questions in a participant-facing Web enrollment process. Several state quitlines also collected custom demographic data during registration (marital status, income, sexual orientation, mental health condition status). Data that were not collected for all states are identified in the table notes.

Every unique participant who enrolls in the Web Coach website is intended to have access to the same account for life, regardless of the number of times the participant re-enrolls in a quitline program. To avoid a time-lapse bias, website utilization data for this study were limited to site interactions occurring in the 6 months (up to 185 days) after registration.

Web Coach website activity (interactions and feature use) was recorded and linked to unique participants automatically through Google Analytics and the website. Efforts were also made to record log-ins and the duration of each log-in session; however, Google Analytics was often blocked by individual users or employer networks, resulting in missing data in the log-in and session minutes fields. To circumvent this issue, engagement was determined by creating a log-in days variable, which counted 1 log-in day for every distinct date on which the participant completed a site interaction.

To classify study participants by engagement level, users were defined as participants who logged in to the Web Coach website at any time in the 6 months following registration. Return users were defined as anyone who logged in at any time in the 6 months following registration and then returned at a later date during the same 6-month time period (ie, logged in to the Web Coach website on at least 2 different days in the 6 months following registration).

Statistical Analysis

Chi-square and t test analyses were used to examine differences in characteristics and Web utilization between Phone/Web and Web-Only enrollees, as well as differences in return rates between subpopulations within each program. Bivariate post hoc analyses were completed as necessary. Multivariable logistic regression analyses were used to examine predictors of program choice and return rates within each program. Models included predictor variables that were collected by all 10 states, had limited amounts of missing data, and measured participant characteristics (age, gender, race/ethnicity, education, insurance status, chronic condition status), tobacco dependence (time to first use, cigarettes per day), and method of program entry. A secondary model for program choice also included mental health condition status, which was only asked by 6 states; this secondary model was examined because mental health condition status was a significant predictor of program choice in bivariate analyses and data were available from the majority of states. Both return user models (within Phone/Web and Web-Only) also included receipt of quitline NRT as a predictor variable; analyses examining return users within Phone/Web additionally included phone treatment intensity (1-call vs multiple-call program) and call completion (0 calls vs ≥1 calls) as predictors. Because these behaviors (eg, call completion, NRT selection, selection of the multiple-call program) occurred after tobacco users selected the Phone/Web or Web-Only program in some or all cases, these variables were not included in the program choice model. State quitline was included as a fixed effect in every model to account for pre-existing differences in services and tobacco control policies within each state. All analyses were conducted in SAS v9.3 (SAS Institute, Inc, Cary, NC, USA).

Because of the large sample size, a number of findings were statistically, but not meaningfully, significant. We used a Bonferroni adjustment to account for the large number of statistical comparisons. Results are reported as significant where P<.0001 and the absolute difference in percentage points between comparison groups rounded to 5 or greater; this significance threshold was determined post hoc after initial review of analysis findings.

Results

Participant Characteristics

Overall, the majority of registrants were female (59.29%), mean age 44.4 (SD 13.8) years, white non-Hispanic (72.90%), heterosexual (93.86%), had a high school degree (27.40%) or higher (47.27%), but an annual household income of less than US$15,000 (51.45%), were uninsured (42.62%) or commercially insured (25.47%), and were daily cigarette smokers (94.02%) at a mean rate of 19 cigarettes per day (SD 11.3) (Table 1). Nearly one-half (48.32%) used tobacco within 5 minutes of waking at the time of enrollment, indicating high nicotine dependence. Approximately one-third (32.62%) reported at least 1 of 4 chronic health conditions (asthma, chronic obstructive pulmonary disease, coronary artery disease, and/or diabetes) and nearly one-half (46.59%) reported a mental health condition diagnosis.

Characteristics of Web-Only Versus Phone/Web Enrollees

Quitline registrants were more likely to select Phone/Web over a Web-Only program (113,019/141,429, 79.91% vs 28,410/141,429, 20.09%). Participant characteristics and differences between those who opted for the Web-Only versus Phone/Web program are shown in Table 1.

Compared to Phone/Web enrollees, participants who enrolled in a Web-Only program were younger (mean 40.8, SD 12.8 vs mean 45.3, SD 14.0 years; P<.001), more likely to be white non-Hispanic (80.10% vs 71.10%; P<.001) and less likely to be black or African American non-Hispanic (6.84% vs 14.80%; P<.001). Web-Only enrollees were more highly educated (59.65% vs 44.15% had greater than a high school degree; P<.001) and reported higher household incomes (36.50% vs 54.39% reported an annual household income of less than US$15,000; 37.33% vs 30.11% reported US$15,000 to US$35,000; 26.17% vs 15.50% reported greater than US$35,000; all P<.001). Participants who opted for the Web-Only program were more likely to be uninsured (49.02% vs 41.00%; P<.001) or commercially insured (38.72% vs 22.14%; P<.001), and less likely to have Medicaid (7.87% vs 22.12%; P<.001) or Medicare coverage (4.39% vs 14.74%; P<.001). Web-Only were also more likely to be married or in a domestic partnership (45.79% vs 34.63%; P<.001), and more likely to live and/or work with other tobacco users (62.81% vs 56.24%; P<.001). Smaller proportions of Web-Only enrollees reported having a chronic health condition (24.07% vs 34.74%; P<.001) or a mental health condition (35.66% vs 49.42%; P<.001) at enrollment. Web-Only were also less likely to be highly nicotine-addicted (42.26% vs 49.86% reported using tobacco within 5 minutes of waking; P<.001) or long-term tobacco users (57.23% vs 69.34% had used tobacco for ≥20 years; P<.001). Web-Only were less likely to have started their enrollment for quitline services over the phone (45.43% vs 91.93%; P<.001) and more likely to have started their enrollment online (52.20% vs 2.91%; P<.001). There were no meaningful differences in program selection as a function of gender, pregnancy status, sexual orientation, cigarettes smoked per day, or frequency of cigarette use at enrollment.

As shown in Table 2, multivariable logistic regression analyses confirmed that participants who opted to enroll in Web-Only were younger, more highly educated, more likely to be white non-Hispanic and less likely to be black non-Hispanic, more likely to be uninsured or commercially insured, less likely to be highly nicotine dependent or have a chronic health condition, and more likely to have started their enrollment online.

Multivariable analyses also indicated that Web-Only enrollees were more likely to be female, which was significant in bivariate analyses but did not meet our meaningful significance threshold requirement of an absolute difference in percentage points rounding to 5 or greater. A secondary model (not shown) limited to the 6 states that assessed mental health condition status at registration confirmed that Web-Only were also more likely to report not having any mental health condition diagnoses (AOR 1.49, 99.99% CI 1.38-1.61; P<.001).

Utilization of Web Services and Nicotine Replacement Therapy Benefit

Table 3 summarizes Web utilization overall and between program types. Half (50.34%) of Phone/Web and all (100.00%) Web-Only registrants “enrolled” in Web services by providing their email address and consenting to be contacted via email. Among those who consented to this enrollment step, Web-Only registrants were significantly more likely than Phone/Web to log in to the Web Coach website (users) in the 6 months following their registration (76.85% vs 42.04%; P<.001). Although Web-Only were more likely to log in at least once, this group was less likely to return to the site on a later day (return users) compared to Phone/Web (40.15% vs 58.39%; P<.001). Among program participants who used the Web Coach website at least once, Phone/Web participants logged in on more days than Web-Only participants (median 2.0, IQR 1-4 vs median 1.0, IQR 1-2; P<.001).

Six states offered an NRT benefit through both their Phone/Web and Web-Only programs. In these states, Web-Only enrollees were significantly less likely to have received NRT from their quitline (50.49% vs 83.11%; P<.001). Among all 10 states (regardless of whether NRT was offered through either program), 46.00% of Web-Only versus 73.58% of Phone/Web were sent quitline NRT.

Return Users: Subpopulations More Likely to Return to Web Services

Table 4 shows the percentages of different subpopulations within Phone/Web and Web-Only logging in to the Web Coach website on 2 or more days (ie, return users). Among both Phone/Web and Web-Only, participants who were female, more highly educated, had used tobacco for 20 years or longer, were sent NRT through their quitline, or had Medicare coverage were more likely to return to the site after their initial log-in day (Table 4). Older participants were also more likely to return among both Phone/Web (age of return users: mean 43.0, SD 13.0 vs age of nonreturn users: mean 40.5, SD 12.8; P<.001) and Web-Only (mean 41.9, SD 12.7 vs mean 39.8, SD 12.5; P<.001). Phone/Web participants were also more likely to return if they had reported a higher household income or identified as white non-Hispanic or “other” race at registration; these differences in return users as a function of income and race/ethnicity were not observed in the Web-Only population. Trends in return users also differed between program groups as a function of smoking frequency reported at enrollment. Phone/Web enrollees who smoked cigarettes every day or some days at enrollment were more likely to return than those who reported smoking not at all. The opposite was found for the Web-Only population: enrollees who initially reported not smoking at all were more likely to return than Web-Only enrollees who smoked daily or only some days. For Phone/Web, participants who completed more coaching calls were also more likely to return to the Web Coach website. There were no meaningful differences in return rates that met our threshold for significance for either program group as a function of pregnancy status, marital status, chronic health or mental health condition status, tobacco environment, or nicotine dependence at enrollment.

Multivariable analyses confirmed that Phone/Web and Web-Only participants who were older, female, more highly educated, and received NRT from their quitline were more likely to return to the Web Coach website (Table 5). Patterns of return as a function of race/ethnicity were also confirmed: Phone/Web participants were more likely to return if they identified as white non-Hispanic or “other” race, whereas no difference was observed among Web-Only. Phone/Web were also significantly more likely to return if they were commercially insured and had completed at least 1 call with Quit Coach staff. Among Web-Only, differences in return rates between insurance groups did not meet our significance threshold. Web-Only were less likely to return if they started their enrollment online rather than over the phone; there were no differences in return rates by method of program entry for Phone/Web. As in bivariate analyses, there were no differences in return rates within either program as a function of chronic condition status or nicotine dependence.

Web Coach Website Feature Use

Figure 8 and Table 6 show the percentages of Web Coach website users (logged in on 1 or more days) who used key site features at least once in the 6 months following their enrollment in a program. Table 6 shows all percentages and P values, whereas Figure 8 is included to facilitate synthesis of findings. Features used by the largest percentages of participants overall included the Tobacco Tracker (65.33% of all Web Coach website users) and the Cost Savings Calculator (60.64%). Participants also completed Quitting Plan behaviors at relatively high rates; 41.91% of all users completed at least 1 behavior: Choose a Medication (38.68%), Set a Quit Date (28.93%), Conquer My Urges (28.54%), Control My Environment (25.75%), and Get Social Support (19.22%). Although 27.92% of users visited the page introducing the 4 Essential Practices of Quitting, less than half of those individuals viewed any of the Practices content in an e-lesson (13.31%), article (9.13%), or video (4.03%). Use of the Community features was also low, with 10.87% of all Web Coach website users visiting the Community area, 10.44% creating a Community account, 6.48% reading a discussion thread, and 1.99% posting in a Community discussion. Small proportions of users reached out to Quit Coach staff: 5.02% of all Phone/Web and Web-Only participants clicked to chat with a coach, and 4.85% clicked to send an email. Phone/Web participants could also request a call from a coach; 4.04% of Phone/Web used this feature.

With regard to differences in feature use between Phone/Web and Web-Only, Web-Only participants were less likely than Phone/Web to use both the Tobacco Tracker (56.87% vs 73.06%; P<.001) and Cost Savings Calculator (58.37% vs 62.72%; P<.001; only approached threshold for meaningful significance), but they were more likely to use all other key features, including completing the 5 Quitting Plan behaviors: Choose a Medication (68.05% vs 11.87%; P<.001), Set a Quit Date (56.17% vs 4.06%; P<.001), Conquer My Urges (48.22% vs 10.58%; P<.001), Control My Environment (43.75% vs 9.33%; P<.001), and Get Social Support (32.47% vs 7.12%; P<.001) (Figure 8 and Table 6). Web-Only participants were also slightly more likely to use the Urge Tracker (10.72% vs 6.42%; P<.001; only approached threshold for meaningful significance), which was available only when a participant had a self-reported status of “I’m Quit.”

‎View this figure

Discussion

Principal Results and Comparison With Prior Work

This examination of tobacco users’ enrollment in and use of Phone/Web and Web-Only tobacco cessation programs through 1 of 10 state quitlines describes program selection in a large, real-world sample, and is also the first examination of return users from Web-based versus integrated Phone/Web programs and among subpopulations within these programs. Four-fifths of the 141,429 tobacco users in the study selected cessation support through the integrated Phone/Web program. Method of entry was the strongest predictor of program selection (92% of those who started their enrollment by phone selected Phone/Web; 52% of those who started enrollment online selected Web-Only), which may indicate that the phone program is highlighted more during phone registration, in quitline advertisements, and/or through word of mouth. It is also possible that participants tend to contact the program using the modality in which they would prefer to receive services.

Those who chose to enroll in the Web-Only program were younger, healthier (in terms of chronic health condition and mental health condition status), more highly educated, less likely to have insurance through Medicaid or Medicare, more likely to be white non-Hispanic and less likely to be black non-Hispanic, and less likely to be highly nicotine-addicted. This profile of smokers may be more tech savvy and not as interested in the more intensive support available via phone. These findings and previous research indicate that Web-Only is an attractive option for the harder-to-reach population of younger smokers [7,9] and those without symptoms of depression [9]. In addition to differences in education and insurance status, among 4 states that assessed income, Web-Only enrollees had higher annual household incomes, which may relate to socioeconomic disparities in Internet access or experience using computers. If Web-based services prove to be effective for tobacco users of higher socioeconomic status, this may present an avenue for conserving tobacco control funds to provide the higher-cost phone programs [7,12] for smokers who are more highly addicted and socioeconomically disadvantaged.

Phone/Web and Web-Only enrollees engaged with Web services differently. After initial enrollment in the Web Coach website, more Web-Only participants actually logged in to the program, as would be expected by their program selection. However, Phone/Web participants who used the site were more likely to return after their initial log-in. Despite being less likely to return to the site, Web-Only participants were more likely than Phone/Web to use most key features, most notably completing the 5 Quitting Plan behaviors. These findings suggest that Web-Only participants tended to use the site for a single, intense session of quit plan development and site exploration, but typically did not return. Phone/Web participants, on the other hand, who already demonstrated their desire for support and program contact by choosing a program with phone interaction, used the Web Coach website planning features at much lower rates (likely because they had already completed planning with Quit Coach staff over the phone), but were more likely to return to the site for additional program contact. Phone/Web participants may have returned to the site because they were encouraged to do so during their ongoing coaching calls (call completion was correlated with return visits to the site), although participants may have completed more calls and returned to the site simply because they were more engaged overall. These motivations and usage profiles should be considered as program designers decide how to present and highlight Web-based cessation content to ensure participants find the most important and relevant content during their visit.

Within both the Phone/Web and Web-Only populations, participants were more likely to return to the Web Coach website after their initial log-in day if they were women, older, or more highly educated, which is in-line with the previous research [13]. Those who were sent NRT through their quitline (for both programs) and those who had commercial health insurance (among the Phone/Web program) were also more likely to return. Given findings that services are most successful [14,15] and cost-effective when utilized at higher rates [12], effective strategies for re-engaging participants across the board, or particularly for less engaged groups, are needed.

The interactive Tobacco Tracker, Cost Savings Calculator, and Quitting Plan behaviors were the most widely used features among all registrants; previous research has suggested that use of interactive Web components is associated with higher quit rates, particularly in nondepressed populations [9,14]. Program designers should continue to focus on interactive features as opposed to static informational sites. Designers should also consider how best to encourage use of key features. Among quitlines that offered NRT to both Phone/Web and Web-Only participants, Web-Only were less likely to have received NRT from their quitline program. It is unclear why Web-Only participants were less likely to take advantage of the NRT benefit through their quitline, but it may be the result of different program processes. Although a Phone/Web participant is typically guided through the process of creating a quit plan (including selection and dosing for a cessation medication) by Quit Coach staff, the Web-Only program is designed to be more self-guided; participants are required to authenticate their account, log in, and then complete the Choose a Medication behavior on their own to access NRT. In addition, Phone/Web participants with a medical use exclusion contraindicating NRT use are mailed an override letter that their physician can fax to the quitline to approve NRT for the participant; this override process has not been an option for Web-Only participants with a use exclusion. Because use of Food and Drug Administration-approved cessation medications is associated with greater odds of achieving abstinence [16], Web programs should employ strategies to promote awareness of medication options and prioritize access to cessation medication benefits.

Limitations and Future Directions

A strength of this study is that analyses were conducted with a large census sample of tobacco users from different regions of the country who registered in 10 state tobacco quitlines. However, the large sample size resulted in numerous statistically significant results that may not reflect meaningful differences; the authors used a Bonferroni adjustment to account for the large number of statistical comparisons and selected a criterion level to provide a consistent benchmark for identifying meaningful differences. We believe a 5 percentage point difference is a reasonable threshold; however, others may view smaller or larger differences to be meaningful.

Several other limitations should be noted. First, all participants self-selected their program of choice, but we do not know what factors influenced participants’ selections nor how aware tobacco users were of the services available to them in each program. In particular, we cannot know how carefully those who enrolled online read the program option descriptions. Second, not all 10 states offered identical services to all tobacco users (eg, the multiple-call program was not available to subgroups of registrants in 2 states); future work should examine the impact of different service offerings on program choice. Third, data were not available from Web-Only participants on motivation, confidence, readiness to quit, or previous quit attempts. These data would better inform whether individuals less ready or who might feel they needed less support with quitting selected the Web-Only service. It is important to note that the demographic differences found between participants who chose Web-Only versus Phone/Web may be due in part or entirely to differences in 1 or more of these variables for which data were not available. Fourth, we focused on the number of distinct log-in days as our metric of engagement and were not able to examine the specific number of log-ins or minutes on the site. Log-in days provides a more consistent estimate of use given potential variations in time before automatic logouts; however, log-ins and minutes could have provided additional context regarding typical program use. Fifth, we did not examine utilization of other resources; Web-Only users or other subgroups less likely to return to the Web Coach website may also be more likely to use other additional sources of support (eg, multiple online programs). Future research could examine this hypothesis and determine whether encouraging sustained engagement in a single evidence-based program produces the best outcomes. Sixth, there was not a mobile accessible/compatible version of the Web Coach website at the time of this study. Future research should examine whether mobile accessible websites change how participants engage with Web-based programs, especially in terms of differences in use between various levels of socioeconomic status. Moreover, because Web features can change, feature use could be impacted by changes in site design, which can affect the replicability of these findings and comparisons across Web-based programs.

This paper was not about outcomes; although the effectiveness of phone-based cessation programs has been established [17] and randomized controlled trials have provided support for Web-based cessation services [10], data were not available to evaluate the effectiveness of the Phone/Web and Web-Only programs for participants in this study, and the Centers for Disease Control and Prevention has not yet deemed Web-based services as having a sufficient evidence base in their 2014 Best Practice Guidelines [18]. More research is needed to better understand the effectiveness of Web-Only cessation services for populations who select into that service, and reasons tobacco users select Web-Only over integrated Phone/Web.

Conclusions

Understanding who is selecting different tobacco cessation program modalities and how they engage with the program in a real-world setting will help the scientific and treatment community to better understand program outcomes and can inform engagement and re-engagement strategies. Our findings suggest that a Web-based program attracts younger, healthier smokers of higher socioeconomic status who interact more intensely with services in a single session, but are also less likely to re-engage or access the important NRT benefits available to them. Further research is needed to examine the efficacy of different engagement techniques and services with different subpopulations of tobacco users.