This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297) Substance Abuse (419)

Published on in Vol 20, No 5 (2018): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9579, first published .
Effects of Treatment Length and Chat-Based Counseling in a Web-Based Intervention for Cannabis Users: Randomized Factorial Trial

Effects of Treatment Length and Chat-Based Counseling in a Web-Based Intervention for Cannabis Users: Randomized Factorial Trial

Effects of Treatment Length and Chat-Based Counseling in a Web-Based Intervention for Cannabis Users: Randomized Factorial Trial

Authors of this article:

Benjamin Jonas1 Author Orcid Image ;   Marc-Dennan Tensil1 Author Orcid Image ;   Peter Tossmann1 Author Orcid Image ;   Evelin Strüber2 Author Orcid Image
Article Authors Cited by (11) Tweetations (5) Metrics

    Original Paper

    1Delphi Gesellschaft für Forschung, Beratung und Projektentwicklung, Berlin, Germany

    2Federal Centre for Health Education, Cologne, Germany

    Corresponding Author:

    Benjamin Jonas, Dipl-Psych

    Delphi Gesellschaft für Forschung, Beratung und Projektentwicklung

    Kaiserdamm 8

    Berlin, 14057

    Germany

    Phone: 49 30 39 40 97 82

    Fax:49 30 39 40 97 89

    Email: jonas@delphi.de


    Background: Digital interventions show promise in reducing problematic cannabis use. However, little is known about the effect of moderators in such interventions. The therapist-guided internet intervention Quit the Shit provides 50 days of chat-based (synchronous) and time-lagged (asynchronous) counseling.

    Objective: In the study, we examined whether the effectiveness of Quit the Shit is reduced by shortening the program or by removing the chat-based counseling option.

    Methods: We conducted a purely Web-based randomized experimental trial using a two-factorial design (factor 1: real-time-counseling via text-chat: yes vs no; factor 2: intervention duration: 50 days vs 28 days). Participants were recruited on the Quit the Shit website. Follow-ups were conducted 3, 6, and 12 months after randomization. Primary outcome was cannabis-use days during the past 30 days using a Timeline Followback procedure. Secondary outcomes were cannabis quantity, cannabis-use events, cannabis dependency (Severity of Dependence Scale), treatment satisfaction (Client Satisfaction Questionnaire), and working alliance (Working Alliance Inventory-short revised).

    Results: In total, 534 participants were included in the trial. Follow-up rates were 47.2% (252/534) after 3 months, 38.2% (204/534) after 6 months, and 25.3% (135/534) after 12 months. Provision of real-time counseling (factor 1) was not significantly associated with any cannabis-related outcome but with higher treatment satisfaction (P=.001, d=0.34) and stronger working alliance (P=.008, d=0.22). In factor 2, no significant differences were found in any outcome. The reduction of cannabis use among all study participants was strong (P<.001, d≥1.13).

    Conclusions: The reduction of program length and the waiver of synchronous communication have no meaningful impact on the effectiveness of Quit the Shit. It therefore seems tenable to abbreviate the program and to offer a self-guided start into Quit the Shit. Due to its positive impact on treatment satisfaction and working alliance, chat-based counseling nevertheless should be provided in Quit the Shit.

    Trial Registration: International Standard Randomized Controlled Trial Number ISRCTN99818059; http://www.isrctn.com/ISRCTN99818059 (Archived by WebCite at http://www.webcitation.org/6uVDeJjfD)

    J Med Internet Res 2018;20(5):e166

    doi:10.2196/jmir.9579

    Keywords

    cannabis; Internet; counseling; random allocation 


    Cannabis is the most widely used illegal drug in Europe with a last-year prevalence of 14% among young adults and around 1% daily or almost daily users [1]. In 2015, cannabis use of 550,000 adults in Germany was clinically relevant [2].

    In the past years, several online interventions targeting individuals with cannabis use disorder (CUD) were made available to the public in Europe [3]. The guided program “Quit the Shit” (QTS) is the only intervention that has been tested in a randomized controlled trial (RCT) [4-7]. Besides the Australian intervention “Reduce your use,” QTS is currently the only evidence-based internet intervention targeting cannabis users that is freely available for the public [8].

    As part of the drug prevention website drugcom, QTS is operated by the German Federal Centre for Health Education (BZgA) since 2004 and is one of the components of the prevention strategy for CUD of the BZgA [4,9]. With 50 days of individual counseling by trained therapists, QTS offers more intense support than most other evaluated interventions related to cannabis use [5,6]. Free and anonymous QTS provides direct (synchronous) and time-lagged (asynchronous) counseling. Synchronous counseling via live chat is mainly offered during admission and termination of the intervention. Asynchronous support is delivered by weekly feedbacks on participants’ input in the cannabis use diary and exercises of QTS. The intervention is described in detail in the Methods section.

    To account for the increasing demand of QTS and to make the program less dependent on prescheduled chat counseling, our interest was to test whether shortening the intervention and whether eliminating chat-based counseling has negative impact on the effectiveness of QTS. A shorter version of QTS would presumably reduce the counselor’s effort per client and thereby allow increasing the number of participants. Moreover, a program without synchronous communication would be more flexible because participants would not be dependent on prescheduled chats to start the intervention. However, those changes should not significantly reduce the effectiveness of QTS or decrease the user satisfaction or working relation between the counselor and client.

    Although internet interventions targeting substance-related disorders have shown to be effective, little is known about moderators of their effectiveness [6,10]. Despite some indication in favor of longer alcohol-related treatments, evidence is still mixed [10]. For internet interventions targeting CUD, there is no such evidence in either direction [6]. Although internet interventions based on synchronous interaction have shown to be generally effective [11], their effects compared with asynchronous-only counseling were not investigated systematically, pointing to a need of further research in this area [12]. Evidence from an earlier meta-analysis indicated no better performance of either synchronous or asynchronous mode of interaction [13].

    To secure the planned changes to QTS with empirical evidence, we therefore examined whether shortening the intervention or eliminating chat-based counseling has negative impact on the program. The main outcome was cannabis use frequency during the past 30 days. Secondary outcomes were cannabis quantity, cannabis use events, cannabis dependency, treatment satisfaction, and working alliance.


    Study Design

    We conducted a purely Web-based pragmatic randomized experimental trial using a two-factorial design. In the study, participants were allocated to 1 of the 4 versions of QTS (Table 1). In the first experimental factor, we compared intervention versions containing chat-based counseling with versions which only consisted of counseling via asynchronous communication channels. In the second factor, program versions with the regular length of 50 days were compared with versions with the reduced duration of 28 days.

    The study was conducted on the website of the intervention. Trial participants were directly recruited from all individuals who were interested in signing up for QTS. At the beginning of the program registration, potential participants were informed about the study. A PDF file containing all relevant study details was offered for download and in the confirmation email for study participants. The PDF file is included in the Multimedia Appendix 1. Individuals who were willing to participate were asked to register and provide their informed consent by checking an “I agree to participate” checkbox. The study outcomes were included in the regular registration questionnaire of QTS. Users of the intervention who opted not to participate in the study or who did not meet the eligibility criteria had full access to the regular version of the intervention and were not included in any follow-up surveys.

    After registration, study participants were to choose an appointment from a schedule provided by QTS. By logging into the program at this appointment, participants were randomized automatically to 1 of the 4 intervention versions and directly forwarded into the program. Neither the researchers nor the counselors could influence or predict the randomization result. Participants were blind to the results of the randomization because they only received detailed information about the program version they were allocated to. None of the intervention versions underwent any changes during the evaluation process.

    Follow-up surveys were conducted in online questionnaires 3, 6, and 12 months after randomization. Each follow up participation was compensated for with a gift voucher for a major internet-based retailer worth 10 euros.

    The study was approved by the ethics committee of the Department of Applied Human Sciences at the University of Magdeburg-Stendal (Ref 4973-35) and was registered with ISRCTN (ISRCTN99818059). Note: In the registry, the study design was erroneously declared as RCT. The right indication, however, should have been randomized factorial trial.

    Table 1. Experimental design.
    Experimental designFactor 2: Intervention length

    		50 days28 days
    Factor 1: Chat-based counseling


    		YesVersion 1Version 2
    NoVersion 3Version 4

    Measures

    Cannabis use was measured using the Timeline Followback method (TLFB) [14]. In the TLFB form, participants had to indicate the number of joints, bongs, and other cannabis use for each day in the past 30 days in a calendar. They were encouraged to use anchor events such as birthdays, appointments, or holidays to get a better orientation. The number of cannabis use days and the number of use events (ie, the sum of joints, bongs, and other cannabis use) were derived from the input from the TLFB form. To measure the cannabis quantity in grams, participants were asked to estimate this sum over the past 30 days.

    Cannabis dependence was measured by the German version of the Severity of Dependence Scale (SDS) [15,16], changing the reference period from 12 to 3 months. Participants with a SDS score of at least 4 points were categorized as cannabis dependent [17]. Effects on treatment satisfaction were measured using the German version of the Client Satisfaction Questionnaire (CSQ-8) [18,19]. Data on the working alliance between counselor and participant were collected using the German version of the Working Alliance Inventory-short revised (WAI-sr) [20,21]. The CSQ-8 and WAI-sr were not listed in the study registry, as they were introduced after registering the study.

    As a measure of the program usage, we tracked the number of logins and the duration of chat-based counseling for each participant.

    Study Criteria

    To be eligible for the study, individuals had to be at least 18 years old and to be first-time users of the intervention. Exclusion criteria were alcohol use disorder operationalized by a score of at least 3 in the CAGE [22], diagnosed psychotic or bipolar disorder based on self-report, current use of other illicit drugs on more than 4 days during the last 30 days, and suicidal thoughts as measured by selected items of Becks Scale for Suicide Ideation [23]. Individuals who displayed suicidal thoughts were given detailed information on suitable psychosocial support options such as telephone helplines or local institutions.

    Interventions

    Version 1 (Regular Quit the Shit)

    The first intervention version of the trial was identical to the regular QTS program and therefore was 50 days long. QTS in general is based on the principles of self-regulation and self-control [24]. The weekly feedbacks are based on the solution-focused approach [25] and motivational interviewing [26]. Therefore, the participants’ responsibility to achieve personal change is accentuated, current personal developments and achievements are reviewed, and clear recommendations for further steps concerning the solution of problems are given. After the registration, the intervention comprises 3 consecutive phases:

    First, admission takes place during a prescheduled appointment in a one-to-one chat with a counselor of QTS. The chat takes 50 min and is mandatory to enter QTS. It aims at clarifying the situation of the client and helps determine individual consumption goals and coping strategies. Usually, each user stays with the same counselor throughout the whole program.

    Second, after the admission chat, the login area of QTS is activated. It contains a diary where participants are required to write down all relevant aspects of their cannabis use over the whole duration of 50 days. Moreover, the login area includes coping exercises, for example, aiming to develop control strategies, enhancing quality of life, or balancing the pros and cons of using cannabis. Once a week, participants receive detailed feedback by their counselor on their entries in diary and exercises. Depending on the involvement of each participant, up to 7 feedbacks are given. They discuss the current cannabis use, the psychosocial situation, and the counseling process as such.

    Third, at the end of the program, clients are invited to a concluding chat, where the initial consumption goals and the applied control strategies are reflected. If necessary, participants are referred to local addiction counseling or therapy.

    Version 2

    Version 2 is an abbreviated variant of the original intervention and thus is only usable for 28 days. As the counselor feedback is timed weekly, version 2 therefore includes only 4 feedbacks as opposed to 7 in version 1. Apart from that, there are no differences.

    Version 3

    In contrast to versions 1 and 2, this variant of QTS does not offer synchronous communication between the counselor and client. Instead of starting the intervention via live chat, users of version 3 are to describe their current situation and their program goals in a self-guided tour. In open text boxes, participants are requested to outline their cannabis use, their personal pros and cons for using cannabis, and their ideas of what or who might help them to reduce their use. After concluding the self-guided tour, the login area, as described above, is activated automatically. The first counselor feedback 1 week later refers to the first-week entries in diary and to the input in the self-guided tour.

    Like version 1, this variant of QTS is 50 days long and therefore provides up to 7 weekly feedbacks. Instead of a live chat at the end of the program, the last feedback is used to summarize the progress made during participation, to reconsider the working strategies, and to determine whether further professional help is required.

    Version 4

    Identical to version 3, version 4 does not include chat-based interaction and instead consists of the same self-guided admission procedure as described above. The only difference in comparison with version 3 is the shortened duration of 28 days. For a comparison of the intervention versions, see Table 2. A screenshot of the QTS home page is shown in Figure 1. Furthermore, a screenshot of the program diary is provided in Multimedia Appendix 2.

    Statistical Analysis

    Generalized estimating equations were used to examine the effects of the experimental factors on all cannabis-related study outcomes (ie, cannabis use and cannabis dependence). Treatment-related satisfaction and working alliance only were collected at the first follow-up. To measure the effects on these outcomes, we therefore used generalized linear models. In a first step of data analysis, we included both factors (Table 1) as main effects, the interaction of each factor with time, the interaction of both factors with each other, and the 3-way interaction of both factors and time in each model. Moreover, we tested whether group differences at baseline and group differences in follow-up participation moderated the factorial effects on each study outcome. If statistically significant, the respective term and its interaction with each factor were included in the models. Otherwise, it was not considered in the effectiveness testing. We assumed factorial effects on each outcome if the interaction between the respective factor and the time variable was statistically significant. To measure the overall development for each outcome, the main effect of time was examined.

    The study was powered to detect a difference between factor levels in the reduction of cannabis use frequency (ie, consumption days in the past 30 days) of at least 20%. We utilized means and SDs of an earlier trial [4] to compute the associated effect size [27]. In that trial, participants of the regular QTS intervention reduced their cannabis use by approximately 14 days (SD=12.0). We, therefore, aimed to detect a difference between 14 days and 11.2 days [14 days × (1−0.2)] reduction. To detect the corresponding effect size of f=0.12, a total sample of n=552 is required (n=138 for each cell of the factorial design; two-sided alpha=.05; power=0.80).

    We conducted intention-to-treat analyses, including all randomized participants according to their group allocation. Missing data were estimated by multiple imputations. We performed 50 imputations. The effectiveness results of the imputed datasets were compared with the results of the nonimputed dataset (completer-only analyses).

    Logistic regression analyses were conducted to compare study participants with study nonparticipants (ie, regular users of QTS) at baseline, to compare factor levels at baseline, and to determine whether baseline measures were predicting follow-up participation.

    All analyses were conducted with R 3.4.1 (R Foundation for Statistical Computing, Vienna, Austria) [28], utilizing the following commands: glm [28], geeglm [29], and des [30]. Multiple imputations were estimated with R’s mice package [31].

    Table 2. Comparison of the interventions.
    CharacteristicsVersion 1aVersion 2Version 3Version 4
    Duration50 days28 days50 days28 days
    Chat-based counselingYesYesNoNo
    Registration (baseline)Registration questionnaireRegistration questionnaireRegistration questionnaireRegistration questionnaire
    Admission to interventionLive chat (~50 min)Live chat (~50 min)Self-guidedSelf-guided
    Cannabis use diary and exercisesUp to 50 daysUp to 28 daysUp to 50 daysUp to 28 days
    Weekly feedbackUp to 7Up to 4Up to 7Up to 4
    Conclusion of interventionLive chat (~30 min)Live chat (~30 min)The last weekly feedbackThe last weekly feedback

    aVersion 1 is identical to the original Quit the Shit intervention.

    ‎
    Figure 1. Home page of “Quit the Shit” (QTS).
    View this figure

    Flow of Participants

    During the study, 876 individuals accessed the baseline questionnaire of QTS and therefore were assessed for eligibility (Figure 2). In total, 339 individuals did not take part in the study, mainly because they refused to participate (n=239). In total, 100 persons did not meet all study criteria, primarily due to problematic alcohol use (n=46), suicidal thoughts (n=34), or because they had used QTS before (n=21). The randomization of the 534 participants resulted in similar-sized study groups. In total, 252 individuals provided data at the first follow-up, 204 at the second, and 135 participants filled out the last follow-up survey 12 months after randomization, resulting in follow-up rates of 47.2%, 38.2%, and 25.3%, respectively.

    Follow-up participation was significantly predicted by a higher age (OR 1.03, 95% CI 1.00-1.06, P=.02, d=0.19), lower cannabis quantity (OR 0.99, 95%-CI: 0.98-1.00, P=.004, d=0.25), higher school education (OR 1.76, 95% CI 1.35-2.30, P<.001), and higher number of logins during program participation (OR 1.05, 95% CI 1.04-1.06, P<.001, d=0.88). The allocation to either factor level, however, was no significant predictor for follow-up participation (Factor 1: OR 0.43, 95% CI 0.18-0.99, P=.05; Factor 2: OR 1.05, 95% CI 0.71-1.54, P=.82).

    Sample Description

    Baseline characteristics and program usage of the study participants are shown in Table 3. The majority of participants were male (65.7%) and had a high educational level with 64.7% attending or having successfully finished the highest German secondary school type (“Gymnasium”) [32]. As expected, cannabis use was high with only few abstinent days during the last month.

    Individuals who used the regular QTS intervention without taking part in the study (n=339, see Figure 2) had comparable values in most baseline variables. The only exception was found in the cannabis use days, which was slightly higher among individuals who were excluded from the study (OR 1.02, 95% CI 1.01-1.04, P=.01, d=0.19).

    Except for a small age-related difference in factor 2 (OR 0.97, 95% CI 0.95-0.996, P=.03, d=0.20), randomization resulted in similar groups. As expected, the duration of chat-based counseling differs within factor 1 (OR 1.08, 95% CI 1.07-1.09, P<.001, d=2.72) with 105.3 min of counseling chats among participants who were in the chat-based versions of QTS. In factor 2, individuals who used the longer versions logged in significantly more often than participants allocated to the shorter versions (OR 1.01, 95% CI 1.01-1.02, P=.001, d=0.32). However, with 31.6 as opposed to 23.2 logins, that number is disproportionally low regarding the difference of 50 versus 28 days program length.

    Comparison of Effectiveness

    The effectiveness results reveal no significant differences between factor levels in any of the cannabis-related outcomes both in the imputed and in the nonimputed dataset (Tables 4 and 5; Multimedia Appendix 3). In working alliance and treatment satisfaction, however, significant differences with small effect sizes were found favoring chat-based communication.

    None of the 3-way interactions (factor 1×factor 2×time) on the cannabis-related outcomes were significant, suggesting no relevant effectiveness differences between particular program versions (eg, versions 1 and 4). In the imputed dataset, this also applies to the two-way interactions (factor 1×factor 2) on working alliance and treatment satisfaction, which were only measured during the first follow-up (see Table 5). However, in the nonimputed dataset, we found significant two-way interactions on working alliance and treatment satisfaction (WAI: beta=−.55, CI −1.00 to −0.10, P=.02; CSQ: beta=−.41, CI −0.72 to −0.10, P=.01; Multimedia Appendix 3). The highest WAI and CSQ ratings were identified in the short version including chat-based counseling and the lowest ratings were found in the short version without chat-based counseling (see Multimedia Appendix 3).

    Significant and strong time effects indicate a great overall reduction of cannabis use and use-related symptoms. The strongest reduction in the imputed dataset is found in the cannabis use days (beta=−.34, CI −0.45 to −0.23, P<.001, d=2.05) followed by the number of use events (beta=−.51, CI −0.68 to −0.34, P<.001, d=1.21; see Multimedia Appendix 3). The proportion of participants with cannabis dependence dropped from 98.5% during baseline to 78.4% (3 months), 67.0% (6 months), and 62.6% (12 months).

    ‎
    Figure 2. CONSORT (Consolidated Standards of Reporting Trials) flow diagram of participants.
    View this figure
    Table 3. Participant characteristics at baseline and usage of the intervention.
    CharacteristicsFactor 1: Chat-based communicationFactor 2: LengthAll participants (n=534)

    		No (n=263)Yes (n=271)28 days (n=266)50 days (n=268)
    Intervention versions3, 41, 22, 41, 31, 2, 3, 4
    Gender





    		Female, n (%)85 (32.3)98 (36.2)91 (34.2)92 (34.3)183 (34.3)

    		Male, n (%)178 (67.7)173 (63.8)175 (65.8)176 (65.7)351 (65.7)

    		Age, mean (SD)27.5 (7.3)27.6 (6.7)28.2 (7.1)26.8 (6.8)27.5 (7.0)
    Educational level, n (%)





    		Basic school (Hauptschule)25 (9.5)29 (10.7)30 (11.3)24 (9.0)54 (10.1)

    		Middle school (Realschule)64 (24.3)57 (21.0)61 (22.9)60 (22.4)121 (22.7)

    		High school (Gymnasium)165 (62.7)181 (66.8)167 (62.8)179 (66.8)346 (64.8)

    		Other school9 (3.4)4 (1.5)8 (3.0)5 (1.9)13 (2.4)
    Cannabis





    		Use daysa, mean (SD)24.7 (7.3)25.1 (6.5)24.9 (7.0)24.9 (6.8)24.9 (6.9)

    		Use eventsa, mean (SD)122.5 (111.6)120.1 (104.1)123.7 (108.9)118.9 (106.8)121.2 (107.7)

    		Quantity (grams)a, mean (SD)23.2 (18.8)21.3 (18.6)23.2 (19.6)21.3 (17.8)22.2 (18.7)

    		SDSb,c, mean (SD)9.9 (2.8)10.0 (2.7)10.1 (2.5)9.8 (2.9)10.0 (2.7)

    		SDSb,c cannabis dependence, n (%)260 (98.9)266 (98.2)265 (99.6)261 (97.4)526 (98.5)

    		Currently no professional help, n (%)196 (74.4)217 (80.1)206 (77.4)207 (77.2)413 (77.3)
    Usage of the intervention, mean (SD)




    		Number of logins25.1 (26.4)29.7 (27.2)23.2 (20.8)31.6 (31.2)27.4 (26.9)

    		Chat-based counseling (min)3.2 (30.1)105.3 (43.8)53.8 (59.9)56.8 (66.9)55.3 (63.5)

    aDuring the past 30 days.

    bSDS: Severity of Dependence Scale.

    cCutoff of ≥4 for cannabis dependence [17].

    Table 4. Outcome scores at all 4 study points.
    CharacteristicsFactor 1: Chat-based communicationaFactor 2: Lengtha

    		No (n=263)Yes (n=271)28 days (n=266)50 days (n=268)
    Cannabis useb (days), mean (SD)




    		Baseline24.7 (7.3)25.1 (6.5)24.9 (7.0)24.9 (6.8)

    		3 months8.1 (9.4)7.6 (9.6)8.6 (9.9)7.1 (9.1)

    		6 months8.5 (9.7)8.5 (9.9)9.4 (10.2)7.6 (9.3)

    		12 months9.6 (10.3)9.1 (10.3)10.3 (10.6)8.4 (9.9)
    Cannabis useb (number of events), mean (SD)




    		Baseline122.5 (111.6)120.1 (104.1)123.7 (108.9)118.9 (106.8)

    		3 months25.2 (43.5)20.3 (35.7)25.3 (44.3)20.1 (35.0)

    		6 months29.1 (48.9)25.5 (45.0)30.4 (49.8)24.2 (44.1)

    		12 months33.9 (55.2)29.3 (51.1)35.2 (56.4)27.9 (49.5)
    Cannabis useb (grams), mean (SD)




    		Baseline23.2 (18.8)21.3 (18.6)23.2 (19.6)21.3 (17.8)

    		3 months6.4 (9.8)5.0 (8.2)6.3 (9.7)5.2 (8.2)

    		6 months6.5 (9.5)5.5 (8.7)6.9 (10.1)5.1 (8.0)

    		12 months7.3 (10.6)6.8 (10.4)7.6 (10.8)6.5 (10.1)
    Cannabis dependence (SDSc), mean (SD)




    		Baseline9.9 (2.8)10.0 (2.7)10.1 (2.5)9.8 (2.9)

    		3 months7.2 (3.5)6.8 (3.6)7.0 (3.5)6.9 (3.6)

    		6 months5.4 (3.5)5.1 (3.8)5.4 (3.6)5.1 (3.7)

    		12 months5.5 (3.6)5.4 (3.8)5.7 (3.6)5.2 (3.8)
    Working alliance (WAI-srd), mean (SD)




    		3 months3.3 (1.0)3.5 (0.9)3.5 (0.9)3.3 (1.0)
    Treatment satisfaction (CSQ-8e), mean (SD)




    		3 months1.8 (0.7)2.1 (0.7)2.0 (0.6)1.9 (0.7)

    aIntention-to-treat analyses following multiple imputation. Results of the nonimputed datasets can be found in Multimedia Appendix 3.

    bDuring the past 30 days.

    cSDS: Severity of Dependence Scale.

    dWAI-sr: Working Alliance Inventory-short revised.

    eCSQ: Client Satisfaction Questionnaire.

    Table 5. Group comparisons and interactions between both factors.
    CharacteristicsGroup difference: chat no versus yesa,bGroup difference: 28 versus 50 daysa,bInteraction factor 1×factor 2×timea,c

    		Beta (95% CI)P valueEffect size d (95% CI)Beta (95% CI)P valueEffect size d (95% CI)Beta (95% CI)P value
    Cannabis used (days), mean (SD)







    		Baseline−.02 (−0.14 to 0.10).74N/Ae−.09 (−0.23 to 0.05).20N/A−.01 (−0.18 to 0.16).91

    		3 monthsN/AN/A0.10 (−0.07 to 0.27)N/AN/A0.15 (−0.02 to 0.32)N/AN/A

    		6 monthsN/AN/A0.07 (−0.10 to 0.24)N/AN/A0.18 (0.01 to 0.35)N/AN/A

    		12 monthsN/AN/A0.10 (−0.07 to 0.27)N/AN/A0.17 (0.00 to 0.35)N/AN/A
    Cannabis used (number of events), mean (SD)






    		Baseline−.05 (−0.25 to 0.16).64N/A−.09 (−0.31 to 0.13).43N/A−.03 (−0.30 to 0.24).82

    		3 monthsN/AN/A0.10 (−0.07 to 0.27)N/AN/A0.09 (−0.08 to 0.26)N/AN/A

    		6 monthsN/AN/A0.05 (−0.12 to 0.22)N/AN/A0.09 (−0.08 to 0.26)N/AN/A

    		12 monthsN/AN/A0.06 (−0.11 to 0.23)N/AN/A0.09 (−0.08 to 0.26)N/AN/A
    Cannabis used (grams), mean (SD)







    		Baseline.01 (−0.17 to 0.19).88N/A−.04 (−0.23 to 0.15).70N/A−.05 (−0.27 to 0.18).69

    		3 monthsN/AN/A0.06 (−0.11 to 0.23)N/AN/A0.02 (−0.15 to 0.19)N/AN/A

    		6 monthsN/AN/A0.00 (−0.17 to 0.17)N/AN/A0.09 (−0.08 to 0.26)N/AN/A

    		12 monthsN/AN/A−0.06 (−0.23 to 0.11)N/AN/A0.01 (−0.16 to 0.18)N/AN/A
    Cannabis dependence (SDSf), mean (SD)






    		Baseline−.02 (−0.09 to 0.05).63N/A−.02 (−0.0 to 0.05).57N/A.01 (−0.09 to 0.10).91

    		3 monthsN/AN/A0.14 (−0.03 to 0.31)N/AN/A−0.10 (−0.27 to 0.07)N/AN/A

    		6 monthsN/AN/A0.11 (−0.06 to 0.28)N/AN/A−0.04 (−0.21 to 0.13)N/AN/A

    		12 monthsN/AN/A0.06 (−0.11 to 0.23)N/AN/A0.00 (−0.17 to 0.17)N/AN/A
    Working alliance (WAI-srg), mean (SD)






    		3 months.36 (0.09 to 0.63).0080.22 (0.05 to 0.39)−.05 (−0.32 to 0.22).71−0.21 (−0.38 to −0.04)−.27 (−0.64 to 0.09).15
    Treatment satisfaction (CSQ-8>h), mean (SD)






    		3 months.33 (0.14 to 0.52).0010.34 (0.17 to 0.51)−.03 (−0.21 to 0.16).76−0.17 (−0.34 to 0.00)−.16 (−0.42 to 0.10).22

    aIntention-to-treat analyses following multiple imputation. Results of the nonimputed datasets can be found in Multimedia Appendix 3.

    bBetween-group comparisons were conducted with the interaction of each factor with time, except for the effects on WAI-sr and CSQ-8 which were analyzed with the main effect of each factor.

    cEffects on WAI-sr and CSQ-8 were analyzed with the interaction between factor 1 and factor 2.

    dDuring the past 30 days.

    eN/A: not applicable.

    fSDS: Severity of Dependence Scale.

    gWAI-sr: Working Alliance Inventory-short revised.

    hCSQ-8: Client Satisfaction Questionnaire.


    Principal Findings

    This study examined whether the effectiveness of the internet intervention QTS is reduced by removing chat-based counseling or by shortening the program. To our knowledge, it is the first trial designed to systematically analyze these key characteristics of guided internet interventions. According to the results, neither of these changes has meaningful impact on the effectiveness of QTS. As study participants were comparable to nonparticipants, we assume results can be generalized to regular users of QTS.

    The results correspond to outcomes of meta-analyses, which also found no effects of changes in program duration on the outcomes of internet interventions for substance users [6,33,34]. In QTS, similar results of the shorter and longer program versions might be explained by a relatively fast onset of effects in the first weeks of participation and by a stabilization of these effects afterwards. Similar developments of use-related outcomes were found in another trial about an internet intervention for cannabis users [35].

    The similar effects of the longer versions might also be related to a disproportionally low user engagement in these versions, a pervasive phenomenon coined as the law of attrition [36]. Therefore, the increment of received support was probably too small to significantly enhance the effects as compared with the shorter program versions.

    The similar performance of the nonchat-based program versions corresponds with results of an earlier meta-analysis, which found no effectiveness difference between synchronous and asynchronous communication [13]. Our results suggest that the removal of chat-based counseling can be compensated by other elements of QTS, like the self-guided tour at the beginning or by enriching the weekly feedbacks with more information. However, extended feedbacks might not be sufficient to compensate a lack of an effective intervention element, as results from another study in this field of research suggest [37]. In contrast to our results, Schaub et al [35] found additional effects of chat counseling in a Web-based intervention for cannabis users. One key reason for the differences between their and our findings may be due to the reference conditions: the nonchat-group (active control) in the Swiss study received an automated self-help program, whereas the nonchat-conditions in our study included therapist guidance. Therefore, in our study, the chat and nonchat groups were probably more similar in terms of received support than the corresponding group in the study of Schaub and colleagues.

    In contrast to the nonsignificant cannabis-related outcomes, we found a stronger working alliance and higher satisfaction ratings among users of the chat-based versions. These results are in line with outcomes of an earlier comparison study [38], supporting the assumption that direct interaction leads to a closer cooperation between the client and counselor and thus to better satisfaction ratings. These findings, therefore, should be taken into account in future modifications of QTS.

    With a within-group effect size of d=2.05 between study baseline and the 3-month follow-up, study participants strongly reduced the frequency of their cannabis use. This effect surpasses the reductions found among QTS participants in our earlier trial (within-group d=1.47 for use frequency) [4,39] and also goes beyond the effects found among users of the Web-based intervention with chat counseling studied by Schaub et al (within-group d=0.75 for use frequency) [35] and the reductions found in the self-guided treatment examined by Rooke and colleagues (within-group d=1.08 for use frequency) [40]. Although within-group changes should always be interpreted carefully, the strong overall reductions in this study presumably reflect the high level of support provided by QTS.

    Strengths and Limitations

    We took several measures to ensure validity of the study results. We strictly adhered to the CONSORT rules, implemented a randomized factorial study design, tested and controlled potential confounders in the analyses, and compared results from the main analyses with those of completer-only analyses. Furthermore, the original intervention was already successfully tested in a randomized study [4] and is conducted by qualified staff with several years’ experiences in online counseling.

    As in other studies in this field of research [4,35,40,41], a major weakness is the low follow-up rate. Although we applied multiple imputations on the dataset, validity of the longer-term results is probably reduced. However, results are coherent across follow ups and, except for the significant interaction on the WAI and CSQ score, across datasets. Nevertheless, future studies in this field of research should look for ways of decreasing participant attrition. Compared with our earlier trial on the effectiveness of QTS [4], we were able to increase follow-up rates significantly. We suppose this mainly goes back to improvements in the follow up-recruitment, like offering shopping vouchers for each follow-up, emphasizing the short duration of each questionnaire, repeatedly inviting every participant for each follow-up, and addressing each nonresponder personally.

    As all purely Web-based RCTs, we relied on self-reported data. This poses a threat to validity, especially for the cannabis-related outcomes, as cannabis use still is illegal in Germany and therefore might be understated by the participants. However, we deemed a biological validation of these outcomes unfeasible, as the collection of biological markers is associated with high costs, a low measurement precision, and a narrow selection of participants to those who are willing to provide these data. It must also be noted that the outcomes “quantity of cannabis” and “number of use events” did not account for the type of cannabis product (eg, hashish, marijuana), its administration (eg, joint, bong), or its THC content. Therefore, the measurement precision of these outcomes is probably reduced. Other studies aim to improve the estimation by a standardization formula [37,40] or by gaining more detailed information on the cannabis product typically used [35]. Despite their apparent advantages, we chose not to use either of these procedures as they are not validated for application in the German setting.

    Conclusions

    The reduction of program length and the waiver of synchronous communication have no meaningful impact on the effectiveness of QTS. It therefore seems tenable to abbreviate the program and to offer a self-guided start into QTS. As chat-based counseling shows higher user ratings, it should be provided for those users who prefer to be supported that way.

    Acknowledgments

    The study was funded by the Federal Centre for Health Education (BZgA). The authors thank all counselors of QTS involved in the study: Ilka Andersen, Ingrid Lechner, Benjamin Lemke, Evi Schunack, and Reglinde Schöbl.

    Authors' Contributions

    All authors conceived the study and participated in its design. BJ coordinated the study, the data collection, conducted the analysis, and drafted the manuscript. MT and ES carried out the modifications of QTS necessary for the study. All the authors approved the final version of the manuscript submitted for publication.

    Conflicts of Interest

    BJ, MT, and PT are researchers at Delphi Gesellschaft, which developed “Quit the Shit” on behalf of the BZgA. ES is the project manager for drugcom website and “Quit the Shit” at the BZgA.

    ‎

    Multimedia Appendix 1

    Information for trial participants.

    PDF File (Adobe PDF File), 92KB
    ‎

    Multimedia Appendix 2

    Screenshot of the program diary.

    PNG File, 198KB
    ‎

    Multimedia Appendix 3

    Additional Tables.

    PDF File (Adobe PDF File), 55KB
    ‎

    Multimedia Appendix 4

    CONSORT‐EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 9MB
    1. European Monitoring Centre for Drugs and Drug Addiction. European Drug Report: Trends and Developments. Luxembourg: Publications Office of the European Union; 2017.
    2. de Matos EG, Atzendorf J, Kraus L, Piontek D. Substanzkonsum in der Allgemeinbevölkerung in Deutschland. SUCHT 2016 Oct;62(5):271-281. [CrossRef]
    3. European Monitoring Centre for Drugs and Drug Addiction. 2014. Internet-based drug treatment (Perspectives on drugs)   URL: http://www.emcdda.europa.eu/publications/pods/internet-based-drug-treatment [WebCite Cache]
    4. Tossmann P, Jonas B, Tensil M, Lang P, Strüber E. A controlled trial of an internet-based intervention program for cannabis users. Cyberpsychol Behav Soc Netw 2011 Nov;14(11):673-679. [CrossRef] [Medline]
    5. Hoch E, Preuss UW, Ferri M, Simon R. Digital interventions for problematic cannabis users in non-clinical settings: findings from a systematic review and meta-analysis. Eur Addict Res 2016;22(5):233-242 [FREE Full text] [CrossRef] [Medline]
    6. Tait RJ, Spijkerman R, Riper H. Internet and computer based interventions for cannabis use: a meta-analysis. Drug Alcohol Depend 2013 Dec 1;133(2):295-304. [CrossRef] [Medline]
    7. European Monitoring Centre for Drugs and Drug Addiction. Treatment of cannabis-related disorders in Europe. Luxembourg: Publications Office of the European Union; 2015.
    8. Rogers MA, Lemmen K, Kramer R, Mann J, Chopra V. Internet-delivered health interventions that work: systematic review of meta-analyses and evaluation of website availability. J Med Internet Res 2017 Mar 24;19(3):e90 [FREE Full text] [CrossRef] [Medline]
    9. Die Drogenbeauftragte der Bundesregierung. 2017. Drogen- und Suchtbericht [Drug and Addiction Report]   URL: http:/​/www.​drogenbeauftragte.de/​fileadmin/​dateien-dba/​Drogenbeauftragte/​4_Presse/​1_Pressemitteilungen/​2017/​2017_III_Quartal/​Drogen-_und_Suchtbericht_2017_V2.​pdf [WebCite Cache]
    10. Sundström C, Blankers M, Khadjesari Z. Computer-based interventions for problematic alcohol use: a review of systematic reviews. Int J Behav Med 2017 Oct;24(5):646-658 [FREE Full text] [CrossRef] [Medline]
    11. Hoermann S, McCabe KL, Milne DN, Calvo RA. Application of synchronous text-based dialogue systems in mental health interventions: systematic review. J Med Internet Res 2017 Jul 21;19(8):e267 [FREE Full text] [CrossRef] [Medline]
    12. Dowling M, Rickwood D. Online counseling and therapy for mental health problems: a systematic review of individual synchronous interventions using chat. J Technol Hum Serv 2013 Jan;31(1):1-21. [CrossRef]
    13. Barak A, Hen L, Boniel-Nissim M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Technol Hum Serv 2008 Jul 03;26(2-4):109-160. [CrossRef]
    14. Litten RZ, Allen JP, editors. Timeline follow-back: A technique for assessing self-reported alcohol consumption. In: Measuring alcohol consumption: Psychosocial and biochemical methods. Totowa, NJ, US: Humana Press; 1992.
    15. Gossop M, Darke S, Griffiths P, Hando J, Powis B, Hall W, et al. The Severity of Dependence Scale (SDS): psychometric properties of the SDS in English and Australian samples of heroin, cocaine and amphetamine users. Addiction 1995 May;90(5):607-614. [Medline]
    16. Steiner S, Baumeister SE, Kraus L. Severity of Dependence Scale: establishing a cut-off point for cannabis dependence in the German adult population. SUCHT 2008;54:57-63. [CrossRef]
    17. Martin G, Copeland J, Gates P, Gilmour S. The Severity of Dependence Scale (SDS) in an adolescent population of cannabis users: reliability, validity and diagnostic cut-off. Drug Alcohol Depend 2006 Jun 9;83(1):90-93. [CrossRef] [Medline]
    18. Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann 1979 Jan;2(3):197-207. [CrossRef]
    19. Schmidt J, Nübling R, Lamprecht F, Wittmann W. Patientenzufriedenheit am Ende psychosomatischer Reha-Behandlungen. Zusammenhänge mit Behandlungs- und Ergebnisvariablen und prognostische Bedeutung. Salutogenese. Ein neues Konzept in der Psychosomatik? Kongreßband der 40. Jahrestagung des Deutschen Kollegiums für Psychosomatische Medizin e-book. Frankfurt a. M. Frankfurt a. M: VAS Verlag für Akademische Schriften; 1994:271-283.
    20. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. J Couns Psychol 1989;36(2):223-233. [CrossRef]
    21. Wilmers F, Munder T, Linster H. Die deutschsprachige Version des Working Alliance Inventory - short revised (WAI-sr) - Ein schulenübergreifendes, ökonomisches und empirisch validiertes Instrument zur Erfassung der therapeutischen Allianz. Klinische Diagnostik Und Evaluation serial online 2008;1(3):343-358. [CrossRef]
    22. Ewing JA. Detecting alcoholism: The CAGE questionnaire. J Am Med Assoc 1984;252(14):1905-1907. [CrossRef]
    23. Beck AT, Rush AJ, Shaw BF, Emery G. Kognitive Therapie der Depression (eBook). Munich: Urban & Schwarzenberg; 1981.
    24. Miller WR, Heather N, editors. Implications of a self-regulation model of therapy for treatment of addictive behaviors. In: Treating Addictive Behaviors: Processes of Change. New York: Plenum Press; 1986.
    25. De Shazer S, Dolan Y, Korman H, McCollum E, Trepper T, Berg I. More Than Miracles: The State Of The Art Of Solution-Focused Brief Therapy. New York, NY, US: Haworth Press; 2007.
    26. Miller WR, Rollnick S. Motivational Interviewing: Preparing People To Change Addictive Behavior. New York: The Guilford Press; 1991.
    27. Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd Edition). Abingdon, United Kingdom: Routledge; 1988.
    28. R-project. The R Project for Statistical Computing   URL: https://www.r-project.org/ [accessed 2018-04-06] [WebCite Cache]
    29. Højsgaard S, Højsgaard S, Yan J. The R package geepack for generalized estimating equations. J Stat Softw 2006;15(2):-. [CrossRef]
    30. AC Del Re. 2013. compute.es: Compute Effect Sizes   URL: http://cran.r-project.org/web/packages/compute.es/ [WebCite Cache]
    31. van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. J Stat Softw 2011;45(3):1-67 [FREE Full text] [CrossRef]
    32. Statistisches Bundesamt. Absolventen/Abgänger nach Abschlussart [Graduates/graduates after graduation]   URL: https:/​/www.​destatis.de/​DE/​ZahlenFakten/​GesellschaftStaat/​BildungForschungKultur/​Schulen/​Tabellen/​AbsolventenAbgaenger_Abschlussart.​html [WebCite Cache]
    33. Rooke S, Thorsteinsson E, Karpin A, Copeland J, Allsop D. Computer-delivered interventions for alcohol and tobacco use: a meta-analysis. Addiction 2010 Aug;105(8):1381-1390. [CrossRef] [Medline]
    34. Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, et al. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One 2014 Jun 17;9(6):e99912 [FREE Full text] [CrossRef] [Medline]
    35. Schaub MP, Wenger A, Berg O, Beck T, Stark L, Buehler E, et al. A web-based self-help intervention with and without chat counseling to reduce cannabis use in problematic cannabis users: three-arm randomized controlled trial. J Med Internet Res 2015;17(10):e232 [FREE Full text] [CrossRef] [Medline]
    36. Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    37. Copeland J, Rooke S, Rodriquez D, Norberg MM, Gibson L. Comparison of brief versus extended personalised feedback in an online intervention for cannabis users: short-term findings of a randomised trial. J Subst Abuse Treat 2017;76:43-48. [CrossRef] [Medline]
    38. Cook JE, Doyle C. Working alliance in online therapy as compared to face-to-face therapy: preliminary results. Cyberpsychol Behav 2002;5(2):95-105. [CrossRef]
    39. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res 2012 Dec;12(6):745-764. [CrossRef] [Medline]
    40. Rooke S, Copeland J, Norberg M, Hine D, McCambridge J. Effectiveness of a self-guided web-based cannabis treatment program: randomized controlled trial. J Med Internet Res 2013 Feb 15;15(2):e26 [FREE Full text] [CrossRef] [Medline]
    41. Arnaud N, Baldus C, Elgán TH, De Paepe N, Tønnesen H, Csémy L, et al. Effectiveness of a web-based screening and fully automated brief motivational intervention for adolescent substance use: a randomized controlled trial. J Med Internet Res 2016;18(5):e103 [FREE Full text] [CrossRef] [Medline]

    ‎
    BZgA: Bundeszentrale für gesundheitliche Aufklärung, Federal Centre for Health Education
    CUD: cannabis use disorder
    CSQ-8: Client Satisfaction Questionnaire
    RCT: randomized controlled trial
    QTS: Quit the Shit
    SDS: Severity of Dependence Scale
    TLFB: Timeline Followback
    WAI-sr: Working Alliance Inventory-short revised

    Edited by G Eysenbach; submitted 06.12.17; peer-reviewed by R Tait, M Schaub; comments to author 28.12.17; revised version received 29.01.18; accepted 23.02.18; published 08.05.18

    Copyright

    ©Benjamin Jonas, Marc-Dennan Tensil, Peter Tossmann, Evelin Strüber. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.05.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.