This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297) Digital Health Reviews (1289) mHealth for Wellness, Behavior Change and Prevention (2871) mHealth in a Clinical Setting (615) Alcohol Abuse, Alcoholism Prevention and Recovery (300) Behavior Change (712)

Published on in Vol 21, No 4 (2019): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10859, first published .
Technology-Based Alcohol Interventions in Primary Care: Systematic Review

Technology-Based Alcohol Interventions in Primary Care: Systematic Review

Technology-Based Alcohol Interventions in Primary Care: Systematic Review

Authors of this article:

Alex T Ramsey1 Author Orcid Image ;   Jason M Satterfield2 Author Orcid Image ;   Donald R Gerke3 Author Orcid Image ;   Enola K Proctor4 Author Orcid Image
Article Authors Cited by (26) Tweetations (20) Metrics

    Review

    1Department of Psychiatry, Washington University School of Medicine, St Louis, MO, United States

    2Department of Medicine, University of California San Francisco, San Francisco, CA, United States

    3Graduate School of Social Work, University of Denver, Denver, CO, United States

    4Brown School of Social Work, Washington University in St Louis, St Louis, MO, United States

    Corresponding Author:

    Alex T Ramsey, PhD

    Department of Psychiatry

    Washington University School of Medicine

    660 South Euclid, Campus Box 8134

    St Louis, MO, 63110

    United States

    Phone: 1 3143625370

    Email: aramsey@wustl.edu


    Background: Primary care settings are uniquely positioned to reach individuals at risk of alcohol use disorder through technology-delivered behavioral health interventions. Despite emerging effectiveness data, few efforts have been made to summarize the collective findings from these delivery approaches.

    Objective: The aim of this study was to review recent literature on the use of technology to deliver, enhance, or support the implementation of alcohol-related interventions in primary care. We focused on addressing questions related to (1) categorization or target of the intervention, (2) descriptive characteristics and context of delivery, (3) reported efficacy, and (4) factors influencing efficacy.

    Methods: We conducted a comprehensive search and systematic review of completed studies at the intersection of primary care, technology, and alcohol-related problems published from January 2000 to December 2018 within EBSCO databases, ProQuest Dissertations, and Cochrane Reviews. Of 2307 initial records, 42 were included and coded independently by 2 investigators.

    Results: Compared with the years of 2000 to 2009, published studies on technology-based alcohol interventions in primary care nearly tripled during the years of 2010 to 2018. Of the 42 included studies, 28 (64%) were randomized controlled trials. Furthermore, studies were rated on risk of bias and found to be predominantly low risk (n=18), followed by moderate risk (n=16), and high risk (n=8). Of the 24 studies with primary or secondary efficacy outcomes related to drinking and drinking-related harms, 17 (71%) reported reduced drinking or harm in all primary and secondary efficacy outcomes. Furthermore, of the 31 studies with direct comparisons with treatment as usual (TAU), 13 (42%) reported that at least half of the primary and secondary efficacy outcomes of the technology-based interventions were superior to TAU. High efficacy was associated with provider involvement and the reported use of an implementation strategy to deliver the technology-based intervention.

    Conclusions: Our systematic review has highlighted a pattern of growth in the number of studies evaluating technology-based alcohol interventions in primary care. Although these interventions appear to be largely beneficial in primary care, outcomes may be enhanced by provider involvement and implementation strategy use. This review enables better understanding of the typologies and efficacy of these interventions and informs recommendations for those developing and implementing technology-based alcohol interventions in primary care settings.

    J Med Internet Res 2019;21(4):e10859

    doi:10.2196/10859

    Keywords

    alcohol drinking; risky health behavior; alcohol-related disorders; internet; computers; mobile health; primary health care; implementation science; review 


    Background

    Alcohol use is a leading risk factor for global disease burden, and recent findings indicate that even light-to-moderate drinking is detrimental to all-cause mortality [1,2]. Additionally, approximately 14% of US adults annually, and nearly 30% for lifetime, engage in harmful drinking consistent with alcohol use disorder [3]. This disorder contributes to over 200 diseases and health problems, including cirrhosis, cancers, fetal alcohol syndrome, assaults, and crash-related fatalities, with costs totaling US $249 billion annually in the United States alone [4-6]. Primary care settings that integrate physical and behavioral health care are uniquely situated to reach this at-risk population through delivery of evidence-based interventions (EBIs) to reduce harmful alcohol use [7,8].

    To enhance the capacity for delivering behavioral health services, new approaches—such as those provided by digital technologies—can assess and intervene to reduce alcohol use (and associated harm) and facilitate referrals to specialty treatment [9]. In fact, health centers are increasingly leveraging technology to reduce medical staff burden, facilitate electronic health record (EHR) integration, improve standardization and fidelity, and enhance service efficiency [10,11]. Additionally, recent randomized controlled trials have begun to highlight the promise of using technology-supported platforms—including computers, kiosks, or tablets—to deliver efficacious alcohol-related interventions in primary care [7,11,12]. Relatedly, telephone-delivered interventions, although less novel, remain popular approaches in primary care, despite a lack of synthesized research on effectiveness and optimal implementation.

    Systematic reviews exist for generalized alcohol interventions in primary care [13,14] and technology-based behavioral health interventions in other settings [15-17]. However, despite emerging effectiveness data in both adult and adolescent populations [17], few efforts have been made to summarize the collective findings of technology-based alcohol interventions in primary care settings. In total, 2 excellent Cochrane reviews on the effectiveness of brief alcohol interventions were recently published; one was based in primary care but did not focus on technology-based interventions [18] and the other focused on technology-based interventions but the focus was largely outside of primary care settings [19]. Another systematic review examined digital and computer-based alcohol interventions in primary care [20]; however, this review includes a broader set of technology-based interventions, addresses important effect modifiers, and provides a substantial update, adding 4 years (2015 to 2018) beyond the previous review. Use of technology in primary care is rapidly evolving, and systematically updating the collective knowledge gained from recent efforts in this area is needed to inform future delivery of technology-based alcohol interventions in these settings.

    Furthermore, although technology-based behavioral health interventions are likely to be diverse in nature, there is currently a lack of conceptual clarity and no system for categorizing these interventions (eg, patient-facing, provider-facilitated, or a combination of both), making it difficult to directly compare interventions with similar purposes and approaches. The conceptualization and categorization provided here is useful for intervention developers and researchers in this area of investigation. The overarching purpose of this study was to review the most recent literature on the use of technology to deliver, enhance, or support the implementation of alcohol-related interventions in primary care.

    Research Questions

    In this review, we focused on addressing the following key questions regarding technology-based alcohol interventions in primary care:

    1. What proportion of technology-based alcohol interventions is delivered (1) directly to the patient, (2) by the provider via technology-based medium, (3) some combination of both, or (4) directly to the provider to improve care?
    2. What factors supported the use of technology-based alcohol interventions (technological platforms, delivery contexts, implementation strategies, and EHR integration)?
    3. What proportion of studies reported that the technology-based intervention (1) reduced drinking or alcohol-related harms and (2) demonstrated superiority to treatment as usual (TAU)?
    4. Did efficacy differ by the (1) context in which the intervention was delivered, (2) type of technology, (3) categorization or target of the technology-based intervention, and (4) use of an implementation strategy?

    Answers to these questions would provide researchers with a better system of categorizing a diverse set of technology-based interventions and an understanding of the efficacy and effect modifiers of these interventions. These important contributions would inform recommendations for those developing and implementing technology-based alcohol interventions in primary care settings.


    Search Strategy

    In line with the Preferred Reporting Items for Systematic reviews and Meta-Analyses Statement and supporting publications to enhance the rigor of systematic reviews [21,22], we conducted a systematic literature review of English-language publications on completed research studies from January 2000 to December 2018. In total, 4 EBSCO databases (CINAHL Plus, Global Health, MEDLINE, and PsycInfo), ProQuest Dissertations, and Cochrane Reviews were searched. Although a comprehensive search of gray literature was not feasible, we did include dissertations and search trial registries (eg, PROSPERO). A total of 77 Boolean search terms were used to identify articles at the intersection of primary care, technology, and alcohol-related problems (see Multimedia Appendix 1). In conjunction with 2 university research librarians, extensive testing was conducted to limit the number of articles outside the inclusion criteria and yet ensure that the search strategy yielded comprehensive results. The systematic review protocol was published and is accessible on PROSPERO, an international prospective register of systematic reviews.

    ‎
    Figure 1. Flow diagram of search and screening results.
    View this figure

    Study Selection

    As illustrated in Figure 1, the initial database search yielded 2307 records, which reduced to 1636 records after removing duplicates. One of the 2 study investigators trained in conducting systematic reviews (AR or DG) reviewed the title and abstract of each record to assess study eligibility, and full text was obtained when appropriate. Studies were excluded if the article title or abstract did not specify that the study included a technology-based intervention that focused on alcohol use and was delivered in a primary care setting. A total of 138 studies were identified for full-text review and final selection for data abstraction.

    Both study investigators (AR and DG) then independently reviewed each full text article for final inclusion, achieving an interrater reliability of .84. All initial discrepancies were resolved mutually through discussion between the same 2 investigators. Studies were excluded during this step if they were a self-defined pilot study or only reported on feasibility outcomes, alcohol use outcomes were not reported separately from other health outcomes, the so-called intervention only constituted screening for alcohol misuse, or the study results had previously been reported elsewhere (eg, main study outcome article). On the basis of this full-text review, 42 studies met our final inclusion criteria for coding [12,23-63].

    Data Abstraction and Analysis

    All data from included articles were recorded using a standardized data abstraction form, which both study investigators (AR and DG) completed independently, again mutually resolving all initial discrepancies through discussion. To address our key questions, articles were coded on a range of topics, including study design and sample size, type of technology, category or target of intervention, location of delivery, implementation strategies used, primary and secondary outcomes, intervention efficacy results, and a risk of bias score to inform study quality (see Multimedia Appendix 2). The wide heterogeneity of primary outcomes and assessment tools precluded the ability to conduct a meta-analysis; instead, data from the articles were primarily summarized descriptively. However, we conducted one-way analyses of variance (ANOVAs) with an efficacy score treated as the dependent variable and several of the factors listed above—location of delivery (ie, delivery context), type of technological platform used, category of intervention, and presence of an implementation strategy—treated as the independent variables.


    Descriptive Analyses

    Of the 42 included studies, 5 (12%) studies were published from 2000 to 2004, 7 (17%) were published from 2005 to 2009, 21 were (50%) published from 2010 to 2014, and 9 (21%) were published from 2015 to 2018. The included studies were conducted in 8 different countries, with 33 of 42 studies (79%) based in the United States. Of the 42 included studies, 28 (64%) featured randomized designs, 10 (24%) were quasi-experimental, and 4 (10%) were observational studies. Multimedia Appendix 2 presents further information on designs for each study.

    Type of Technology

    Technology-based interventions were also categorized by the technological platform used to deliver the intervention. Of the 47 technological platforms identified, 13 (28%) were telephone or telehealth, 13 (28%) were stand-alone computer or software, 10 (21%) were Web-based, 5 (11%) were mobile (eg, tablet and smartphone), 3 (6%) were interactive voice response, 2 (4%) were kiosk, and 1 (2%) was video.

    Category or Target of Intervention

    Using a typology informed by research on behavior change at multiple levels [64], technology-based interventions were categorized into 4 main types: (1) Patient-facing; (2) Provider-facilitated; (3) Patient-facing plus provider-facilitated; and (4) Provider-directed. Table 1 summarizes these types of technology-based interventions, including the conceptual definitions and representative examples of each type. The coding team determined that, of the 42 included studies, 14 (33%) interventions were patient-facing, 11 (26%) were provider-facilitated, 11 (26%) were patient-facing plus provider-facilitated, and 6 (14%) were provider-directed.

    Delivery Context

    Articles were coded into 3 broad delivery contexts: in clinic (eg, waiting room and exam room), out of clinic (eg, home and work), or both in and out of clinic (eg, part in waiting room and part at home). Of the 42 included articles, 18 (43%) reported interventions delivered in clinic, 17 (40%) reported interventions delivered at home (or otherwise outside of the clinic setting), and 7 (17%) reported interventions delivered both in clinic and at home.

    Implementation Strategies

    Implementation strategies, conceptualized as “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice” [65], constitute an important component to delivery of evidence-based practices [66,67], including those facilitated by the use of technology [68,69]. Previous research has categorized implementation strategies into the following categories: planning, educating, financing, restructuring, managing quality, and attending to the policy context [70]. Although underreported and underspecified in clinical research [65], implementation strategies are necessary to maximize the translation of research-based interventions into practice settings [71,72]. For this review, we only considered approaches to be implementation strategies when health care professionals (rather than research staff) were involved in efforts to improve uptake or delivery of the intervention. Of the 42 studies included, 15 (36%) specified the use of an implementation strategy to support the delivery of technology-based alcohol interventions (see Multimedia Appendix 2).

    We documented 17 total and 11 unique implementation strategies using the Expert Recommendations for Implementing Change compilation [71]. The reported implementation strategies included the following: Conduct ongoing training (n=3), Make training dynamic (n=3), Provide ongoing consultation (n=2), Remind clinicians (n=2), Relay clinical data to providers (n=2), Facilitation (n=2), Develop educational materials (n=1), Organize clinician implementation team meetings (n=1), Preparepatients/consumers to be active participants (n=1), Develop a formal implementation blueprint (n=1), Assess for readiness (n=1), and Conduct cyclical small tests of change (n=1). These strategies varied widely in type and intensity, ranging from in-depth training on use of the technology-based intervention to the use of reminder cards and posters for providers. Implementation strategies also included providing frequent supervision, comparison between health care staff versus self (patient)-referral to the technology-based intervention and linking clinical management to the technology-based platform. Of note, only 6 of 42 articles (14%) mentioned any type of integration between the technology-based intervention and existing EHR systems.

    Risk of Bias

    We used a common classification scheme [73] to rate risk of bias pertaining to selection (eg, allocation concealment), performance (eg, blinding), detection (eg, validity of outcome assessment), attrition (eg, withdrawal rates), and reporting (eg, selective outcome reporting). We judged each of the 5 types of bias to be high (2), unclear (1), or low (0), and then calculated a sum risk of bias score on a scale of 0 to 10. Studies were then determined to have low (0 to 1), moderate (2 to 4), or high (5 to 10) overall risk of bias. Of the 42 studies, 18 (43%) were rated to be low risk of bias, 16 (38%) moderate risk of bias, and 8 (19%) high risk of bias.

    Primary and Secondary Outcomes

    We classified primary and secondary efficacy outcomes into 7 categories: quantity of alcohol use (eg, number of drinks per drinking day), frequency of alcohol use (eg, total number of drinking days), severity of alcohol use or risk scores (eg, Alcohol Use Disorders Identification Test or Alcohol, Smoking and Substance Involvement Screening Test score), binge or heavy episodic drinking (eg, number of binge episodes in the past week), status of at-risk alcohol use (eg, proportion of individuals with categorically defined at-risk drinking), any use (proportion of individuals with any past 90-day alcohol use), and drinking consequences (academic or legal problems related to drinking). Multimedia Appendix 3 organizes results across these categories of outcomes for each of the 4 technology types.

    Table 1. Types of technology-based alcohol interventions.
    TypeConceptualizationExamples
    Patient-facingIntervention is delivered directly to the patient via technology with very limited or no provider involvementStand-alone touchscreen kiosk-based brief intervention
    Provider-facilitatedIntervention is delivered by provider to patient via technology-based mediumTelephone-based brief intervention
    Patient-facing plus provider-facilitatedPackaged intervention that has at least one patient-facing component and at least one provider-facilitated componentTablet-based screening and brief intervention plus telephone-based counseling
    Provider-directedIntervention is delivered to provider to improve or support patient care deliveryWeb-based training and clinical management dashboard

    Efficacy of Intervention

    Multimedia Appendix 3 summarizes study results according to statistical significance on 2 key outcomes—whether the intervention reduced drinking or drinking-related harms and whether the benefits were superior in comparison with TAU—for each primary and secondary outcome of each study. Of note, several studies reported multiple primary and secondary outcomes. At the study level, we assessed the proportion of primary and secondary efficacy outcomes that indicated reduced harm and that were determined to be superior to TAU. Similar methods of operationalizing and summarizing intervention efficacy have been used in previous systematic reviews in lieu of meta-analytic procedures [74].

    Of the 24 studies with primary or secondary efficacy outcomes related to drinking and drinking-related harms, all 24 (100%) indicated reduced drinking or harm in at least half of the primary and secondary outcomes and 17 (71%) indicated reduced drinking or harm in all of the primary and secondary outcomes. Of the 31 studies with direct comparisons with TAU, 16 (52%) indicated that none of the primary and secondary outcomes were superior to TAU. However, of these 31 studies, 13 (42%) indicated that at least half of the primary and secondary outcomes were superior to TAU, and 8 (26%) indicated that all the primary and secondary outcomes were superior to TAU.

    In examining predictors of intervention efficacy, we used the outcome of whether at least half of the primary and secondary efficacy outcomes were determined to be superior to TAU to maximize variability in the outcome, use a sufficiently rigorous cutoff, and focus on studies with direct comparisons with TAU. Of the 31 studies comparing intervention to TAU, 6 of 16 (38%) low risk of bias studies, 5 of 13 (38%) moderate risk of bias studies, and 2 of 2 high risk of bias studies reported at least half of their primary and secondary efficacy outcomes to be superior to TAU. This outcome was uncorrelated with the risk of bias (P=.812) variable.

    Predictors of Efficacy

    Descriptive analyses and one-way ANOVAs were used to examine intervention efficacy based on (1) whether the intervention was delivered in the clinic, at home, or both in the clinic and at home, (2) the type of technology used (computer or Web, telephone or video, and mobile), (3) the category or target of intervention (eg, patient-facing), and (4) the specification of an implementation strategy.

    1. Delivery context. Of the 31 studies comparing intervention to TAU, 3 of 14 (21%) studies in the clinic, 5 of 10 (50%) studies at home, and 5 of 7 (71%) studies both in the clinic and at home showed at least half of the outcomes superior to TAU. The level of intervention efficacy did not differ significantly based on whether the intervention was conducted in the clinic (mean 0.21 (SD 0.43)), at home (mean 0.50 (SD 0.53)), or both in the clinic and at home (mean 0.71 (SD 0.49); F2,28=2.81; P=.077). However, it should be noted here that there were only 7 cases in the category of both in the clinic and at home.
    2. Type of technology. Of the 34 technological platforms within the 31 studies comparing intervention to TAU, 6 of 17 (35%) studies of computer or Web-based interventions, 7 of 13 (54%) telephone or video-based interventions, and 2 of 4 (50%) mobile-based interventions showed at least half of the outcomes superior to TAU. The level of intervention efficacy did not differ significantly based on whether the technology-based alcohol intervention was delivered via computer or Web (mean 0.35 (SD 0.49)), telephone or video (mean 0.54 (SD 0.52)), or mobile (mean 0.50 (SD 0.58); F2,31=0.51; P=.603).
    3. Category of intervention. The first analysis examined the level of intervention efficacy for each separate category of intervention; the second analysis examined the level of intervention efficacy between interventions that were only patient-facing versus those that had a provider-based component. Of the 31 studies comparing intervention with TAU, 2 of 12 (17%) patient-facing, 6 of 9 (67%) provider-facilitated, 5 of 9 (56%) patient-facing plus provider-facilitated, and 0 of 1 provider-directed studies showed at least half of their intervention outcomes superior to TAU. There were no significant differences in intervention efficacy between the individual categories of patient-facing (mean 0.17 (SD 0.39)), provider-facilitated (mean 0.67 (SD 0.50)), patient-facing plus provider-facilitated (mean 0.56 (SD 0.53)), and provider-directed, yet the results trended toward significance such that the patient-facing interventions showed lower efficacy than the other groups; (F3,27= 2.54; P=.078). Indeed, when collapsing the groups that included a provider component and comparing them with patient-only interventions, we found that intervention efficacy was significantly higher for interventions that had a provider-based component (mean 0.58 (SD 0.51)) than for those that were patient-facing only (mean 0.17 (SD 0.39); F1,29=5.76; P=.023, adjusted R2=0.14 [medium effect size]).
    4. Specification of implementation strategy. As implementation strategies were conceptualized as methods to improve the delivery of a clinical intervention, the provider-directed technologies were considered synonymous and indistinguishable from implementation strategies. Implementation strategies were analyzed as effect modifiers for the other types of interventions (eg, patient-facing); therefore, it was determined inappropriate to include the provider-directed technologies in this analysis as the intervention would have been the same as the effect modifier. Therefore, of the 31 studies comparing intervention with TAU, the provider-directed study (n=1) was removed for this particular analysis. In the remaining 30 studies, 6 of 6 (100%) studies reporting use of an implementation strategy showed at least half of the outcomes superior to TAU versus 7 of 24 (29%) studies not reporting use of an implementation strategy. Intervention efficacy was significantly higher when an implementation strategy was employed to facilitate delivery of the intervention (mean 1.00 (SD 0.00)) than for those with no specified implementation strategy (mean 0.29 (SD 0.46); F1,28=13.60; P=.001; adjusted R2=0.30 [large effect size]).

    Principal Findings

    The use of technology-facilitated interventions in primary care settings is a burgeoning issue in behavioral health; however, research-based guidance is needed to inform development and implementation to ensure that these tools enhance, rather than impede, the efficiency and effectiveness of alcohol interventions in this setting. Our review attends to a number of key factors that may influence effectiveness of these interventions. Specifically, our review suggests a benefit to involving a provider in the delivery process, as compared with technology-based alcohol interventions that only engage the patient. This aligns with much research on technology-based interventions for use of tobacco and other substances [75,76]. Extensive research has shown that the specification and use of implementation strategies improve outcomes such as the adoption, reach, and sustainability of interventions [77-80]. However, to the extent that studies reported the use of implementation strategies when they were used, our review is among the first to find that employing an implementation strategy may actually enhance the effectiveness of a behavioral health intervention.

    Our systematic review also highlights a pattern of growth in the number of studies evaluating technology-based alcohol interventions in primary care, with nearly 3 times as many studies on technology-based alcohol interventions in primary care published during the years of 2010 to 2018, as compared with the years of 2000 to 2009. The increasing number of studies on these interventions reflects an important angle of the changing health care landscape. As innovative technology-based approaches to delivering alcohol interventions continue to rapidly develop, it is necessary to take stock of the existing efforts and identify areas for further growth and improvement. Results indicated robust potential of technology-based interventions to support alcohol-related behavior change, with the majority indicating reduced drinking or harm in all of the reported primary and secondary outcomes. Similarly, when compared directly with TAU, there appeared to be strong efficacy for technology-based alcohol interventions over and above nontechnology-based alcohol intervention, with 42% (13/31) of studies with direct comparisons with TAU reporting that at least half of the primary and secondary efficacy outcomes of the technology-based interventions were superior to TAU.

    Our review also highlights factors that appear to influence intervention efficacy. For instance, we identified 4 broad categories of technology-based alcohol interventions, and results indicated that interventions with a provider-based component (particularly provider-facilitated and patient-facing plus provider-facilitated interventions) were more efficacious than those that were patient-facing only. These findings suggest that provider involvement in the delivery of technology-based alcohol interventions may boost efficacy; however, further research is needed in this area.

    Another key finding was that studies describing use of an implementation strategy reported more effective technology-based alcohol interventions. Although this effect has been documented in limited previous research [81], this remains a relatively novel finding that contributes to the accumulating evidence of the value-added benefit of employing implementation strategies to facilitate EBIs. Nevertheless, this finding is consistent with theoretical advances in the implementation science field that reject the assumption of voltage drop as an intervention moves from efficacy trials to real-world implementation studies [82]. Instead, it is reasonable to expect that with active strategies to adapt and tailor interventions to contexts and patients, the voltage of a technology-based intervention may even be enhanced. We encourage further empirical study of this potential effect in future research.

    Yet, the determinants of efficacy remain largely unexplained, even after accounting for the presence of a provider-based component and a specified implementation strategy. Intermediate outcomes, or implementation outcomes (eg, acceptability, feasibility, and sustainability) [83,84], which are often not reported on in clinical research may help to further explain intervention efficacy. For instance, the degree to which patients and providers find particular technology-based alcohol interventions to be acceptable, feasible, and sustainable to use may influence the effectiveness of those interventions.

    Future research should also strive to report more frequently on implementation strategies that occur during intervention studies [65]. Although outside the scope of this study, future reviews may benefit from examining whether or not the effectiveness of technology-based interventions can be predicted by the type of implementation strategy used. For instance, it is conceivable that variation in training and ongoing technical assistance, mandates from leadership, or efforts to engage patients and increase consumer demand for technology-based alcohol interventions could lead to greater effectiveness of these interventions. Relatedly, implementation studies should continue to examine the systematic use of strategies to support or improve the delivery of technology-based alcohol interventions.

    Finally, this review highlights a potential lack of current integration between technology-based alcohol interventions and existing EHR systems; this technological integration has been posited to be a critical limiting factor in realizing the public health impact promised by technology-based behavioral health interventions [85]. Greater efforts to integrate technology-based alcohol interventions with existing EHR systems in primary care will be necessary to ensure the scale up and sustainability of technology-based alcohol interventions.

    Limitations

    We acknowledge that our systematic review may be limited to some degree by publication bias. It is important to recognize that there is a general bias, in both authors and publishers, toward prioritizing the publication of positive findings (ie, evidence in support of tested interventions) over null or negative findings. This can lead to misleadingly favorable conclusions about the efficacy of interventions. Although unable to fully address this limitation, we included dissertations in our systematic review to help mitigate this concern.

    It is also worth noting that, given the nascent stage of the implementation science field, authors are often inconsistent in their reporting of implementation strategies used to deliver their interventions [65]. Some authors may not have reported or specified implementation strategies that were actually used, increasing the risk of reporting bias. Additionally, different authors may have specified the same implementation strategy in different ways. It is possible that these inconsistencies influenced our findings regarding the relationship between implementation strategies and efficacy.

    As is generally the case in systematic reviews, our study is subject to the limitations of potential errors or biases in searching, including, and coding articles. For instance, it is possible that our search strategy failed to identify all relevant studies (eg, non-English language articles), that we were too conservative in our inclusion of relevant studies, or that we erroneously coded content in some articles. To offset these concerns, we (1) solicited the expertise of 2 university librarians to conduct extensive database and search string testing to ensure that our search strategy was appropriately comprehensive and (2) used 2 study investigators during the inclusion and coding processes to limit reviewer fatigue and ensure interrater agreement. Finally, we limited our review to primary care and general practice settings, and although previous reviews have addressed technology-based behavioral health interventions in the emergency department and social work settings [15,16,86], future research focused on other settings, including specialty care or dental health, may further contribute to this area of investigation.

    Implications

    This systematic review contributes substantially to the conceptualization of technology-based alcohol interventions, an understanding of the range of implementation contexts and formats in which these interventions are delivered, and an initial assessment of the efficacy and effect modifiers for technology-based alcohol interventions. The use of technology-based tools in primary care settings represents a promising approach to enhance the efficiency, service delivery flexibility, and effectiveness of interventions for alcohol-related problems. Our systematic review identifies that the past 2 decades have borne witness to the delivery of technology-based alcohol interventions through a variety of different technological platforms (eg, computer, Web, or mobile), settings (eg, clinic, home, or both), and targets (eg, patients, providers, or both). These findings provide initial support for the efficacy of technology-based alcohol interventions, particularly when deployed through a specific implementation strategy and involving a provider in the delivery process, and we encourage future research to further establish the efficacy, moderators of efficacy, and implementation strategies for delivering these types of interventions in primary care settings and beyond.

    Acknowledgments

    The authors would like to acknowledge Lori Siegel, Reference Librarian at the Washington University Brown School of Social Work and Laura Simon, Clinical Librarian at the Bernard Becker Medical Library, for their substantial assistance with this literature review. Research reported in this publication was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number K12DA041449, NIDA F31DA039776, NIMH T32MH019960, a grant from the Foundation for Barnes-Jewish Hospital, and the Washington University Institute for Public Health, Center for Dissemination and Implementation Pilot Program. Funding for this project was also provided by the Institute of Clinical and Translational Sciences through the Clinical and Translational Science Award Program of the National Center for Advancing Translational Sciences at the National Institutes of Health, Grant Number UL1 TR000448.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Search string of 77 Boolean terms searched by Title, Abstract, and Subject.

    PDF File (Adobe PDF File), 68KB
    ‎

    Multimedia Appendix 2

    Studies of technology-based alcohol interventions in primary care (n=42).

    PDF File (Adobe PDF File), 221KB
    ‎

    Multimedia Appendix 3

    Summary of results on alcohol-related outcomes.

    PDF File (Adobe PDF File), 148KB
    1. GBD 2016 Alcohol Collaborators. Alcohol use and burden for 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 2018 Dec 22;392(10152):1015-1035. [CrossRef] [Medline]
    2. Hartz SM, Oehlert M, Horton AC, Grucza RA, Fisher SL, Culverhouse RC, et al. Daily drinking is associated with increased mortality. Alcohol Clin Exp Res 2018 Nov;42(11):2246-2255. [CrossRef] [Medline]
    3. Grant BF, Goldstein RB, Saha TD, Chou SP, Jung J, Zhang H, et al. Epidemiology of dsm-5 alcohol use disorder: results from the National Epidemiologic Survey on Alcohol and Related Conditions III. JAMA Psychiatry 2015 Aug;72(8):757-766 [FREE Full text] [CrossRef] [Medline]
    4. World Health Organization. 2014. Global status report on alcohol and health   URL: https://www.who.int/substance_abuse/publications/global_alcohol_report/msb_gsr_2014_1.pdf?ua=1 [accessed 2019-03-04] [WebCite Cache]
    5. May PA, Gossage JP, Kalberg WO, Robinson LK, Buckley D, Manning M, et al. Prevalence and epidemiologic characteristics of FASD from various research methods with an emphasis on recent in-school studies. Dev Disabil Res Rev 2009;15(3):176-192. [CrossRef] [Medline]
    6. Sacks JJ, Gonzales KR, Bouchery EE, Tomedi LE, Brewer RD. 2010 national and state costs of excessive alcohol consumption. Am J Prev Med 2015 Nov;49(5):e73-e79. [CrossRef] [Medline]
    7. Kaner E, Bland M, Cassidy P, Coulton S, Dale V, Deluca P, et al. Effectiveness of screening and brief alcohol intervention in primary care (SIPS trial): pragmatic cluster randomised controlled trial. Br Med J 2013;346:e8501 [FREE Full text] [CrossRef] [Medline]
    8. Pilowsky DJ, Wu L. Screening for alcohol and drug use disorders among adults in primary care: a review. Subst Abuse Rehabil 2012 Apr;3(1):25-34 [FREE Full text] [CrossRef] [Medline]
    9. Glass JE, McKay JR, Gustafson DH, Kornfield R, Rathouz PJ, McTavish FM, et al. Treatment seeking as a mechanism of change in a randomized controlled trial of a mobile health intervention to support recovery from alcohol use disorders. J Subst Abuse Treat 2017 Jun;77:57-66. [CrossRef] [Medline]
    10. Wallace P, Bendtsen P. Internet applications for screening and brief interventions for alcohol in primary care settings - implementation and sustainability. Front Psychiatry 2014;5:151 [FREE Full text] [CrossRef] [Medline]
    11. Quanbeck AR, Gustafson DH, Marsch LA, McTavish F, Brown RT, Mares M, et al. Integrating addiction treatment into primary care using mobile health technology: protocol for an implementation research study. Implement Sci 2014 May 29;9:65 [FREE Full text] [CrossRef] [Medline]
    12. Kypri K, Langley JD, Saunders JB, Cashell-Smith ML, Herbison P. Randomized controlled trial of web-based alcohol screening and brief intervention in primary care. Arch Intern Med 2008 Mar 10;168(5):530-536. [CrossRef] [Medline]
    13. Bertholet N, Daeppen J, Wietlisbach V, Fleming M, Burnand B. Reduction of alcohol consumption by brief alcohol intervention in primary care: systematic review and meta-analysis. Arch Intern Med 2005 May 9;165(9):986-995. [CrossRef] [Medline]
    14. Harris SK, Louis-Jacques J, Knight JR. Screening and brief intervention for alcohol and other abuse. Adolesc Med State Art Rev 2014 Apr;25(1):126-156. [Medline]
    15. Ramsey AT, Montgomery K. Technology-based interventions in social work practice: a systematic review of mental health interventions. Soc Work Health Care 2014 Oct;53(9):883-899 [FREE Full text] [CrossRef] [Medline]
    16. Choo EK, Ranney ML, Aggarwal N, Boudreaux ED. A systematic review of emergency department technology-based behavioral health interventions. Acad Emerg Med 2012 Mar;19(3):318-328. [CrossRef] [Medline]
    17. Donoghue K, Patton R, Phillips T, Deluca P, Drummond C. The effectiveness of electronic screening and brief intervention for reducing levels of alcohol consumption: a systematic review and meta-analysis. J Med Internet Res 2014;16(6):e142 [FREE Full text] [CrossRef] [Medline]
    18. Kaner EF, Beyer FR, Muirhead C, Campbell F, Pienaar ED, Bertholet N, et al. Effectiveness of brief alcohol interventions in primary care populations. Cochrane Database Syst Rev 2018 Dec 24;2:1-248. [CrossRef] [Medline]
    19. Kaner EF, Beyer FR, Garnett C, Crane D, Brown J, Muirhead C, et al. Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. Cochrane Database Syst Rev 2017 Dec 25;9:1-185. [CrossRef] [Medline]
    20. Nair NK, Newton NC, Shakeshaft A, Wallace P, Teesson M. A systematic review of digital and computer-based alcohol intervention programs in primary care. Curr Drug Abuse Rev 2015;8(2):111-118. [CrossRef] [Medline]
    21. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015 Jan;4:1 [FREE Full text] [CrossRef] [Medline]
    22. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009 Jul 21;6(7):e1000100 [FREE Full text] [CrossRef] [Medline]
    23. Bendtsen P, Stark Ekman D, Johansson A, Carlfjord S, Andersson A, Leijon M, et al. Referral to an electronic screening and brief alcohol intervention in primary health care in sweden: impact of staff referral to the computer. Int J Telemed Appl 2011:1-10 [FREE Full text] [CrossRef] [Medline]
    24. Bischof G, Grothues JM, Reinhardt S, Meyer C, John U, Rumpf H. Evaluation of a telephone-based stepped care intervention for alcohol-related disorders: a randomized controlled trial. Drug Alcohol Depend 2008 Mar 1;93(3):244-251. [CrossRef] [Medline]
    25. Brown RL, Saunders LA, Bobula JA, Mundt MP, Koch PE. Randomized-controlled trial of a telephone and mail intervention for alcohol use disorders: three-month drinking outcomes. Alcohol Clin Exp Res 2007 Aug;31(8):1372-1379. [CrossRef] [Medline]
    26. Butler SF, Chiauzzi E, Bromberg JI, Budman SH, Buono DP. Computer-assisted screening and intervention for alcohol problems in primary care. J Technol Hum Serv 2003 Jun;21(3):1-19. [CrossRef]
    27. Cucciare MA, Weingardt KR, Ghaus S, Boden MT, Frayne SM. A randomized controlled trial of a web-delivered brief alcohol intervention in Veterans Affairs primary care. J Stud Alcohol Drugs 2013 May;74(3):428-436. [CrossRef] [Medline]
    28. Dickinson WP, Glasgow RE, Fisher L, Dickinson LM, Christensen SM, Estabrooks PA, et al. Use of a website to accomplish health behavior change: if you build it, will they come? And will it work if they do? J Am Board Fam Med 2013;26(2):168-176 [FREE Full text] [CrossRef] [Medline]
    29. Dimeff L, McNeely M. Computer-enhanced primary care practitioner advice for high-risk college drinkers in a student primary health-care setting. Cogn Behav Pract 2000 Dec;7(1):82-100. [CrossRef]
    30. Fink A, Elliott MN, Tsai M, Beck JC. An evaluation of an intervention to assist primary care physicians in screening and educating older patients who use alcohol. J Am Geriatr Soc 2005 Nov;53(11):1937-1943. [CrossRef] [Medline]
    31. Gryczynski J, Mitchell SG, Gonzales A, Moseley A, Peterson TR, Ondersma SJ, et al. A randomized trial of computerized vs in-person brief intervention for illicit drug use in primary care: outcomes through 12 months. J Subst Abuse Treat 2015 Mar;50:3-10 [FREE Full text] [CrossRef] [Medline]
    32. Harris SK, Csémy L, Sherritt L, Starostova O, van Hook S, Johnson J, et al. Computer-facilitated substance use screening and brief advice for teens in primary care: an international trial. Pediatrics 2012 Jun;129(6):1072-1082 [FREE Full text] [CrossRef] [Medline]
    33. Hasin DS, Aharonovich E, O'Leary A, Greenstein E, Pavlicova M, Arunajadai S, et al. Reducing heavy drinking in HIV primary care: a randomized trial of brief intervention, with and without technological enhancement. Addiction 2013 Jul;108(7):1230-1240 [FREE Full text] [CrossRef] [Medline]
    34. Helstrom AW, Ingram E, Wang W, Small D, Klaus J, Oslin D. Treating heavy drinking in primary care practices: evaluation of a telephone-based intervention program. Addict Disord Their Treat 2014;13(3):101-109. [CrossRef]
    35. Helzer JE, Rose GL, Badger GJ, Searles JS, Thomas CS, Lindberg SA, et al. Using interactive voice response to enhance brief alcohol intervention in primary care settings. J Stud Alcohol Drugs 2008 Mar;69(2):251-258. [CrossRef] [Medline]
    36. Holtrop JS, Dosh SA, Torres T, Thum YM. The community health educator referral liaison (CHERL): a primary care practice role for promoting healthy behaviors. Am J Prev Med 2008 Nov;35(5 Suppl):S365-S372. [CrossRef] [Medline]
    37. Kalapatapu RK, Ho J, Cai X, Vinogradov S, Batki SL, Mohr DC. Cognitive-behavioral therapy in depressed primary care patients with co-occurring problematic alcohol use: effect of telephone-administered vs face-to-face treatment-a secondary analysis. J Psychoactive Drugs 2014;46(2):85-92 [FREE Full text] [CrossRef] [Medline]
    38. Kypri K, Saunders JB, Williams SM, McGee RO, Langley JD, Cashell-Smith ML, et al. Web-based screening and brief intervention for hazardous drinking: a double-blind randomized controlled trial. Addiction 2004 Nov;99(11):1410-1417. [CrossRef] [Medline]
    39. Lin JC, Karno MP, Tang L, Barry KL, Blow FC, Davis JW, et al. Do health educator telephone calls reduce at-risk drinking among older adults in primary care? J Gen Intern Med 2010 Apr;25(4):334-339 [FREE Full text] [CrossRef] [Medline]
    40. McCausland C. A Randomized Clinical Trial of Computer Delivered Personal Feedback for Hazardous Drinkers in Primary Care. Unpublished Dissertation: The University of Memphis; 2011.
    41. McMenamin J, Nicholson R, Leech K. Patient Dashboard: the use of a colour-coded computerised clinical reminder in Whanganui regional general practices. J Prim Health Care 2011 Dec 1;3(4):307-310. [CrossRef] [Medline]
    42. Moore AA, Blow FC, Hoffing M, Welgreen S, Davis JW, Lin JC, et al. Primary care-based intervention to reduce at-risk drinking in older adults: a randomized controlled trial. Addiction 2011 Jan;106(1):111-120 [FREE Full text] [CrossRef] [Medline]
    43. Olson AL, Gaffney CA, Hedberg VA, Gladstone GR. Use of inexpensive technology to enhance adolescent health screening and counseling. Arch Pediatr Adolesc Med 2009 Feb;163(2):172-177. [CrossRef] [Medline]
    44. Oslin DW, Sayers S, Ross J, Kane V, Ten HT, Conigliaro J, et al. Disease management for depression and at-risk drinking via telephone in an older population of veterans. Psychosom Med 2003;65(6):931-937. [CrossRef] [Medline]
    45. Pereira CA, Wen CL, Tavares H. Alcohol abuse management in primary care: an e-learning course. Telemed J E Health 2015 Mar;21(3):200-206. [CrossRef] [Medline]
    46. Possemato K, Bishop TM, Willis MA, Lantinga LJ. Healthcare utilization and symptom variation among veterans using Behavioral Telehealth Center services. J Behav Health Serv Res 2013 Oct;40(4):416-426. [CrossRef] [Medline]
    47. Ruf D, Berner M, Kriston L, Lohmann M, Mundle G, Lorenz G, et al. Cluster-randomized controlled trial of dissemination strategies of an online quality improvement programme for alcohol-related disorders. Alcohol Alcohol 2010;45(1):70-78. [CrossRef] [Medline]
    48. Schwartz RP, Gryczynski J, Mitchell SG, Gonzales A, Moseley A, Peterson TR, et al. Computerized versus in-person brief intervention for drug misuse: a randomized clinical trial. Addiction 2014 Jul;109(7):1091-1098 [FREE Full text] [CrossRef] [Medline]
    49. Stoner SA, Mikko AT, Carpenter KM. Web-based training for primary care providers on screening, brief intervention, and referral to treatment (SBIRT) for alcohol, tobacco, and other drugs. J Subst Abuse Treat 2014;47(5):362-370 [FREE Full text] [CrossRef] [Medline]
    50. Tanner TB, Wilhelm SE, Rossie KM, Metcalf MP. Web-based SBIRT skills training for health professional students and primary care providers. Subst Abus 2012;33(3):316-320 [FREE Full text] [CrossRef] [Medline]
    51. Vinson DC, Devera-Sales A. Computer‐generated written behavioral contracts with problem drinkers in primary medical care. Subst Abus 2000 Dec;21(4):215-222. [CrossRef] [Medline]
    52. Walton MA, Bohnert K, Resko S, Barry KL, Chermack ST, Zucker RA, et al. Computer and therapist based brief interventions among cannabis-using adolescents presenting to primary care: one year outcomes. Drug Alcohol Depend 2013 Oct 1;132(3):646-653 [FREE Full text] [CrossRef] [Medline]
    53. Walton MA, Resko S, Barry KL, Chermack ST, Zucker RA, Zimmerman MA, et al. A randomized controlled trial testing the efficacy of a brief cannabis universal prevention program among adolescents in primary care. Addiction 2014 May;109(5):786-797 [FREE Full text] [CrossRef] [Medline]
    54. Williams EC, Achtmeyer CE, Kivlahan DR, Greenberg D, Merrill JO, Wickizer TM, et al. Evaluation of an electronic clinical reminder to facilitate brief alcohol-counseling interventions in primary care. J Stud Alcohol Drugs 2010 Sep;71(5):720-725 [FREE Full text] [Medline]
    55. Wongpakaran T, Petcharaj K, Wongpakaran N, Sombatmai S, Boripuntakul T, Intarakamhaeng D, et al. The effect of telephone-based intervention (TBI) in alcohol abusers: a pilot study. J Med Assoc Thai 2011 Jul;94(7):849-856. [Medline]
    56. Zanjani F, Bush H, Oslin D. Telephone-based psychiatric referral-care management intervention health outcomes. Telemed J E Health 2010 Jun;16(5):543-550 [FREE Full text] [CrossRef] [Medline]
    57. Acosta MC, Possemato K, Maisto SA, Marsch LA, Barrie K, Lantinga L, et al. Web-delivered CBT reduces heavy drinking in OEF-OIF veterans in primary care with symptomatic substance use and PTSD. Behav Ther 2017 Dec;48(2):262-276 [FREE Full text] [CrossRef] [Medline]
    58. Dawson-Rose C, Draughon JE, Cuca Y, Zepf R, Huang E, Cooper BA, et al. Changes in Specific Substance Involvement Scores among SBIRT recipients in an HIV primary care setting. Addict Sci Clin Pract 2017 Dec 12;12(1):34 [FREE Full text] [CrossRef] [Medline]
    59. Duru OK, Xu H, Moore AA, Mirkin M, Ang A, Tallen L, et al. Examining the impact of separate components of a multicomponent intervention designed to reduce at risk drinking among older adults: the Project SHARE study. Alcohol Clin Exp Res 2015 Jul;39(7):1227-1235 [FREE Full text] [CrossRef] [Medline]
    60. Hunter R, Wallace P, Struzzo P, Vedova RD, Scafuri F, Tersar C, et al. Randomised controlled non-inferiority trial of primary care-based facilitated access to an alcohol reduction website: cost-effectiveness analysis. BMJ Open 2017 Nov 3;7(11):e014577 [FREE Full text] [CrossRef] [Medline]
    61. Knight JR, Kuzubova K, Csemy L, Sherritt L, Copelas S, Harris SK. Computer-facilitated screening and brief advice to reduce adolescents' heavy episodic drinking: a study in two countries. J Adolesc Health 2018 Jan;62(1):118-120 [FREE Full text] [CrossRef] [Medline]
    62. Quanbeck A, Gustafson DH, Marsch LA, Chih M, Kornfield R, McTavish F, et al. Implementing a mobile health system to integrate the treatment of addiction into primary care: a hybrid implementation-effectiveness study. J Med Internet Res 2018 Jan 30;20(1):e37 [FREE Full text] [CrossRef] [Medline]
    63. Rose GL, Badger GJ, Skelly JM, MacLean CD, Ferraro TA, Helzer JE. A randomized controlled trial of brief intervention by interactive voice response. Alcohol Alcohol 2017 May 1;52(3):335-343 [FREE Full text] [CrossRef] [Medline]
    64. Grol R, Wensing M, Eccles M, Davis D. Improving Patient Care: The Implementation of Change in Health Care. Chichester: John Wiley & Sons Inc; 2013.
    65. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013 Dec 1;8:139 [FREE Full text] [CrossRef] [Medline]
    66. Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series. Washington, DC: National Academies Press; 2006.
    67. Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, et al. An implementation research agenda. Implement Sci 2009;4:18 [FREE Full text] [CrossRef] [Medline]
    68. Ramsey A, Lord S, Torrey J, Marsch L, Lardiere M. Paving the way to successful implementation: identifying key barriers to use of technology-based therapeutic tools for behavioral health care. J Behav Health Serv Res 2016 Jan;43(1):54-70. [CrossRef] [Medline]
    69. Lord S, Moore SK, Ramsey A, Dinauer S, Johnson K. Implementation of a substance use recovery support mobile application in community settings: qualitative study of clinician and staff perspectives of facilitators and barriers. JMIR Ment Health 2016 Jun 28;3(2):e24 [FREE Full text] [CrossRef] [Medline]
    70. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev 2012 Apr;69(2):123-157 [FREE Full text] [CrossRef] [Medline]
    71. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015 Feb 12;10:21 [FREE Full text] [CrossRef] [Medline]
    72. Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported mental health interventions. Res Soc Work Pract 2014 Mar 1;24(2):192-212 [FREE Full text] [CrossRef] [Medline]
    73. Ryan R, Hill S, Prictor M, McKenzie J. Cochrane Consumers and Communication Review Group. 2013 May. Study Quality Guide   URL: https://cccrg.cochrane.org/author-resources [accessed 2017-11-27] [WebCite Cache]
    74. Eysenbach G, Powell J, Englesakis M, Rizo C, Stern A. Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions. Br Med J 2004 May 15;328(7449):1166 [FREE Full text] [CrossRef] [Medline]
    75. Newman MG, Szkodny LE, Llera SJ, Przeworski A. A review of technology-assisted self-help and minimal contact therapies for drug and alcohol abuse and smoking addiction: is human contact necessary for therapeutic efficacy? Clin Psychol Rev 2011 Feb;31(1):178-186. [CrossRef] [Medline]
    76. Gainsbury S, Blaszczynski A. A systematic review of Internet-based therapy for the treatment of addictions. Clin Psychol Rev 2011 Apr;31(3):490-498. [CrossRef] [Medline]
    77. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res 2017 Apr;44(2):177-194 [FREE Full text] [CrossRef] [Medline]
    78. Lyon AR, Lewis CC. Designing health information technologies for uptake: development and implementation of measurement feedback systems in mental health service delivery. Adm Policy Ment Health 2016 May;43(3):344-349 [FREE Full text] [CrossRef] [Medline]
    79. Ornstein SM, Miller PM, Wessell AM, Jenkins RG, Nemeth LS, Nietert PJ. Integration and sustainability of alcohol screening, brief intervention, and pharmacotherapy in primary care settings. J Stud Alcohol Drugs 2013 Jul;74(4):598-604 [FREE Full text] [Medline]
    80. Williams EC, Johnson ML, Lapham GT, Caldeiro RM, Chew L, Fletcher GS, et al. Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review. Psychol Addict Behav 2011 Jun;25(2):206-214. [CrossRef] [Medline]
    81. Glisson C, Schoenwald SK, Hemmelgarn A, Green P, Dukes D, Armstrong KS, et al. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol 2010 Aug;78(4):537-550 [FREE Full text] [CrossRef] [Medline]
    82. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci 2013;8:117 [FREE Full text] [CrossRef] [Medline]
    83. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
    84. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 2009 Jan;36(1):24-34 [FREE Full text] [CrossRef] [Medline]
    85. Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry 2013 Aug;35(4):332-338 [FREE Full text] [CrossRef] [Medline]
    86. Haskins BL, Davis-Martin R, Abar B, Baumann BM, Harralson T, Boudreaux ED. Health evaluation and referral assistant: a randomized controlled trial of a web-based screening, brief intervention, and referral to treatment system to reduce risky alcohol use among emergency department patients. J Med Internet Res 2017 May 1;19(5):e119 [FREE Full text] [CrossRef] [Medline]

    ‎
    ANOVA: analysis of variance
    EBI: evidence-based intervention
    EHR: electronic health record
    TAU: treatment as usual

    Edited by G Eysenbach; submitted 23.04.18; peer-reviewed by J Glass, K Champion, A Fink; comments to author 03.09.18; revised version received 29.01.19; accepted 30.01.19; published 08.04.19

    Copyright

    ©Alex T Ramsey, Jason M Satterfield, Donald R Gerke, Enola K Proctor. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.04.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.