#### Original Paper

### Abstract

Background: Almost a decade ago, Sweden became the first country to implement a national system enabling student health care centers across all universities to routinely administer (via email) an electronic alcohol screening and brief intervention to their students. The Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS-1) trial aimed to assess the effect of the student health care centers’ routine practices by exploiting the lack of any standard timing for the email invitation and by masking trial participation from students. The original analyses adopted the conventional null hypothesis framework, and the results were consistently in the expected direction. However, since for some tests the *P* values did not pass the conventional .05 threshold, some of the analyses were necessarily inconclusive.

Objective: The outcomes of the AMADEUS-1 trial were derived from the first 3 items of the Alcohol Use Disorders Identification Test (AUDIT-C). The aim of this paper was to reanalyze the two primary outcomes of the AMADEUS-1 trial (AUDIT-C scores and prevalence of risky drinking), using the same models used in the original publication but applying a Bayesian inference framework and interpretation.

Methods: The same regression models used in the original analysis were employed in this reanalysis (linear and logistic regression). Model parameters were given uniform priors. Markov chain Monte Carlo was used for Bayesian inference, and posterior probabilities were calculated for prespecified thresholds of interest.

Results: Where the null hypothesis tests showed inconclusive results, the Bayesian analysis showed that offering an intervention at baseline was preferable compared to offering nothing. At follow-up, the probability of a lower AUDIT-C score among those who had been offered an intervention at baseline was greater than 95%, as was the case when comparing the prevalence of risky drinking.

Conclusions: The Bayesian analysis allows for a more consistent perspective of the data collected in the trial, since dichotomization of evidence is not looked for at some arbitrary threshold. Results are presented that represent the data collected in the trial rather than trying to make conclusions about the existence of a population effect. Thus, policy makers can think about the value of keeping the national system without having to navigate the treacherous landscape of statistical significance.

Trial Registration: ISRCTN Registry ISRCTN28328154; http://www.isrctn.com/ISRCTN28328154

**J Med Internet Res 2019;21(11):e14419**

doi:10.2196/14419

### Keywords

### Introduction

#### Background

Alcohol consumption contributed to more than 4.5% of deaths globally in 2016 [

] and was the leading risk factor among the population aged 15-49 years old. It has been suggested that alcohol policies might need to be revised worldwide to lower the overall population-level consumption [ ]. While policies controlling price and availability may be one way forward [ ], the advent of electronic health (eHealth) interventions has made us better equipped to deliver personal behavior change interventions to larger populations.Early initiatives to use digital means of delivering alcohol interventions came in the form of electronic screening and brief interventions (eSBIs) [

- ]. Typically, these interventions ask participants to complete a questionnaire, after which feedback is given on their responses and some advice on behavior change is offered (based on recommended drinking levels). The feedback and advice are commonly designed around behavior change theories and models, such as protection motivation theory [ ], social cognitive theory [ ] and the theory of planned behavior [ ]. In general, eHealth interventions for alcohol behavior change have shown promise when they have included components that focus on behavior substitution, problem solving, goal setting, review of behavioral goals, self-monitoring, and normative feedback [ , ].Meta-analyses suggest that there exists a small positive effect of eSBIs on the amount of alcohol consumed weekly in the short term, with a Cohen *d*=−0.17 (95% CI −0.27 to −0.18) found in one analysis [ ], a Cohen *d*=−0.14 (95% CI −0.24 to −0.03) in another analysis [ ], and a weighted mean difference of alcohol in grams=−16.59 (95% CI −23.70 to −9.48) in a third analysis [ ]. Although long-term effects have not been measurable, these brief interventions are nevertheless useful for reaching many individuals at a low cost.

Almost a decade ago, Sweden became the first country to implement a national system enabling student health care centers across all universities to routinely administer eSBIs. The system, which is still routinely used today, sends an email to all university students with an invitation and a hyperlink to a 10-item web questionnaire which is then followed by personal feedback and advice. At the time this system was introduced, there was some evidence of the effectiveness of eSBIs but there was a paucity for large-scale, multisite, effectiveness trials of routine care systems.

#### The Alcohol Email Assessment and Feedback Study Dismantling Effectiveness for University Students Trial

The Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS-1) trial [

, ], conducted in 2011, aimed to assess the effect of the student health care centers’ routine practices by exploiting the lack of any standard timing of the email invitation and by masking trial participation from students. The trial outcome was originally reported in 2013 [ ].During the autumn term of 2011, all students in semesters 1, 3 and 5 at two universities in Sweden (Linköping and Luleå) were included in the AMADEUS-1 trial. Notably, students’ email addresses were randomized into 3 groups (Group 1, Group 2, and Group 3) prior to any invitation or contact with the students. Ethical concerns with the use of this type of masking was considered and approved by the Regional Ethical Committee in Linköping, Sweden (No 2010/291-31). In a subsequent trial (the AMADEUS-2 trial [

, ]) a more conventional approach was used to estimate the effect of eSBIs on harmful and hazardous drinkers. A Bayesian reanalysis of the AMADEUS-2 trial has also been reported [ ].On September 5, 2011, Group 1 and Group 2 were sent an email from the student health care center with a hyperlink to a web questionnaire comprising 10 items which assessed their current alcohol consumption, masked as part of routine care. Group 1 was additionally told that they would also get feedback, which they received immediately after responding to the questionnaire. Group 2 was thanked for their participation and offered a hyperlink to a website with general information about alcohol, which was not believed to have any content helpful for supporting behavior change. Group 3 was not contacted at this time.

Three months after the initial email to Group 1 and Group 2, all three groups were sent identical emails with an invitation to participate in a web-based general lifestyle survey where 3 out of the 15 items were the first 3 items of the Alcohol Use Disorders Identification Test (AUDIT-C [

]). Crucially, this invitation made no reference to the alcohol assessment conducted three months earlier and it was not disclosed as a follow-up questionnaire in a randomized trial.#### Objectives

Outcomes of the AMADEUS-1 trial were derived from the 3 AUDIT-C items in the general lifestyle survey. This reanalysis will focus on two primary outcomes: AUDIT-C scores and prevalence of risky drinking. In Sweden, risky drinkers are those who fulfil at least one of two criteria: (1) heavy episodic drinking of at least 4 (female) or 5 (male) standard drinks of alcohol on one occasion the past month; or (2) consuming more than 9 (female) or 14 (male) standard drinks of alcohol per week. One standard drink is defined as 12 grams of alcohol in Sweden.

The current goal is to reanalyze the two primary outcomes of the AMADEUS-1 trial, using the same models used in the original publication but also using a Bayesian inference framework and interpretation.

### Methods

#### Overview

In the original analysis of the AMADEUS-1 trial, normal regression was used to contrast AUDIT-C scores (log-transformed) and logistic regression was used to contrast risky drinking. Both models were adjusted for baseline variables. In this Bayesian analysis, the same regression models were used, and uniform priors were applied to all model parameters. The full specifications of the Bayesian models can be seen in the following 2 equations. Separate analyses were done comparing Group 1 versus Group 3 and Group 2 versus Group 3. In all cases, Group 3 was considered the control group and Group 1 and Group 2 were considered intervention groups.

Equation 1:

Equation 2:

#### Alcohol Use Disorders Identification Test

When contrasting AUDIT-C scores (Equation 1), the primary interest of the analyses was the regression coefficient for the group variable (α_{1}). A negative value for α_{1} suggests that the group which was randomized to receive an intervention (Group 1 and Group 2 respectively) had, on average, lower AUDIT-C scores at follow-up than the group which was randomized to the control setting (Group 3). Coefficients were back transformed prior to inspection. Informed by the original analysis, it was decided that thresholds of interest for which the marginal posterior distribution for α_{1} should be inspected were 0, –0.02, and –0.04. The thresholds were chosen to communicate whether offering an intervention is preferable to not doing so (the 0 threshold), and to indicate the magnitude of the difference between groups (–0.02 and –0.04).

#### Risky Drinking

When contrasting risky drinking (Equation 2), the primary interest was the regression coefficient for the group variable (β_{1}), that is the log of the odds ratio (OR) between the group which was randomized to an intervention (Group 1 and Group 2 respectively) and the group which was randomized to the control setting (Group 3). Coefficients were exponentiated before inspection, thus a value of β_{1} lower than 1 would suggest that the odds of risky drinking in the intervention group was lower than the odds in the control. Informed by the original analysis, it was decided that thresholds of interest for which the marginal posterior distribution for β_{1} should be inspected was 1, 0.9 and 0.8. Again, the thresholds were chosen to communicate whether offering an intervention is preferable to not doing so (the 1 threshold), and to indicate the magnitude of the difference between the groups (0.9 and 0.8).

#### Inference

Markov chain Monte Carlo was used for Bayesian inference (RStan version 2.16.2). For each model, 50,000 iterations were run with 25,000 warmup iterations in four chains. Inference for AUDIT-C scores (Equation 1) took 3.5 minutes, and for risky drinking (Equation 2) 5.5 minutes. All computations were done on a MacBook Pro (2017 model).

### Results

#### Primary Findings

A total of 14,910 students were randomized into the 3 arms of the trial. In Group 1, 36.2% (1798/4969) of participants completed the eSBI, 32.6% (1621/4969) of Group 2 participants completed the alcohol screening questionnaire, and as previously discussed Group 3 was not contacted at this point. Approximately half of all students responded to the general lifestyle survey that was sent three months after randomization: 51.2% (2546/4969) in Group 1, 52.2% (2594/4969) in Group 2, and 53.7% (2669/4972) in Group 3.

#### Original Analysis: Null Hypothesis Framework

The original analysis for the AMADEUS-1 trial is presented in *P*=.07) while Group 2 and Group 3 did (*P*=.04), with Group 2, on average, reporting a lower AUDIT-C score than Group 3. Risky drinking was found to be statistically significantly different between Group 1 and Group 3 (*P*=.006), with risky drinking less prevalent in Group 1 than in Group 3, but not so for Group 2 and Group 3 (*P*=.08).

As a reminder, *P* values indicate how likely it is that we would have seen the data that we did in the trial in a hypothetical world where the population effect is exactly zero. Convention says that if the data is less likely than 5%, then we should reject the hypothetical world. However, it does not mean that if the 5% threshold is not broken that we should accept the hypothetical world. Instead, the confidence interval indicates which hypothetical worlds cannot be rejected given the data that we have seen in the trial.

Categories | Group 1 (n=2546) | Group 2 (n=2594) | Group 3 (n=2669) | Group 1 versus 3 | Group 2 versus 3 | ||

Regression coefficient^{a}, 95% CI | P value | Regression coefficient^{a}, 95% CI | P value | ||||

AUDIT-C^{b} | 3.46 (3.09)^{c} | 3.44 (3.17)^{c} | 3.60 (3.14)^{c} | –0.032 (–0.066 to 0.003) | .07 | –0.038 (–0.072 to –0.002) | .04 |

Risky drinking^{d} | 1136 (44.6)^{e} | 1194 (46.0)^{e} | 1288 (48.3)^{e} | 0.85 (0.76 to 0.95) | .006 | 0.90 (0.81 to 1.01) | .08 |

^{a}Linear coefficient for AUDIT-C scores (back transformed) and odds ratio for risky drinking (adjusted for sex, age, university, and semester).

^{b}AUDIT-C: Alcohol Use Disorders Identification Test.

^{c}Geometric mean (SD). Approximate standard deviation back-calculated from the log-scale.

^{d}Risky drinking: heavy episodic drinking ≥1 a month or weekly consumption >14 for men and >9 for women (Swedish national guidelines).

^{e}n (%).

#### Bayesian Analysis

The computational result of a Bayesian analysis using Markov chain Monte Carlo uses samples from the posterior distribution of each parameter of interest. Histograms of these samples are shown in _{1} in Equation 1, back transformed) and the risky drinking models (β_{1} in Equation 2, exponentiated). For instance, in we can see that there is a majority of samples to the left of 0, indicating that it is more likely than not that there was a difference in AUDIT-C scores at follow-up between Groups 1 and 3. Similarly, in we can see that a majority of the samples are to the left of 1, indicating that the prevalence of risky drinking in Group 2 was lower than in Group 3 at follow-up (ie, the OR was lower than 1). For the enclosed analyses, no trends were found in the sampling when inspecting trace plots (see ).

Rather than just visually inspecting the histograms, the samples drawn during inference can be used to calculate probabilities of interest (

and ). For example, when comparing the prevalence of risky drinking between Group 1 and Group 3 in , the ratio of samples that were lower than 1 was 99.7%, thus there was a 99.7% probability that the OR was less than 1 (indicating fewer risky drinkers in Group 1 compared to Group 3). Furthermore, there was an 82.4% probability that the OR was less than 0.9.Group 1 versus 3 | Group 2 versus 3 | |||||

Threshold 1 | Threshold 2 | Threshold 3 | Threshold 1 | Threshold 2 | Threshold 3 | |

Regression coefficient^{a} (AUDIT-C^{b}) | <0 | <–0.02 | <–0.04 | <0 | <–0.02 | <–0.04 |

Marginal posterior probability (%) | 96.4 | 75.7 | 32.9 | 98.1 | 83.7 | 44.4 |

^{a}Back transformed linear regression coefficient (model adjusted for sex, age, university, and semester).

^{b}AUDIT-C: Alcohol Use Disorders Identification Test.

Group 1 versus 3 | Group 2 versus 3 | |||||

Threshold 1 | Threshold 2 | Threshold 3 | Threshold 1 | Threshold 2 | Threshold 3 | |

Odds ratio^{a} (Risky drinking) | <1 | <0.9 | <0.8 | <1 | <0.9 | <0.8 |

Marginal posterior probability (%) | 99.7 | 82.4 | 13.4 | 96.1 | 46.7 | 1.6 |

^{a}Logistic regression coefficient in terms of odds ratios (model adjusted for sex, age, university, and semester).

### Discussion

#### Key Findings

When comparing the analysis done in a null hypothesis framework with one done within the Bayesian framework, it is important to remind oneself of what the quantities represent as the questions being asked and answered are different.

The null hypothesis testing approach aims to put forth evidence about the population value of a parameter (ie, the existence of an effect on the entire population). The *P* value indicates how extreme the collected data are, given a fixed population value which is often set at a no-effect level. If the data is unlikely to have been generated from a population where the intervention has no effect, then the null hypothesis is rejected, and we can state that, with statistical significance, we believe that the intervention has a population effect. The fundamental issues with this approach have been discussed elsewhere [ - ], as has the problematic misinterpretation of *P* values and confidence intervals [ , ].

On the other hand, the Bayesian approach only concerns itself with the data at hand. It does not attempt to say anything about a population level effect, but instead calculates posterior distributions over model parameters. We can use these posterior distributions to calculate the probability of there being a difference between groups with respect to different trial outcomes.

#### Alcohol Use Disorders Identification Test

##### Null Hypothesis Framework

When contrasting AUDIT-C scores (Equation 1) using the null hypothesis framework (*P*=.07), but there was a significant difference between Group 2 and Group 3 (*P*=.04). This is somewhat counterintuitive, as Group 1 and 2 were given identical questionnaires, but Group 1 was also given feedback and advice. However, due to the nature of null hypothesis testing, we cannot discuss the effect of the feedback and advice component, since the very existence of an effect cannot be determined. Yet, we are to conclude that there does exist an effect with respect to responding to the questionnaire itself. Both *P* values are close to the conventional threshold of .05, and it is noteworthy that the entire discussion about effects would have changed had we adopted a .08 threshold or a .03 threshold.

In the original report [*P* values crossing the conventional threshold.

##### Bayesian Framework

The Bayesian approach (

) suggests that the probability that Group 1 had a lower AUDIT-C score on average compared to Group 2 at follow-up was 96.4%, and when comparing Group 2 and Group 3 this probability was 98.1%. The model does not make a dichotomous decision about the existence of an effect but states the probability that a difference existed between the groups. Licensed by the randomization component of the trial design, we may conclude that this difference is due to the groups receiving different treatments. We may also conclude that the difference between groups is more likely than not to be greater than 0.02 units on the AUDIT-C scale but also is more likely than not to be less than 0.04.#### Risky Drinking

##### Null Hypothesis Framework

In

we can see that the difference in prevalence of risky drinking between Group 1 and Group 3 was statistically significant, but not between Group 2 and Group 3. The existence of an effect on risky drinking is thus confirmed for the questionnaire plus feedback and advice intervention, but the evidence is inconclusive for the effect of the questionnaire alone. Recall that the situation was the opposite when analyzing AUDIT-C scores.##### Bayesian Framework

The Bayesian approach (

) suggests that there is a 99.7% probability that the prevalence of risky drinking was lower in Group 1 compared to Group 3, and that this probability was 96.1% when comparing Group 2 and Group 3. Note, however, that it was more likely than not that the OR was less than 0.9 when comparing Group 1 and Group 3, but not so when comparing Group 2 and Group 3. We can also see this when comparing and , as most of the samples drawn when comparing Group 1 and Group 3 are to the left of 0.9, while they are centered around 0.9 when comparing Group 2 and Group 3. As was the case with AUDIT-C scores, we may attribute the difference between groups to the different treatments licensed by the trial design.#### Clinical Significance

Clinically significant effect sizes are not universal, as they depend on the context in which the intervention can be offered and must be decided upon given cost, alternative interventions, ethical and practical concerns, and so on. One of the benefits of using a Bayesian approach is that we have access to a posterior distribution over the parameters of our model, which allows us to answer questions such as, “What is the probability that the effect is X or greater?” Therefore, we can evaluate the probability of clinically significant effect sizes in several different contexts. For instance, at the time of the AMADEUS-1 trial, student health care centers in Sweden did not have any means of reaching the entire student population with a brief intervention, thus there were no alternative interventions to the eSBI on trial. In addition, there was very little cost involved in adopting the eSBI into routine practice.

and indicate that there was a 4-percentage point difference in risky drinking between Groups 1 and 3 (OR<0.9; 82.4% probability), and this was considered a significant enough effect size to mandate a full-scale adoption of the intervention.The years to come after the AMADEUS-1 trial saw many more trials of eSBIs, and as was mentioned earlier, meta-analyses suggest a small positive effect of eSBIs on the amount of alcohol consumed weekly in the short term.

#### Limitations

The AMADEUS-1 trial was unconventional in the sense that participants were randomized prior to being invited to the trial. This design allowed for a naturalistic study context and allowed for methodological advantages. However, participation rates were lower than would be expected in a more traditional setting where participants are randomized after registering interest in the trial (eg, only 36.2% [1798/4969] of participants allocated to Group 1 completed the eSBI). The overall follow-up rate was not remarkable at 52% (7764/14,910), which at the time was considered average for eHealth trials. Since missing at random cannot be guaranteed, effect sizes should be considered in the light that bias might have been introduced due to lower than ideal follow-up rates.

#### Summary

In the original publication of the AMADEUS-1 trial, we summarized the main results as follows [*P*<.10 for four of five comparisons for both groups 1 and 2.

Since not all *P* values from the hypotheses tests passed conventional thresholds, we could not conclude that the different groups had been affected by treating them differently. Unfortunately, the desire to dichotomize evidence prohibited any further discussion. This dichotomization of evidence creates issues when interpreting results from trials, as clearly the contradicting results when contrasting AUDIT-C scores is difficult to explain and communicate. Thus, we may ask if it is prudent for the student health care centers to decide to change their policy of offering eSBIs to all students since the .05 threshold was not broken? What if a different threshold was chosen? Should the results simply be discarded as inconclusive?

In this Bayesian reanalysis, we may instead summarize our findings as: There is 96.4% probability that Group 1 had a lower AUDIT-C score on average than did Group 3, and there is a 99.7% probability that the prevalence of risky drinking was lower in Group 1 compared to Group 3 (and a further 82.4% probability that the OR was less than 0.9). This then allows us to go forth and inspect the posterior distributions at effect sizes that are clinically significant in different contexts and discuss whether the intervention should be adopted into routine practice.

#### Conflicts of Interest

MB owns a private company that develops and distributes evidence-based lifestyle interventions to be used in health care settings, including student health care centers.

Trace plots.

PDF File (Adobe PDF File), 397 KB#### References

- WHO, Management OSAT. Global Status Report on Alcohol and Health 2018. Geneva: WHO; 2018.
- GBD 2016 Alcohol Collaborators. Alcohol use and burden for 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 2018 Sep 22;392(10152):1015-1035 [FREE Full text] [CrossRef] [Medline]
- Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: No Ordinary Commodity: Research and Public Policy. Oxford: Oxford University Press; 2010.
- McCambridge J, Bendtsen P, Bendtsen M, Nilsen P. Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS-1): study protocol for a randomized controlled trial. Trials 2012 Jul 06;13:49 [FREE Full text] [CrossRef] [Medline]
- Bendtsen P, McCambridge J, Bendtsen M, Karlsson N, Nilsen P. Effectiveness of a proactive mail-based alcohol Internet intervention for university students: dismantling the assessment and feedback components in a randomized controlled trial. J Med Internet Res 2012 Oct 31;14(5):e142 [FREE Full text] [CrossRef] [Medline]
- McCambridge J, Bendtsen M, Karlsson N, White IR, Bendtsen P. Alcohol assessment & feedback by e-mail for university student hazardous and harmful drinkers: study protocol for the AMADEUS-2 randomised controlled trial. BMC Public Health 2013 Oct 10;13:949 [FREE Full text] [CrossRef] [Medline]
- McCambridge J, Bendtsen M, Karlsson N, White IR, Nilsen P, Bendtsen P. Alcohol assessment and feedback by email for university students: main findings from a randomised controlled trial. Br J Psychiatry 2013 Nov;203(5):334-340 [FREE Full text] [CrossRef] [Medline]
- Bendtsen P, Bendtsen M, Karlsson N, White IR, McCambridge J. Online Alcohol Assessment and Feedback for Hazardous and Harmful Drinkers: Findings From the AMADEUS-2 Randomized Controlled Trial of Routine Practice in Swedish Universities. J Med Internet Res 2015 Jul 09;17(7):e170 [FREE Full text] [CrossRef] [Medline]
- Rogers R. Cognitive and physiological processes in fear appeals and attitude change: A revised theory of protection motivation. In: Social Psychophysiology: A Sourcebook. New York City: Guilford Press; 1983:153-177.
- Bandura A. Self-efficacy: The exercise of control. New York City: W H Freeman/Times Books/ Henry Holt & Co; 1997.
- Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes 1991 Dec;50(2):179-211. [CrossRef]
- Michie S, Whittington C, Hamoudi Z, Zarnani F, Tober G, West R. Identification of behaviour change techniques to reduce excessive alcohol consumption. Addiction 2012 Aug;107(8):1431-1440. [CrossRef] [Medline]
- Garnett CV, Crane D, Brown J, Kaner EFS, Beyer FR, Muirhead CR, et al. Behavior Change Techniques Used in Digital Behavior Change Interventions to Reduce Excessive Alcohol Consumption: A Meta-regression. Ann Behav Med 2018 May 18;52(6):530-543 [FREE Full text] [CrossRef] [Medline]
- Smedslund G, Wollscheid S, Fang L, Nilsen W, Steiro A, Larun L. Effects of early, computerized brief interventions on risky alcohol use and risky cannabis use among young people. Campbell Systematic Reviews 2017 Apr 07;13(1):1-192. [CrossRef]
- Carey KB, Scott-Sheldon LAJ, Elliott JC, Garey L, Carey MP. Face-to-face versus computer-delivered alcohol interventions for college drinkers: a meta-analytic review, 1998 to 2010. Clin Psychol Rev 2012 Dec;32(8):690-703 [FREE Full text] [CrossRef] [Medline]
- Donoghue K, Patton R, Phillips T, Deluca P, Drummond C. The effectiveness of electronic screening and brief intervention for reducing levels of alcohol consumption: a systematic review and meta-analysis. J Med Internet Res 2014 Jun 02;16(6):e142 [FREE Full text] [CrossRef] [Medline]
- Bendtsen M. Electronic screening and brief intervention for hazardous and harmful drinkers among swedish university students: reanalysis of findings from a randomized controlled trial using a Bayesian framework (under review). J Med Internet Res 2019 Under review(forthcoming).
- Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med 1998 Sep 14;158(16):1789-1795. [CrossRef] [Medline]
- Bendtsen M. A Gentle Introduction to the Comparison Between Null Hypothesis Testing and Bayesian Analysis: Reanalysis of Two Randomized Controlled Trials. J Med Internet Res 2018 Oct 24;20(10):e10873 [FREE Full text] [CrossRef] [Medline]
- Amrhein V, Greenland S. Remove, rather than redefine, statistical significance. Nat Hum Behav 2018 Jan 25;2(1):4-4. [CrossRef] [Medline]
- Benjamin DJ, Berger JO, Johannesson M, Nosek BA, Wagenmakers E, Berk R, et al. Redefine statistical significance. Nat Hum Behav 2018 Jan 1;2(1):6-10. [CrossRef] [Medline]
- Berry D. A P-Value to Die For. Journal of the American Statistical Association 2017 Oct 30;112(519):895-897. [CrossRef]
- Briggs WM. The Substitute for P-Values. Journal of the American Statistical Association 2017 Oct 30;112(519):897-898. [CrossRef]
- Gelman A, Carlin J. Some Natural Solutions to the P-Value Communication Problem—and Why They Won’t Work. Journal of the American Statistical Association 2017 Oct 30;112(519):899-901. [CrossRef]
- Laber EB, Shedden K. Statistical Significance and the Dichotomization of Evidence: The Relevance of the ASA Statement on Statistical Significance and p-Values for Statisticians. J Am Stat Assoc 2017;112(519):902-904 [FREE Full text] [CrossRef] [Medline]
- Nuzzo R. Scientific method: statistical errors. Nature 2014 Feb 13;506(7487):150-152. [CrossRef] [Medline]
- McShane BB, Gal D. Statistical Significance and the Dichotomization of Evidence. Journal of the American Statistical Association 2017 Oct 30;112(519):885-895. [CrossRef]
- McShane BB, Gal D. Rejoinder: Statistical Significance and the Dichotomization of Evidence. Journal of the American Statistical Association 2017 Oct 30;112(519):904-908. [CrossRef]

#### Abbreviations

AMADEUS-1: Alcohol email assessment and feedback study dismantling effectiveness for university students |

AUDIT-C: Alcohol Use Disorders Identification Test |

eHealth: electronic health |

eSBI: electronic screening and brief intervention |

OR: odds ratio |

Edited by G Eysenbach; submitted 17.04.19; peer-reviewed by D Gratzer, B Smith; comments to author 17.06.19; revised version received 06.08.19; accepted 14.08.19; published 07.11.19

Copyright©Marcus Bendtsen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.11.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.