Pilot Study of a Multilevel Mobile Health App for Substance Use, Sexual Risk Behaviors, and Testing for Sexually Transmitted Infections and HIV Among Youth: Randomized Controlled Trial

Background Preventing and reducing substance use disorders, sexually transmitted infections (STIs)/HIV, and teen pregnancy, and the associated risk behaviors (ie, substance use and sexual risk behaviors) among youth remain public health priorities in the United States. Equally important is improving the uptake of STI/HIV testing among the youth. Mobile health (mHealth) apps may be a solution to ameliorate these public health concerns; however, few mHealth preventive interventions have demonstrated efficacy in reducing substance use or sexual risk behaviors or improving the uptake of STI/HIV testing among the youth, particularly in clinic settings. Objective This small-scale study aimed to examine the feasibility of conducting a pilot randomized controlled trial (RCT). We evaluated the effects of Storytelling 4 Empowerment (S4E), relative to enhanced usual practice, on the potential mechanisms by which behavior change occurs, namely clinician-youth risk communication, prevention knowledge, and substance use and sexual risk refusal self-efficacy. We also assessed the ability to measure targeted outcomes of past 30-day substance use (ie, alcohol, tobacco, and other drug use), condomless sex, and alcohol or drug use before sex, as well as the uptake of STI/HIV testing. Methods Employing community-based participatory research principles, 50 youths aged 13 to 21 years were recruited from a youth-centered community health clinic in Southeast Michigan, randomized sequentially to either S4E or enhanced usual practice, and assessed at baseline, immediately postintervention, and 30 days postintervention. S4E consists of 3 modules, including alcohol and drug use, tobacco, and STI/HIV. Results Relative to youth in the enhanced usual practice group, S4E participants demonstrated higher youth-clinician risk communication (mean 3.22, SD 1.67) and increases in prevention knowledge (∆ score mean 0.36, SD 0.51) and self-efficacy (∆ score mean 0.16, SD 0.47). In addition, youth in the S4E group showed reductions in the proportions of past 30-day overall substance use (Cohen h=0.71, 95% CI 0.15 to 1.27), as well as past 30-day alcohol (Cohen h=0.71, 95% CI 0.15 to 1.27), tobacco (Cohen h=0.17, 95% CI −0.39 to 0.73), and drug use (Cohen h=1.28, 95% CI 0.72 to 1.84). The results also suggest a reduction in the proportion of youths who reported past 30-day condomless sex (Cohen h=0.18, 95% CI −0.38 to 0.74) and alcohol use before sex (Cohen h=0.44, 95% CI −0.12 to 1.00). Finally, the findings also demonstrated an increase in the proportion of youths who reported STI/HIV testing over time (Cohen h=0.16, 95% CI −0.39 to 0.72). Conclusions The findings suggest the feasibility of a small-scale pilot RCT. S4E demonstrated shifts in the hypothesized direction, reducing substance use, sexual risk behaviors, and improving the uptake of STI/HIV testing among youth in a clinic setting. The findings suggest that a larger RCT may be warranted. Trial Registration ClinicalTrails.gov NCT03855410, https://clinicaltrials.gov/ct2/show/NCT03855410.

Is this a full powered effectiveness trial or a pilot/feasibility trial? * Manuscript tracking number * If this is a JMIR submission, please provide the manuscript tracking number under "other" (The ms tracking number can be found in the submission acknowledgement email, or when you login as author in JMIR. If the paper is already published in JMIR, then the ms tracking number is the four-digit number at the end of the DOI, to be found at the bottom of each published article in JMIR) ms #16251 You're editing your response. Sharing this URL allows others to also edit your response.

1a-i) Identify the mode of delivery in the title
Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "A Pilot Study of a Multilevel Mobile-Health Application for Substance Use, Sexual Risk Behaviors, and STI/HIV Testing among Youth: A Randomized Controlled Trial" You're editing your response. Sharing this URL allows others to also edit your response. 1a-ii) Non-web-based components or important co-interventions in title

FILL OUT A NEW RESPONSE
Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer 1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Substance Use; Sexual Risk Behaviors; STI/HIV Testing; Youth You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Employing community-based participatory principles, 50 youth aged 13-21 years were recruited from a youth-centered community health clinic in Southeast Michigan; randomized sequentially to either S4E or enhanced usual practice (for further information see manuscript, heading "Abstract")

1b-ii) Level of human involvement in the METHODS section of the ABSTRACT
Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) You're editing your response. Sharing this URL allows others to also edit your response. subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-ii?

FILL OUT A NEW RESPONSE
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials Does your paper address subitem 1b-iii? Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "50 youth aged 13-21 years were recruited from a youth-centered community health clinic in Southeast Michigan" You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The purpose of the present study was to pilot test the effects of S4E, relative to enhanced usual practice, in reducing substance use and sexual risk behaviors and improving uptake of STI/HIV testing over time among a diverse sample of youth in a clinic setting. (for further information see manuscript, heading "Introduction")

FILL OUT A NEW RESPONSE
You're editing your response. Sharing this URL allows others to also edit your response. FILL OUT A NEW RESPONSE 3a) Description of trial design (such as parallel, factorial) including allocation ratio 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This study consisted of a 2 (Group) x 2 (Time) pilot RCT. Youth participants were randomly assigned to either the S4E experimental group or enhanced usual practice control group via sequential randomization" (for further information see manuscript, heading "Methods") Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were no important changes to methods after trial commencement.

3b-i) Bug fixes, Downtimes, Content Changes
Bug fixes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug fixes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have influenced study design such as staff changes, system failures/downtimes, etc. [2].
You're editing your response. Sharing this URL allows others to also edit your response.

4a) Eligibility criteria for participants
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To be eligible for this study, youth had to: (1) be between 13-21 years of age, (2) live in Southeast Michigan without plans to move out of the area during the study period, (3) have a scheduled appointment with a participating clinician, and (4) report no prior history of psychiatric hospitalization."

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clarified.
Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer You're editing your response. Sharing this URL allows others to also edit your response.
FILL OUT A NEW RESPONSE 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these.
Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We recruited youth and clinicians between October 2016 and July 2017 from a youthcentered community health clinic located in Southeast Michigan that offers a full range of health care, mental health, and supportive services to young people as they transition to adulthood. Youth recruitment occurred during the clinic's health appointment reminder phone calls."

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
You're editing your response. Sharing this URL allows others to also edit your response.

FILL OUT A NEW RESPONSE
4b) Settings and locations where the data were collected subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Both youth and clinicians who expressed interest in participating in the study were contacted by the study team to screen for eligibility, to enroll, and to complete study consent protocols." Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We recruited youth and clinicians between October 2016 and July 2017 from a youthcentered community health clinic located in Southeast Michigan that offers a full range of health care, mental health, and supportive services to young people as they transition to adulthood."

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
You're editing your response. Sharing this URL allows others to also edit your response. subitem not at all important 1 2 3 4 5 essential 5) The interventions for each group with sufficient details to allow replication, including how and when they were actually administered Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Data were collected on tablets and captured using REDCap, a HIPAA-compliant, web-based application that is hosted on secure servers at the University of Michigan Medical School."

4b-ii) Report how institutional affiliations are displayed
Report how institutional affiliations are displayed to potential participants [on ehealth media], as affiliations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results) Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address subitem 5-i?

FILL OUT A NEW RESPONSE
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.
You're editing your response. Sharing this URL allows others to also edit your response. . This is accomplished through a multilevel mHealth application that provides interactive, targeted, and tailored content focused on the prevention of substance use and sexual risk behaviors." (for further information see manuscript, heading "Methods")

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
You're editing your response. Sharing this URL allows others to also edit your response.

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientific reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
You're editing your response. Sharing this URL allows others to also edit your response.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.

Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
You're editing your response. Sharing this URL allows others to also edit your response.

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

FILL OUT A NEW RESPONSE
No prompts/reminders were used. The application was used once during the youth participant clinician appointment and then at their followup appointment 30 days later.

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability. You're editing your response. Sharing this URL allows others to also edit your response. FILL OUT A NEW RESPONSE 6a) Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No co-interventions were provided in addition to the targeted m-health intervention. Youth were provided the tablet with the modules to view.
Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Youth's lifetime and past 30-day substance use behaviors were assessed using items adapted from the Monitoring the Future study [42,43]. Sample items include, "Have you ever had any beer, wine, wine cooler, or liquor to drink", and , "Have you had more than a few sips of alcohol on more than one occasion during the past 30 days?". Participant's lifetime and past 30-day sexual risk behaviors were assessed using items extracted from the Sexual Behavior Instrument [44]. "(for further information see manuscript, heading "Methods") 6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].
You're editing your response. Sharing this URL allows others to also edit your response. 6a-ii) Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored (logins, logfile analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.

Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text Your answer 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).
You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were no changes to trial outcomes after the trial commenced.

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.
You're editing your response. Sharing this URL allows others to also edit your response.

FILL OUT A NEW RESPONSE
7b) When applicable, explanation of any interim analyses and stopping guidelines 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group 8b) Type of randomisation; details of any restriction (such as blocking and block size) Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "This study consisted of a 2 (Group) x 2 (Time) pilot RCT. Youth participants were randomly assigned to either the S4E experimental group or enhanced usual practice control group via sequential randomization [40]." You're editing your response. Sharing this URL allows others to also edit your response. FILL OUT A NEW RESPONSE 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Sequential randomization was employed, there were no restrictions.
Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Mechanism used to implement the sequential randomization was a sequential treatment assignment system hosted by the University of Michigan servers. http://newrandtest.appspot.com/ Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Random allocation sequence was generated from the sequential treatment assignment system, and trained research assistants enrolled participants and assigned participant to the intervention.
You're editing your response. Sharing this URL allows others to also edit your response.

FILL OUT A NEW RESPONSE
11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

Does your paper address subitem 11a-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No study participants were blinded. Study team members conducting the data analysis analyzed de identified data.
11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".
You're editing your response. Sharing this URL allows others to also edit your response.

11b) If relevant, description of the similarity of interventions
(this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This item is not relevant. The comparator was an enhanced usual care.
You're editing your response. Sharing this URL allows others to also edit your response.

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not use imputation techniques to address attrition/missing values. Intent to treat approach was taken, and we retained 98% (49/50) of the sample.
You're editing your response. Sharing this URL allows others to also edit your response.

X26) REB/IRB Approval and Ethical Considerations [recommended as
subheading under "Methods"] (not a CONSORT item) subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Given the pilot nature of our study, we did not conduct a subgroup analysis.

X26-i) Comment on ethics committee approval
Does your paper address subitem X26-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "All study procedures were approved by the University of Michigan Institutional Review Board (HUM00097290). A Certificate of Confidentiality (CC-CA-17-048-AI) was obtained from the National Institutes of Health/National Cancer Institute. This study is registered with ClinicalTrails.gov (NCT03855410)." You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address subitem X26-ii?

FILL OUT A NEW RESPONSE
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To protect confidentiality for youth between 13-17 years of age, a waiver of parental permission was obtained. This waiver was in accordance with the state of Michigan's Title X and Public Health Code, MCL 333.6121, which states that a minor 17 years of age or younger can consent to sexual and reproductive health and substance use services without parental knowledge."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer You're editing your response. Sharing this URL allows others to also edit your response.

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "In line with other research pilot testing mHealth preventive interventions [24,26], the sample size for the present study was 50, 25 youth participants randomized to the S4E group and 25 randomized to the Enhanced Usual Care group." Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT flow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We lost one participant because they moved out of the study area and were no longer receiving care at the study site.
You're editing your response. Sharing this URL allows others to also edit your response. subitem not at all important 1 2 3 4 5 essential 14a) Dates defining the periods of recruitment and follow-up

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement.

Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We included a Consort Diagram that demonstrates both recruitment process and attrition at follow up.
Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We recruited youth and clinicians between October 2016 and July 2017 You're editing your response. Sharing this URL allows others to also edit your response. 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., significant changes in Internet resources available or "changes in computer hardware or Internet delivery resources"

FILL OUT A NEW RESPONSE
Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " In line with other research pilot testing mHealth preventive interventions , the sample size for the present study was 50." You're editing your response. Sharing this URL allows others to also edit your response.

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Unknown. We provided tablets to participants.
You're editing your response. Sharing this URL allows others to also edit your response. 16-i) Report multiple "denominators" and provide definitions Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention.

FILL OUT A NEW RESPONSE
Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "As shown in Table 1, chi-square tests and analyses of variance results suggest no significant S4E vs. Control group differences at baseline on any demographic characteristic (e.g., race), lifetime or past 30-days substance use, sexual risk behaviors, and lifetime STI/HIV testing. The absence of significant differences in these variables at baseline suggests that our trial's randomization procedures were successful. (for further information see manuscript, heading "Results")

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
You're editing your response. Sharing this URL allows others to also edit your response. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Overall, participant reports of substance use at baseline were not significantly different. However, the S4E group reported a greater reduction in any substance use (i.e., ATOD) relative to the control group (3/25, 12% vs. 0/25, 0%). Although chi-square testing (χ22=4.5, P=.10) suggests that these proportion differences between groups were not significant, the estimated proportion change effect size difference between groups (Cohen h=0.71) yielded a large effect size. (for further information see manuscript, heading "Results")

17a-i) Presentation of process outcomes such as metrics of use and intensity of use
In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a).
You're editing your response. Sharing this URL allows others to also edit your response.

FILL OUT A NEW RESPONSE
17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "As shown in Table 3, of the 25 youth in the S4E group, 12% (3/25) reported decrease of any substance use from baseline assessment to 30-day follow-up (χ21=10.97, p < .001, Cohen h =0.24). We then decomposed past 30-day substance use into past 30-day alcohol, tobacco, and other drug use to determine within-group intervention effects on each of these outcomes. (for further information see manuscript, heading "Results")

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).
Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
You're editing your response. Sharing this URL allows others to also edit your response. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer 22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use)

FILL OUT A NEW RESPONSE
Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).

Does your paper address subitem 22-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Evaluation of the S4E intervention suggests effects in reducing overall licit and illicit substance use behaviors among youth. When we decomposed substance use behaviors, S4E helped decrease the proportion of youth who engaged in alcohol, tobacco, or other drug use at 30-day follow-up.(for further information see manuscript, heading "Discussion") You're editing your response. Sharing this URL allows others to also edit your response.
FILL OUT A NEW RESPONSE Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "It is also important to note that some researchers have found that technology-based interventions have a moderate health impact on exercise over 6 weeks. Thus, in general, an important future research direction is to examine both the short and long-term effects of mHealth preventive interventions.(for further information see manuscript, heading "Discussion")

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
You're editing your response. Sharing this URL allows others to also edit your response. Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Several limitations are worth noting. First, the sample in the present study is not representative of the clinic population, limiting the generalizability of our findings. However, we demonstrated hypothesized shifts on prominent substance use, sexual risk behaviors, and STI/HIV testing in S4E youth, the majority of whom identified as racial and ethnic minorities.(for further information see manuscript, heading "Discussion")

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study ClinicalTrials.gov Identifier: NCT03855410 You're editing your response. Sharing this URL allows others to also edit your response. Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.

Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study ClinicalTrails.gov (NCT03855410) You're editing your response. Sharing this URL allows others to also edit your response. subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Trial protocol can be access at clinicaltrials.org Does your paper address CONSORT subitem 25? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

X27-i) State the relation of the study team towards the system being evaluated
In addition to the usual declaration of interests (financial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
You're editing your response. Sharing this URL allows others to also edit your response.