Effects of ACT Out! Social Issue Theater on Social-Emotional Competence and Bullying in Youth and Adolescents: Cluster Randomized Controlled Trial

Background Schools increasingly prioritize social-emotional competence and bullying and cyberbullying prevention, so the development of novel, low-cost, and high-yield programs addressing these topics is important. Further, rigorous assessment of interventions prior to widespread dissemination is crucial. Objective This study assesses the effectiveness and implementation fidelity of the ACT Out! Social Issue Theater program, a 1-hour psychodramatic intervention by professional actors; it also measures students’ receptiveness to the intervention. Methods This study is a 2-arm cluster randomized control trial with 1:1 allocation that randomized either to the ACT Out! intervention or control (treatment as usual) at the classroom level (n=76 classrooms in 12 schools across 5 counties in Indiana, comprised of 1571 students at pretest in fourth, seventh, and tenth grades). The primary outcomes were self-reported social-emotional competence, bullying perpetration, and bullying victimization; the secondary outcomes were receptiveness to the intervention, implementation fidelity (independent observer observation), and prespecified subanalyses of social-emotional competence for seventh- and tenth-grade students. All outcomes were collected at baseline and 2-week posttest, with planned 3-months posttest data collection prevented due to the COVID-19 pandemic. Results Intervention fidelity was uniformly excellent (>96% adherence), and students were highly receptive to the program. However, trial results did not support the hypothesis that the intervention would increase participants’ social-emotional competence. The intervention’s impact on bullying was complicated to interpret and included some evidence of small interaction effects (reduced cyberbullying victimization and increased physical bullying perpetration). Additionally, pooled within-group reductions were also observed and discussed but were not appropriate for causal attribution. Conclusions This study found no superiority for a 1-hour ACT Out! intervention compared to treatment as usual for social-emotional competence or offline bullying, but some evidence of a small effect for cyberbullying. On the basis of these results and the within-group effects, as a next step, we encourage research into whether the ACT Out! intervention may engender a bystander effect not amenable to randomization by classroom. Therefore, we recommend a larger trial of the ACT Out! intervention that focuses specifically on cyberbullying, measures bystander behavior, is randomized by school, and is controlled for extant bullying prevention efforts at each school. Trial Registration Clinicaltrials.gov NCT04097496; https://clinicaltrials.gov/ct2/show/NCT04097496 International Registered Report Identifier (IRRID) RR2-10.2196/17900


1a-i) Identify the mode of delivery in the title
Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The ACT Out! **Social Issue Theater** Cluster Randomized Trial: Short-term Outcomes" subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 1a-ii) Non-web-based components or important co-interventions in title Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (though the outcome relates to cyberbullying, the intervention itself is fully o ine)

1a-iii) Primary condition or target group in the title
Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No, but it is in the rst line of the abstract. I will edit if requested by journal. Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The comparator was treatment as usual. The intervention is described as such in the Abstract: "This study assessed the effectiveness and implementation delity of the ACT Out! Social Issue Theater program, a one-hour psychodramatic intervention by professional actors; it also measured students' receptiveness to the intervention." 1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (in-person intervention) 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional offline trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Yes, via the following content in the Abstract: "This study was a two-arm cluster randomized control trial with 1:1 allocation that randomized at the classroom level (n = 76 classrooms in 12 schools across ve counties in Indiana, comprised of n = 1,571 students at pretest in 4th, 7th, and 10th grades). The primary outcomes were self-reported social-emotional competence, bullying perpetration, and bullying victimization, and the secondary outcomes were receptiveness to the intervention, implementation delity (independent observer observation), and prespeci ed subanalyses of social-emotional competence for 7th and 10th grade students. All outcomes were collected at baseline and two-week posttest, with planned threemonth posttest data collection being prevented by the COVID-19 pandemic." 1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Partially not applicable -due to the type of trial, this was reported for the prior subitem. For other results: "Intervention delity was uniformly excellent (> 96% adherence), and students were highly receptive to the program. However, trial results did not support the hypothesis that the intervention would increase participants' social-emotional competence. The intervention's impact on bullying was complicated to interpret and included some evidence of small interaction effects (reduced cyberbullying victimization and increased physical bullying perpetration). However, there were also statistically and clinically signi cant reductions in physical, verbal, and relational bullying victimization in the full sample (pooled across both study arms). While not appropriate for causal attribution, the occurrence of these effects across 12 schools in different counties, with 76 classrooms, within a span of two weeks, by chance, was implausible, though we cannot state with certainty why these effects manifested."

1b-v) CONCLUSIONS/DISCUSSION in abstract for negative trials
Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

INTRODUCTION 2a) In INTRODUCTION: Scientific background and explanation of rationale
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-v?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This was a mixed results trial, so conclusion suggests potentially why, and next steps: "We conceptually discuss several potential alternative explanations. As a next step, we encourage research into whether the ACT Out! intervention may engender a bystander effect not amenable to randomization by classroom. We therefore recommend a larger trial of the ACT Out! intervention that is focused speci cally on cyberbullying, measures bystander behavior, is randomized by school, and is controlled for extant bullying prevention efforts at each school."

2a-i) Problem and the type of system/solution
Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program? Intended for a particular patient population? Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions? (Note: Details about the intervention are provided in "Methods" under 5) The intervention is intended to reduce cyberbullying/bullying perpetration and victimization, and to affect social-emotional competence.
The rationale is: "Given the di culty in addressing school-based bullying with lengthy and multipartite curricula, one might wonder why a short (one-hour) dramatic performance would be hypothesized to have even a short-term effect on SEC or bullying... ...ACT Out! Social Issue Theater is different than each of these examples because it uses trained, professional actors and requires no involvement from schools outside of planning the visit (when implemented outside of a study).
We were unable to nd a precedent for this intervention structure in the literature." 2a-ii) Scientific background, rationale: What is known about the (type of) system Scientific background, rationale: What is known about the (type of) system that is the object of the study (be sure to discuss the use of similar systems for other conditions/diagnoses, if appropiate), motivation for the study, i.e. what are the reasons for and what is the context for this specific study, from which stakeholder viewpoint is the study performed, potential impact of findings [2]. Briefly justify the choice of the comparator.

2b) In INTRODUCTION: Specific objectives or hypotheses
Does your paper address subitem 2a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study SEL: "There have been numerous SEL programs implemented and evaluated in schools in recent years; a summary of over 300 studies contained in four metaanalyses identi ed generally positive short-term outcomes [7] across multiple domains (e.g., substance use). In general, performances by the ACT Out! Ensemble are structured in such a way that they model aspects of SEL, such as healthy relationships, regardless of the additional topic being addressed (e.g., bullying)." Bullying: "A recent meta-analysis of traditional bullying identi ed 65 school-based bullying prevention programs, but only eight had been evaluated more than once [13]. In general, such programs tend to be slightly more effective in reducing bullying perpetration and less effective in reducing victimization." Psychodrama: "A small body of literature has examined psychodrama as a prevention or behavior change mechanism in youth, but these studies have covered diverse behaviors (e.g., [25]), have involved multiple separate components such as teacher training (e.g., [26]), and/or have used students or school employees rather than professional actors as dramatis personae (e.g., [27])" Choice of comparator was justi ed in the published protocol paper.

METHODS
3a) Description of trial design (such as parallel, factorial) including allocation ratio Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The present study primarily aimed to assess whether a one-hour exposure to ACT Out! Social Issue Theater was superior to treatment as usual (TAU) in developing SEC and reducing bullying (both bullying behavior and victimization) in elementary, middle, and high school students at two-week posttest. The study secondarily aimed to determine whether the same intervention was superior to TAU in developing speci c sub-domains of SEC (social awareness, emotion regulation, relationship skills, and responsible decision making) among middle and high school students at two-week posttest. Finally, the study also aimed to assess student receptivity to ACT Out! Social Issue Theater using previously validated measures indicating student agreement with positive (e.g., "enjoyable") and negative (e.g., "boring") adjectives. Additional details are available in the trial protocol [2]." 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The ACT Out! trial was a proof-of-concept cluster randomized superiority trial with two groups and 1:1 allocation. The unit of measurement was individual students, but the unit of randomization was the classroom, strati ed by school (with one exception, see Attachment 1). For each school, half of the classrooms were randomly assigned to the intervention arm and half to the control arm. Schools with an odd number of classrooms had a single classroom randomly selected for exclusion (though if the school requested, that classroom was permitted to complete the survey for appearance's sake, and the results were then discarded by the study team). " Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study All deviations from the methods are documented either within the manuscript or in the attachment that documents smaller deviations. Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.
Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The ACT Out! trial was conducted among 12 public and charter schools in Marion (n=4 schools), Ripley (n=3), Boone (n=2), Lawrence (n=2), and Monroe (n=1) Counties in Indiana. For reasons described in the protocol, clusters were selected only from grades four, seven, and ten [2]. All students in the selected classrooms and schools were eligible to participate... Schools were selected based on their willingness to participate in the project as described, which included classroomlevel randomization and inclusion of all eligible classrooms in the study's allocation processes."

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clarified.
Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.
4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these. Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Once a school enrolled in the study, each classroom was provided with a study packet containing surveys and response forms, a manila envelope, a white envelope, and an administrator checklist (see supplements to [2]). Each classroom was also assigned a unique code consisting of the grade level, study arm, and a randomly generated cluster ID. This code was pre-lled on the back of each survey form as well as on the front of each envelope to facilitate data quality control. Classroom teachers administered the surveys by following the step-by-step administrator checklist. Surveys that were handed out to students were placed back in the manila envelope, regardless of whether they were completed, while extra surveys were placed in the white envelope (unused). The pretest was completed between zero and three days prior to the intervention, depending on school schedules and availability. The posttest was completed between 14 and 27 days after the intervention, with most classrooms completing the posttest within 14 to 17 days, depending on school schedules and ability to facilitate the posttest. "

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
4b) Settings and locations where the data were collected subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study All documents provided during recruitment are published as Attachments to this study or as Attachments to the protocol paper.

Does your paper address CONSORT subitem 4b? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See responses within 4a.

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See responses within 4a. Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, except on the Informed Consent sheet.

5-i) Mention names, credential, affiliations of the developers, sponsors, and owners
Mention names, credential, affiliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conflict of interest" section or mentioned elsewhere in the manuscript). subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The ACT Out Ensemble was founded in 1995 and is currently operated by Claude McNeal Productions (CMP), a professional theater troupe incorporated as a notfor-pro t." This study was funded by a subaward from Claude McNeal Productions to produce an independent assessment of their program's e cacy (see Con icts of Interest for extra steps taken to rewall).

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.

Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Our decision to analyze this intervention was based not only on our a priori understanding of the value such a brief intervention might yield, but also on the remarkable community-and school-level support for the program. Prior to this study, more than 500,000 individuals had viewed a performance by the ACT Out

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (e.g., content is described in the Attachments).

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Data were collected using a customized form created with Scantron Design Expert and scanned directly into a database with an Insight 700c scanner to avoid data entry errors. However, one of the survey matching elements required a handwritten response; these were typed manually into the database. To verify intervention delity, at least one individual who was not a member of the ACT Out! ensemble attended every performance and documented the concordance between a prespeci ed checklist of elements for the intervention and the performance itself. These checklists were developed separately for each grade (since the scenarios vary) and are available as Attachments 3 through 5. To establish coding reliability, a second individual attended performances for six clusters to conduct delity checks, and interrater reliability was computed (see Attachment 1)." 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientific reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See Attachments. All data and analytic code are provided.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (not an online intervention). Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The intervention was a psychodramatic, improvisational performance that was delivered to classrooms (separately, except in one case where two small classrooms attended together) by members of the CMP professional theater company. Interventions were scheduled to last approximately one hour and were delivered during the school day. Each one-hour performance consisted of ve vignettes focused on bullying and cyberbullying and was designed to be interactive, meaning that after each scenario, the student audience was invited to converse with the performers while they remained in character. In each case, a moderator from CMP also managed the overall performance (e.g., calling on students to ask questions of the characters). While the scenarios were improvisational in nature, they were designed to remain true to core concepts that were prespeci ed and agreed upon by CMP and the research team (e.g., the identity of the characters, the nature of the con ict, and methods of bullying). To ensure this, delity data were captured from all performances and are described in the Quality Control subsection. The written speci cations for each vignette, by grade level, are provided as Attachment 2." Additional theoretical rationale is included in the Introduction.

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (not an online intervention).

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (not an online intervention). Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable (not an online intervention).

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability. Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No co-interventions occurred. Does your paper address subitem 6a-i?
Copy and paste relevant sections from manuscript text Not applicable (not an online intervention).
6a-ii) Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored (logins, logfile analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text Not applicable (not an online intervention) subitem not at all important 1 2 3 4 5 essential 6b) Any changes to trial outcomes after the trial commenced, with reasons 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text No qualitative data were collected, though receptivity was measured as a secondary outcome.
Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No changes to trial outcomes were made, except for outcomes not collected because schools shut down for COVID-19 before the nal wave of data collection. subitem not at all important 1 2 3 4 5 essential 7b) When applicable, explanation of any interim analyses and stopping guidelines 7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size Describe whether and how expected attrition was taken into account when calculating the sample size.

Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The rationale for choices made in preparing the sample size calculation is provided in the protocol [2]. We estimated the sample size required to detect a moderate effect (Cohen's d = 0.30), with two-sided signi cance of .05, and a power of .80 to be 340 participants. We estimated an intra-class correlation (ICC) of 0.153 based on a prior school-based cluster study on cigarette smoking with a similar methodology (as described in [2]), and assumed approximately 20 students per classroom, yielding a design effect of 3.907. We took the resultant estimate of 1,328 students and adjusted for an expected 20% loss due to survey matching procedures, producing the nal sample size target of 1,594 students across approximately 80 classrooms." Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We addressed the reason for lack of these in the protocol paper.
8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group 8b) Type of randomisation; details of any restriction (such as blocking and block size) 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Your answer
Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Simple randomization occurred at the cluster level using a smartphone app produced by Random.org [32]. Randomization of clusters occurred within schools with a 1:1 allocation. In the speci c instance where clusters rst had to be created (a single school), the website version of Random.org was used to randomly assign students to evenly sized clusters, and then to assign clusters to study arm." https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2… 36/59 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " Since generation of the allocation sequence was computerized, it was concealed to all members of the research team until the moment of assignment. Because of needs driven by school planning, there was some variability in the generation and assignment process. Decisions were made as follows: -Schools that agreed to participate were asked to identify all clusters within the selected grade level (e.g., fourth, seventh, or tenth).
-If schools were willing and able to accommodate it, allocation sequences were generated by the delity checker immediately prior to the intervention (e.g., on-site, in the schools). However, most schools (10 of 12) were unable to accommodate this method.
-Subsequently, most schools were asked to list classrooms by a xed characteristic (e.g., time the homeroom met, teacher's name) at the time of school enrollment. One of the researchers generated a random sequence, applied it directly to classrooms, and shared the sequence with school administrators, identifying which classrooms would be allocated to which arm, and asked the administrators not to share this information with teachers until necessary for planning efforts. " Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See prior response (item 9). 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

Does your paper address subitem 11a-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Due to the nature of this study, blinding of participants, school o cials, and researchers was not feasible. However, multiple independent statisticians were involved in conducting and reviewing analyses, and some were blinded to the meaning of study arm coding." 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

11b) If relevant, description of the similarity of interventions
(this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The comparator was treatment as usual.
Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable. subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "All outcomes were continuous, so linear mixed models (LMM) using restricted maximum likelihood (REML) were tted for each analysis (SAS PROC MIXED) with repeated measures for each participant. Time of survey administration (pretest or posttest), study arm, and the interaction of time and study arm were treated as xed effects. The interaction of time by study arm was the hypothesis test for causal effects (e.g., intervention group improved signi cantly more than the control group) All analyses allowed for clustering of students within schools and classrooms as random intercepts to alleviate the issue of in ated standard errors and used Kenward-Rogers degrees of freedom approximation to account for the cluster randomized trial design. P-values were two-sided and treated as signi cant at 0.05 or less; however, in keeping with recommendations from the American Statistical Society, we did not use p-values as the sole determinant of outcome importance. We instead provided the full dataset and analytic code, and interpreted the output based on a combination of effect size, clinical signi cance, standard errors, and signi cance [36]. Similarly, we produced four sets of output, the intention-to-treat (ITT) analysis, in which all data were analyzed in the arm to which they were randomized (with and without MI; 1,537 pretest and 1,209 posttest), and the per protocol (PP) analysis, in which only data resulting from a completed protocol were analyzed (with and without MI; 1,184 pretest and posttest), in accordance with reporting recommendations (e.g., [37]) (see Figure 1)"

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).

12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses X26) REB/IRB Approval and Ethical Considerations [recommended as
subheading under "Methods"] (not a CONSORT item) Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Multiple imputation using the Markov Chain Monte Carlo (MCMC) approach was completed using SAS 9.4 (SAS Inc., Cary NC) using PROC MI and MIANALYZE with the assumption that data were missing at random (MAR). All variables that were collected were imputed for all analyses. Numbers of iterations were based on missingness in the per-protocol (PP) analysis, which had 1,184 pretests and posttests, and thus 2,368 surveys. Percent missingness ranged from a low of 1.22% to a high of 8.57%. SEC items not asked of 4th grade students were present on 2,078 surveys (removing 145 4th grade participants). Percent missingness within those variables ranged from 2.84% to 4.96%. Given this information, we selected 10 imputations for our analyses (integer greater than the missingness in the variable with the highest level of missingness, 10 > 8.57 [35]). Bias was also mitigated by presenting outcomes from four approaches (see Statistical Analysis)." Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Secondary outcomes were tested using the same mechanisms as primary outcomes. subitem not at all important 1 2 3 4 5 essential Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Authorizing o cials for schools or school corporations were required to provide a signed letter of agreement prior to participating in the study. At the individual level, the project used a waiver of parental consent (opt out), as approved by the institutional review board (IRB). Parents and legal guardians were permitted to review study procedures and were provided with a description of the study a minimum of two weeks prior to any individual-level interaction with subjects, along with instructions for how to opt out; students, their parents, and their guardians all had the ability to opt a student out from participating either formally or by survey noncompletion. The rationale for this approach was a combination of the low risk posed by the study as well as the desire to avoid unintentional exclusion of underrepresented minorities and high-risk populations, as described in the protocol [2]."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline)

Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No harm was anticipated from participation, but instructions, including participant rights, were printed on each survey instrument (see Attachments).

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Initially, a total of 13 schools from across Indiana participated in the trial, comprising 84 classrooms for the eligible grade levels. Of those classrooms, 42 were randomized to the intervention arm, and 42 were randomized to the control arm. One school did not follow protocol and failed to correctly administer pretest surveys to either arm prior to intervention delivery, so it was summarily removed from the trial (n = 4 classrooms). Three schools provided incorrect counts of classrooms to be randomized (n = 3 control, and n = 1 intervention classrooms), so sequences were generated that included classrooms that did not exist. On discovery of this discrepancy, sequences and assignments were not altered because it would have affected allocation concealment. Thus, despite 1:1 allocation, the number of baseline classrooms was 76 (n = 37 control, n = 39 intervention). Finally, an additional 11 classrooms (n = 5 control, and n = 6 intervention) at a single school were slightly delayed in completing posttests, and then schools were shut down for the academic year due to COVID-19 prior to data collection. Thus, the classroom count completing posttests was 65 (n = 32 control, and n = 33 intervention). Table 1 shows the baseline characteristics of the two trial arms for all students who provided data, excluding blank surveys. " Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Shown in CONSORT diagram.

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement.

Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study CONSORT diagram indicates attrition. 14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The formal study start date was October 16, 2019, and interventions were delivered from November 6, 2019 to February 28th, 2020. School recruitment was terminated in February once the anticipated numbers of clusters and participants reached the planned total." 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., significant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We note the substantial impact of COVID-19, which resulted in 11 lost classrooms for wave 2 data collection, and the complete elimination of wave 3 data collection. Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Yes -COVID-19.

15) A
Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Available as Table 1.

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.

16-i) Report multiple "denominators" and provide definitions
Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention.
Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See Tables 2 and 3.

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Primary analysis was intent-to-treat but we consider and provide results from ITT (with and without MI) and per-protocol (with and without MI) to illustrate the range of outcomes -some borderline signi cance was observed (as the sample was smaller than planned due to COVID-19) and we felt this was the most transparent way to describe the outcomes.
Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See Tables 2 and 3 (we provide estimated means and standard errors, as well as pvalues for differences within arms and for the interaction).

17a-i) Presentation of process outcomes such as metrics of use and intensity of use
In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a).
17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.
Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.
Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No analyses were performed that were not pre-speci ed.

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).
Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study See our response above regarding ITT and per-protocol (because almost all attrition was due to COVID-19 in a "batch" and not for other reasons, we reported both analysis types).
Does your paper address CONSORT subitem 19? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "There was also limited evidence of a small effect of similar magnitude for increased physical bullying perpetration via the interaction effects. Does your paper address subitem 19-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.

19-ii) Include qualitative feedback from participants or observations from staff/researchers
Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.

Does your paper address subitem 19-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We extensively describe our unexpected ndings in the Discussion, and suggest appropriate next steps given a balanced interpretation of the ndings in the Conclusion. Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).

Does your paper address subitem 22-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The Discussion does this extensively, but the response is far too long due to nuances to include as a quote here. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Given our post-hoc assessment of the intervention, a rigorous follow-up study with a new sample, addressing issues related to interpretability of bullying victimization data, including measures of possible bystander effects and randomization at the school level while -if feasible -controlling for ongoing and recent bullying prevention programs, would be valuable. In doing so, scenario emphasis might also reasonably be shifted more toward cyberbullying and away from physical bullying, for which potential iatrogenic effects in perpetration were computed in one of the models, though overall physical victimization declined. Additional data should also be collected on sustainability of the effects beyond two weeks (barring another event like the COVID-19 pandemic) as well as whether there is a dose-response relationship (e.g., would two performances within a semester more strongly reduce victimization?)...

22-ii) Highlight unanswered new questions, suggest future research
On the practical side, given the high intervention delity and student receptiveness, and preliminary evidence related to cyberbullying victimization, it would not be unreasonable for CMP to offer a performance of scenarios focused on cyberbullying prevention, not as a replacement for extant bullying prevention programming, but as a supplement to it. In practice (e.g., outside of a trial) this intervention has comparatively low scal cost (so high scalability), only an hour of time is utilized, it requires no teacher time or preparation, and it may have some bene ts. Cyberbullying victimization is indeed a "tough needle to move" and so we believe that further research into use of skilled psychodramatic performance in prevention programming is warranted."

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2… 55/59 subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "This study was truncated unexpectedly by the COVID-19 pandemic, which had the effect of preventing planned three-month outcome data collection and moderately affecting the number of clusters available for analysis for short-term outcomes (loss of 11 clusters). There were also several unplanned deviations from the study protocol, each of which has been documented in Attachment 1. Participating schools were from both urban and rural counties in Indiana, and student participants were generally more diverse than the population of Indiana, as a whole, but some caution should be used when generalizing these ndings outside of the participating schools -especially since participating schools were those that volunteered to participate in a randomized trial. Further, given sample proportions, the results more readily can be generalized to middle and high school students than elementary school students. The study also had several notable strengths, including prespeci cation of all analyses, use of multiple objective external Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable -this was a practical RCT implemented in routine settings.
Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "This study was prospectively registered with Clinicaltrials.gov (NCT04097496)" Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The study protocol is available at https://doi.org/10.2196/17900."

X27-i) State the relation of the study team towards the system being evaluated
In addition to the usual declaration of interests (financial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Claude McNeal Productions and their representatives own the rights to the ACT Out! Social Issue Theater program. No one from that organization was involved in preparing the study protocol, interpreting ndings, conducting analyses, or writing this manuscript -both as a matter of practice, and per written agreement in the subaward to Prevention Insights." yes, major changes yes, minor changes no yes no Other: As a result of using this checklist, did you make changes in your manuscript? * What were the most important changes you made as a result of using this checklist?
N/A -the manuscript was written alongside a CONSORT checklist, but we just highlighted elements on a printed sheet when they were included.
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript *

minutes
As a result of using this checklist, do you think your manuscript has improved? * n/a STOP -Save this form as PDF before you click submit To generate a record that you filled in this form, we recommend to generate a PDF of this page (on a Mac, simply select "print" and then select "print as PDF") before you submit it.
When you submit your (revised) paper to JMIR, please upload the PDF as supplementary file.
Don't worry if some text in the textboxes is cut off, as we still have the complete information in our database. Thank you!