This paper is in the following e-collection/theme issue:

Games and Gamification for Health (609) Serious Games for Health and Medicine (644) HIV/AIDS/STI Prevention and Care (819)

Published on in Vol 17, No 10 (2015): October

Using Videogame Apps to Assess Gains in Adolescents’ Substance Use Knowledge: New Opportunities for Evaluating Intervention Exposure and Content Mastery

Using Videogame Apps to Assess Gains in Adolescents’ Substance Use Knowledge: New Opportunities for Evaluating Intervention Exposure and Content Mastery

Using Videogame Apps to Assess Gains in Adolescents’ Substance Use Knowledge: New Opportunities for Evaluating Intervention Exposure and Content Mastery

Authors of this article:

Erika Montanaro1 Author Orcid Image ;   Lynn E Fiellin2 Author Orcid Image ;   Tamer Fakhouri3 Author Orcid Image ;   Tassos C Kyriakides4 Author Orcid Image ;   Lindsay R Duncan5 Author Orcid Image
Article Authors Cited by (24) Tweetations (14) Metrics

    Original Paper

    1Yale University, New Haven, CT, United States

    2play2PREVENT Lab at Yale, Yale School of Medicine, New Haven, CT, United States

    3One Medical Group, San Francisco, CA, United States

    4Yale School of Public Health, New Haven, CT, United States

    5play2PREVENT Lab at Yale, McGill University, Montreal, QC, Canada

    *all authors contributed equally

    Corresponding Author:

    Erika Montanaro, PhD (HK), M Paed De

    Yale University

    Yale School of Public Health, 135 College Street, Suite 200

    New Haven, CT, 06510

    United States

    Phone: 1 203 764 4333

    Fax:1 203 764 4353

    Email: erika.montanaro@yale.edu


    Background: Videogame interventions are becoming increasingly popular as a means to engage people in behavioral interventions; however, strategies for examining data from such interventions have not been developed.

    Objective: The objective of this study was to describe how a technology-based intervention can yield meaningful, objective evidence of intervention exposure within a behavioral intervention. This study demonstrates the analysis of automatic log files, created by software from a videogame intervention, that catalog game play and, as proof of concept, the association of these data with changes in substance use knowledge as documented with standardized assessments.

    Methods: We analyzed 3- and 6-month follow-up data from 166 participants enrolled in a randomized controlled trial evaluating a videogame intervention, PlayForward: Elm City Stories (PlayForward). PlayForward is a videogame developed as a risk reduction and prevention program targeting HIV risk behaviors (substance use and sex) in young minority adolescents. Log files were analyzed to extract the total amount of time spent playing the videogame intervention and the total number of game levels completed and beaten by each player.

    Results: Completing and beating more of the game levels, and not total game play time, was related to higher substance use knowledge scores at the 3- (P=.001) and 6-month (P=.001) follow-ups.

    Conclusions: Our findings highlight the potential contributions a videogame intervention can make to the study of health behavior change. Specifically, the use of objective data collected during game play can address challenges in traditional human-delivered behavioral interventions.

    Trial Registration: Clinicaltrials.gov NCT01666496; https://clinicaltrials.gov/ct2/show/NCT01666496 (Archived by WebCite at http://www.webcitation.org/6cV9fxsOg)

    J Med Internet Res 2015;17(10):e245

    doi:10.2196/jmir.4377

    Keywords

    video games; intervention studies; substance use; HIV; evaluation; eHealth 


    In recent decades, eHealth interventions, defined as the use of information technology in the promotion, prevention, treatment, and maintenance of health care [1], have emerged as a new and compelling form of intervention that can reach all ages and ethnic groups [2]. eHealth interventions are a promising form of intervention delivery because they (1) have the potential to reach wider audiences who are not motivated to use in-person approaches, (2) have lower delivery costs than human-delivered interventions, (3) allow for the standardization of delivery content, and (4) offer multiple dissemination channels (eg, smartphones, videogames) [2,3]. Videogames, in particular, are emerging as an effective platform for the delivery of behavioral health interventions [4-6]. For example, Kato et al [7] developed a videogame intervention to address issues related to cancer care and treatment for patients between the ages of 13 and 28 years. After playing either the intervention game or a control game for 1 hour per week for 3 months, patients in the intervention condition had significantly higher self-efficacy and knowledge related to their treatment and management of symptoms as well as greater adherence to medications [7]. This intervention used only videogame play time as an indication of exposure to the intervention. Videogames allow for an interactive experience in a virtual environment, which affords players the opportunity to experience the full spectrum of outcomes related to different choices without actually having to live through the potentially harmful consequences (eg, HIV infection) [8]. Theory-based interventions (ie, those that target and change psychological constructs) are more effective at producing behavior change [9] than non-theory-based interventions and the advancement of videogame interventions and technology in general allows researchers to better understand the entire process.

    Active Ingredients of Change

    Interventions designed from health behavior theory are more successful than those that are not theory-based [9,10]. Behavior change interventions are often complex, targeting multiple theoretical constructs in hopes of changing behavior [11]. The complex framework in which many interventions are built means there are a variety of factors being intervened on at once; thus, a limitation of conventional behavior change interventions is the ability to accurately catalog exposure to a given intervention component. Because behavioral interventions are now delivered via emerging technologies, opportunities to improve on traditional assessments of intervention implementation are being established [12]. Indeed, for more than a decade, researchers have been encouraged to think creatively to establish systems for gathering data through these eHealth interventions to explore the active ingredients of change [13,14]. Videogame-based interventions provide a concrete example of how innovative techniques to measure exposure can be implemented through eHealth interventions.

    Importance of Implementation Guides

    Videogames have the advantage of providing highly accurate and granular exposure to information through the analysis of each player’s unique game play and decisions. Videogame software can create automatic logs with timestamps of all aspects of a player’s experience (eg, button presses, game levels entered and exited, actions taken) to a level of detail allowing for the recreation of the entire game play session. New avenues for data collection using videogames, and broadly all Internet-delivered interventions, are emerging [11]. Implementation guides for these new assessment methods, such as using event log files, are needed to catalog these techniques in a purposeful, coherent, and understandable way to advance the field [15]. These guides will allow for the level of detail needed to create new research questions regarding game play interactions, participant experience, and process evaluation.

    Substance Use

    Adolescents use substances such as alcohol and marijuana. In 2011, 70% of students in grade 12 reported having tried alcohol and 50% reported being drunk at least once [16]. Marijuana is another substance used frequently in adolescence with 46% of those in grade 12 reporting using marijuana at least once in their lifetime [16]. Substance use is often associated with unprotected sexual activities [17,18] and an increased number of accidental deaths among adolescents [19]. However, it is unclear if adolescents are knowledgeable about the negative consequences related to substance use. It is critically important to examine what adolescents know about substance use to create effective risk reduction interventions.

    This Study

    This study acts as a guide, demonstrating (1) the analysis of automatic log files created by videogame software that catalog game play and (2) the association of these data with changes in substance use knowledge documented by assessments. This videogame-based intervention was grounded in social cognitive theory [20] and message framing [21]—derived from prospect theory [22]—and was developed to reduce HIV risk behaviors among young adolescents. The purpose of this paper is to describe how a technology-based intervention can yield meaningful, objective evidence of intervention exposure and participant experience within a behavioral intervention. As proof of concept, we will then demonstrate how these data relate to an important psychological construct related to behavioral outcomes: knowledge [23]. We sought to determine how performance in the game (eg, by points earned) relates to knowledge as measured using a standardized validated instrument. The implications of this strategy for extracting data from the game software and evaluating participants’ game processes within a behavioral intervention can be extrapolated to interventions employing a wide variety of other technologies. Utilizing log files to capture a player’s unique game play decisions and process will ultimately facilitate a broader understanding of behavior change and potentially support the use of videogames not only as effective interventions, but also as valid assessment tools.


    Participants

    A total of 333 participants, aged 11 to 14 years, were enrolled in a randomized controlled trial in which they played either an intervention game, PlayForward: Elm City Stories, or a set of attention- and time-control games for a maximum of 16 hours over 6 weeks. Participants were recruited from community afterschool, school, and summer programs in New Haven, CT. All procedures were approved by the Yale University Human Investigation Committee.

    The Videogame Intervention

    PlayForward: Elm City Stories is a videogame developed as a risk reduction and prevention program targeting HIV risk behaviors, primarily sex- and substance use–related, in young minority adolescents. PlayForward was developed through extensive formative work with the target audience [5,8,24,25]. The main storyline of the game is comprised of “challenge stack” levels in which players travel through a virtual life from grade 7 to 12 and engage in role-playing scenarios where they must make decisions around risky behaviors (eg, unprotected sex, alcohol use) and experience the positive and negative consequences of those behaviors. The participants acquire risk-related knowledge, navigate peer relationships, and negotiate against peer pressure. Players encounter realistic stories experienced by middle school and high school students, such as sneaking into a significant other’s house, unplanned pregnancy, vandalism, and drunk driving. Players must also earn points in mini-games designed to build knowledge or behavioral skills needed to avoid risk, such as refusal, negotiation, or peer-assessment skills. Through these mini-games, players acquire the “senses” and “powers” needed to resolve the stories (for a complete list of the mini-games see [24]). To demonstrate our analysis strategy, we will focus on Know Power.

    Know Power provides the player with accurate and relevant information about the consequences of engaging in risky behaviors. This mini-game, comprised of 10 levels, emphasizes the development and clarification of adolescents’ values and evaluations of risk behavior and provides them with information about the consequences of and alternatives to engaging in these behaviors. Positive attitudes toward avoiding substance use, delaying sexual initiation, and general risk reduction are cultivated in this component of the game. Know Power specifically focuses on increasing knowledge and creating positive attitudes about HIV risk reduction behaviors.

    Measures

    Knowledge

    Participants were asked to complete a 22-item multiple-choice assessment of substance use and sexual health knowledge at each time point. The 22 items related to knowledge were adapted from several adolescent knowledge content sources, including an evidence-based curriculum that has been proven effective in reducing the risk of HIV, sexually transmitted infections, and teen pregnancy in young minority adolescents [26-28]. For this study, the subset of 8 questions addressing substance use knowledge was used. Participant response options were true, false, and not sure. These items included:

    1. Drug users who use needles to inject drugs into their bodies have a greater chance of getting HIV if they share needles with other people.
    2. People who use drugs occasionally can’t become addicted to them.
    3. Taking someone else’s prescription drugs is safe as long as the person giving them to you says it is okay.
    4. Using any drug can be more dangerous when taken with alcohol or other drugs.
    5. People are more likely to make unsafe decisions about sex (eg, not wearing a condom) if they have been using drugs or drinking alcohol.
    6. Drinking coffee or taking a cold shower can help sober someone up who is drunk.
    7. If a person drinks too much alcohol, they might get sick from alcohol poisoning.
    8. Alcohol poisoning can cause a person to stop breathing or choke to death on their own vomit.
    Game Play Log Files

    PlayForward was designed to capture granular information about each player’s gameplay experience, actions, and behavior in event logs. There were 2 kinds of data collected: (1) player game state data, the traditional save/load data required so that a player can save their progress and then pick up where they left off later and (2) activity logging data that captures relevant actions players took during game play. Activity logging data included which game content players were exposed to, how long it took them to solve particular game levels, how much time was spent playing various portions of the game, when players beat particular levels of the game, and how many levels were beaten. Every action that the player performed in the game-selecting options, entering/exiting a game area, or making a choice was recorded with a timestamp. Timestamped data were stored in comma-delimited log files, which were then imported into a database for data analysis. Two game play variables from the activity logging data were extracted: the total amount of time spent playing PlayForward and the total number of game levels (challenge stacks) beaten. Total game play time was calculated by identifying timestamped game start and end events, calculating a time interval for each game session, and then summing the time across all play sessions during the trial. Total number of game levels beaten was calculated by adding up the number of game level completion events for each player. We did not focus on which game content players were exposed to because PlayForward is sequenced in a linear fashion, such that players are required to beat particular mini-game levels in a specific sequence to progress through challenge stacks. Therefore, players with the same number of total levels beaten were exposed to the same content.

    Players in the PlayForward cohort were divided into 2 subsets (high scoring and low scoring) based on whether they were above (high scoring) or below (low scoring) the median number (median=8) of total game levels beaten. When participants were instructed to restart the game, game play time from their second attempt was factored into the total number of hours played, but completion of game levels a second time did not count toward a higher score (scores were capped at 12 because there are 12 levels in the game).

    Data Analysis

    Substance use knowledge was examined for participants in the PlayForward condition at baseline, 3 months, and 6 months postintervention. First, a repeated measures ANOVA was used to examine mastery effects on substance use knowledge over time. Bivariate relationships were also examined among log file variables and substance use knowledge. A 2-sample t test was used to compare substance use knowledge scores between the high- and low-scoring groups at each time point. Multivariate regression methods were used to assess the impact of baseline knowledge score, total amount of time spent playing PlayForward, and total number of game levels (challenge stacks) beaten by each player on the gain in substance use knowledge at 3- and 6-months postintervention.


    For the current study, only participants in the experimental condition (ie, PlayForward) were examined because these were the participants with game play data available (N=166; mean age 12.95, SD 1.03 years; mean game play time 7.27, SD 3.55 hours; median 8.24, IQR 3.70-10.14 hours; see Table 1 for demographics). Of the participants in the PlayForward group, 72 were in the high-scoring group, 72 were in the low-scoring group, and 22 players either had missing or corrupted log file data due to game software or transcription errors. Those 22 players with missing or corrupted log files were not included in the analyses. There was a significant difference between the age groups such that older participants were also more likely to be high-scoring players. A total of 37 players beat the PlayForward videogame and were instructed to restart the game and play through a second time.

    Table 1. Baseline demographic characteristics of participants (N=144).
    CharacteristicGroup by game play scoreTest statisticP

    		High score n=72Low score n=72χ2 (df)t142
    Gender (female), n (%)36 (50)31 (43)0.7 (1)
    		.40
    Age (years), mean (SD)13.18 (1.03)12.72 (1.03)
    		2.68.01
    Race/Ethnicity, n (%)

    		0.6 (4)
    		.96

    		Caucasian3 (4)3 (4)



    		Black27 (38)23 (32)



    		Hispanic25 (35)29 (40)



    		Biracial8 (11)8 (11)



    		Other9 (13)9 (13)


    Group Differences in Substance Use Knowledge

    Knowledge scores were assigned based on the total number of correct answers on the test (maximum score=8; mean 4.52, SD 1.81). There were no statistically significant differences between the groups with respect to baseline substance use knowledge score (high scoring: mean 4.65, SD 1.80; low scoring: mean 4.42, SD 1.82; t102=–0.64, P=.52). Players with a high score in the game had significantly higher scores on the assessment of substance use knowledge at both 3-month (high scoring: mean 6.72, SD 1.45; low scoring: mean 4.74, SD 2.42; t102=–5.11, P=.001) and 6-month (high scoring: mean 6.56, SD 1.53; low scoring: mean 4.78, SD 2.38; t102=–4.57, P=.001) follow-ups. Specifically, players who mastered the intervention material (ie, high-scoring players) were also more knowledgeable of substance use facts (eg, “If a person drinks too much alcohol they might get sick from alcohol poisoning”). A significant mastery by time interaction existed for substance use knowledge (F2,101=9.41, P=.001). High-scoring players had greater substance use knowledge at the 3- and 6-month follow-up periods.

    Factors Associated with Substance Use Knowledge

    We examined the bivariate relationships among the log file data (number of levels beaten: mean 7.64, SD 3.61; number of hours playing game: mean 7.27, SD 3.55) and substance use knowledge at baseline (mean 4.52, SD 1.81) and at the 3-month (mean 5.68, SD 2.24) and 6-month (mean 5.65, SD 2.18) follow-ups (see Table 2). Baseline substance use knowledge was not associated with either number of levels beaten or total time spent playing the game. Both 3- and 6-month substance use knowledge were positively associated with number of levels beaten or total time spent playing the game. Number of levels beaten was more strongly related to 3- and 6-month substance use knowledge than total time spent playing the game.

    Table 2. Relationship between knowledge and game play variables.
    Knowledge and game play12345

    		R2PR2PR2PR2PR2P
    1. Baseline knowledgea__








    2. 3-Month knowledgea.411.001—






    3. 6-Month knowledgea.331.001.809.001—




    4. Number of levels beatenb.113
    		.528.001.531.001—


    5. Number of hours playing game–.033
    		.205.03.219.02.584.001—

    a Knowledge scores range from 0 to 8.

    b Number of levels beaten maximum=12.

    Predictors of Substance Use Knowledge at Follow-Up

    Three-Month Follow-Up

    Age differences were found at baseline (Table 1) such that older participants were more likely to also be high-scoring players; therefore, age was included in the model to determine if age differences were responsible for mastery of game material. Collapsing across groups (ie, high scoring vs low scoring), results of the multivariate regression analyses examining game play variables as predictors of substance use knowledge were significantly related to substance use knowledge at the 3-month follow-up (Table 3) (R2=.396, F4,102=18.36, P=.001). Only 2 predictors were significantly uniquely associated with substance use knowledge at 3 months; baseline knowledge (B=.281, t106=3.47, P=.001) and game levels beaten (B=.489, t106=5.12, P=.001) were positively related to substance use knowledge at 3 months. Higher scores on these constructs were associated with higher substance use knowledge at 3 months. Age and total time playing PlayForward, on the other hand, were not statistically significantly related to substance use knowledge at 3 months.

    Table 3. Association of game play with gains in substance use knowledge.
    Substance use knowledgeBSEβt102P
    3 Month





    		Number of levels beaten.310.06.495.12.001

    		Number of hours playing game–.070.07–.09–1.00.32

    		Baseline knowledge.350.10.283.47.001

    		Age.380.18.161.85.07
    6 Month





    		Number of levels beaten.110.05.182.19.03

    		Number of hours playing game–.020.05–.03–0.35.71

    		3-month knowledge.720.07.7410.23.001

    		Age–.130.13–.06–0.98.33
    Six-Month Follow-Up

    Similar patterns appeared for substance use knowledge at the 6-month follow-up (Table 3) (R2=.644, F4,102=48.97, P=.001). Only 2 predictors were significantly uniquely associated with substance use knowledge at 6 months; 3 month knowledge (B=.735, t106=10.23, P=.001) and game levels beaten (B=.179, t106=2.19, P=.03) were positively related to substance use knowledge at 6 months. Higher scores on these constructs were associated with higher substance use knowledge at 6 months. Age and total time playing PlayForward, on the other hand, were not statistically significantly related to substance use knowledge at 6 months. Overall, it was number of levels beaten (ie, mastery of intervention content) that was associated with a gain in substance use knowledge at 3 months and 6 months.


    This study describes how a videogame intervention can produce objective data demonstrating exposure to the intervention and yield meaningful evidence of the association between a participant’s success within the videogame and changes in an important psychological construct—knowledge. We examined how 2 separate measures of game play—total amount of time spent playing the PlayForward videogame and the total number of game levels (challenge stacks) beaten by each player—might be related to substance use knowledge at 3- and 6-month follow-ups. These 2 measures of game play broaden the scope of traditional face-to-face intervention assessment by accurately tracking the participant’s success within the game. For example, it appears that mastery of the intervention material (eg, game levels beaten), and not time spent playing each level, influences acquisition of substance use knowledge.

    High-scoring game players gained and retained significantly more knowledge about substance use at the follow-up time points than their low-scoring counterparts did. Interestingly, completing and beating more of the game levels, and not total game play time, resulted in greater changes in knowledge over time. Mastery of the material increases substance use knowledge for up to 6 months after game play has been completed.

    Implications for eHealth Interventions

    There is a growing body of literature demonstrating eHealth interventions as efficacious tools for creating behavior change [12,29]. However, it is still largely unknown how this change occurs. The current study examined one behavioral antecedent of behavior change—knowledge—to illustrate how objective data collected throughout game play can contribute to our understanding of how change happens. An implicit assumption in recommendations for behavior change interventions is that more time spent in interventions will produce the greatest amount of behavior change [30]. For example, an intervention examining the effects of a single-session versus a multiple-session smoking cessation intervention suggests multiple sessions (ie, more time spent in the intervention) are related to significantly higher abstinence rates than a single session [31]. Our data, on the other hand, suggest that it is not time but mastery of the material that is related to changes of important psychological mediators (ie, knowledge) of behavior change. This is a valuable methodological point. Many eHealth interventions merely present information to participants [13], much like a brochure, but do not include content that must be mastered. Our data suggest that simple exposure to intervention content is not enough to create change; instead, what is needed is the opportunity to process intervention content more deeply. It is not enough to give people information, we must give them a means by which to master intervention content to change behavior.

    Traditional human-delivered interventions rely heavily on self-report data or a measure of overall intervention time as a proxy measure to examine an individual participant’s experience throughout an intervention. eHealth interventions, such as videogames, allow researchers to objectively track a participant’s intervention experience through the game software by easily collecting data on time spent in individual segments of the intervention, overall time spent completing the intervention, and mastery of intervention material. This information can then be used to learn which material is most important to the behavior change process. For example, the data collection features of eHealth interventions could allow us to determine if one portion of an intervention dedicated to knowledge acquisition mastery is enough to produce knowledge gains at follow-up interval periods or if more sessions are necessary. This is just one of the questions that can be answered by using a technology-based intervention as a data collection tool. Before eHealth interventions, these questions were burdensome and resource-intensive to investigate [32]. These are empirical questions that can—and should—be answered to facilitate optimal intervention development and to inform the progress of theoretical innovations in the behavior change domain.

    Behavior change interventions target multiple constructs at once to create behavior change [11]. Thus, conventional behavior change interventions are unable to determine which intervention components are contributing to behavior change. A dismantling study design could help determine if a focused manipulation of one construct can have diffuse effects on other constructs [33] and/or if such a focused manipulation of one construct is as good as or better than manipulation of multiple constructs simultaneously. Given the lower delivery cost, standardization of delivery content, multiple dissemination channels, and objective data collection of participant experience [2,3], eHealth interventions are in a unique and prime position to answer these questions.

    Limitations

    With this study, as a demonstration of the novel methods of evaluating gameplay data, we chose to examine the behavioral antecedent of knowledge, not behavioral outcomes, which will be included in future analyses. Another limitation of this study was that knowledge relied on a self-report measure; this limitation is shared with the majority of intervention research. However, log file data offer unique assessment techniques to examine intervention exposure and outcomes whereby adding another valid source of data and potentially reducing the need to rely solely on self-report data. Additionally, only a small number of items (ie, 8) were used to assess substance use knowledge.

    Conclusions

    Increasingly, eHealth interventions, particularly videogames, are becoming a standard and effective form of intervention delivery [2-5]. Interventions delivered via emerging technologies have the potential to enhance traditional forms of assessment with real-time objective data collection techniques. Importantly, videogames can and do serve a dual purpose of delivering health interventions and providing a unique assessment tool with minimal additional effort required from participants. Participant log files are just one potential technique. Our findings highlight the valuable contributions eHealth interventions make to the study of health behavior change by using objective data collection techniques that are difficult to replicate in traditional human-delivered behavioral interventions. This research illustrates the need for the rigorous exploration of new data collection opportunities provided by eHealth interventions. These innovative methods hold the potential to document specific engagement, interactions, and exposure that may provide key information regarding which intervention components are most effective in influencing participant outcomes.

    Acknowledgments

    This research was supported by NICHD R01 HD062080-01 (ClinicalTrials.gov: NCT01666496) from the National Institute of Child Health and Human Development (PI: Lynn E Fiellin, MD). Dr Montanaro is supported by T32MH020031 from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. Additional thanks to Kimberly Hieftje, PhD and Tyra Pendergrass, MEM for their assistance with data collection.

    Conflicts of Interest

    Drs Fiellin and Duncan are affiliated with KnackTime Interactive, a small commercial venture that focuses on the distribution of evidence-based videogames for risk reduction and prevention in youth and young adults. This relationship is extensively managed by Drs Fiellin and Duncan and their respective institutions.

    1. Eysenbach G. What is e-health? J Med Internet Res 2001;3(2):E20 [FREE Full text] [CrossRef] [Medline]
    2. Lehto T, Oinas-Kukkonen H. Persuasive features in web-based alcohol and smoking interventions: A systematic review of the literature. J Med Internet Res 2011;13(3):e46 [FREE Full text] [CrossRef] [Medline]
    3. Noar SM, Black HG, Pierce LB. Efficacy of computer technology-based HIV prevention interventions: A meta-analysis. AIDS 2009 Jan 2;23(1):107-115. [CrossRef] [Medline]
    4. Hieftje K, Edelman EJ, Camenga DR, Fiellin LE. Electronic media-based health interventions promoting behavior change in youth: A systematic review. JAMA Pediatr 2013 Jun;167(6):574-580 [FREE Full text] [CrossRef] [Medline]
    5. Fiellin LE, Hieftje KD, Duncan LR. Videogames, here for good. Pediatrics 2014 Nov;134(5):849-851. [CrossRef] [Medline]
    6. Ruggiero L, Moadsiri A, Quinn LT, Riley BB, Danielson KK, Monahan C, et al. Diabetes island: Preliminary impact of a virtual world self-care educational intervention for African Americans with type 2 diabetes. JMIR Serious Games 2014;2(2):e10 [FREE Full text] [CrossRef] [Medline]
    7. Kato PM, Cole SW, Bradlyn AS, Pollock BH. A video game improves behavioral outcomes in adolescents and young adults with cancer: A randomized trial. Pediatrics 2008 Aug;122(2):e305-e317. [CrossRef] [Medline]
    8. Hieftje K, Duncan LR, Fiellin LE. Novel methods to collect meaningful data from adolescents for the development of health interventions. Health Promot Pract 2014 Sep;15(5):714-722 [FREE Full text] [CrossRef] [Medline]
    9. Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health 2010;31:399-418. [CrossRef] [Medline]
    10. Noar SM. Behavioral interventions to reduce HIV-related sexual risk behavior: Review and synthesis of meta-analytic evidence. AIDS Behav 2008 May;12(3):335-353. [CrossRef] [Medline]
    11. Norman GJ, Zabinski MF, Adams MA, Rosenberg DE, Yaroch AL, Atienza AA. A review of eHealth interventions for physical activity and dietary behavior change. Am J Prev Med 2007 Oct;33(4):336-345 [FREE Full text] [CrossRef] [Medline]
    12. Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: Are our theories up to the task? Transl Behav Med 2011 Mar;1(1):53-71 [FREE Full text] [CrossRef] [Medline]
    13. Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth). Annu Rev Clin Psychol 2007;3:53-76. [CrossRef] [Medline]
    14. Neuhauser L, Kreps GL. Rethinking communication in the e-health era. J Health Psychol 2003 Jan;8(1):7-23. [CrossRef] [Medline]
    15. Barak A, Klein B, Proudfoot JG. Defining internet-supported therapeutic interventions. Ann Behav Med 2009 Aug;38(1):4-17. [CrossRef] [Medline]
    16. Johnston L, O’Malley P, Bachman J, Schulenberg J. Monitoring the Future National Survey Results on Drug Use, 1975–2011. Volume I: Secondary School Students. Ann Arbor, MI: University of Michigan; 2012.
    17. Weinhardt LS, Carey MP. Does alcohol lead to sexual risk behavior? Findings from event-level research. Annu Rev Sex Res 2000;11:125-157 [FREE Full text] [Medline]
    18. Anderson BJ, Stein MD. A behavioral decision model testing the association of marijuana use and sexual risk in young adult women. AIDS Behav 2011 May;15(4):875-884 [FREE Full text] [CrossRef] [Medline]
    19. Eaton D, Kann L, Kinchen S, Shanklin S, Flint KH, Hawkins J, Centers for Disease Control and Prevention. Youth risk behavior surveillance—United States, 2011. MMWR Morb Mortal Wkly Rep 2012;61(4):1-162.
    20. Bandura A. Health promotion from the perspective of social cognitive theory. Psychol Health 1998;13(4):623-649.
    21. Rothman AJ, Salovey P. Shaping perceptions to motivate healthy behavior: The role of message framing. Psychol Bull 1997 Jan;121(1):3-19. [Medline]
    22. Kahneman D, Tversky A. Prospect theory: An analysis of decision under risk. Econometrica 1979;263:291.
    23. Fisher JD, Fisher WA, Misovich SJ, Kimble DL, Malloy TE. Changing AIDS risk behavior: Effects of an intervention emphasizing AIDS risk reduction information, motivation, and behavioral skills in a college student population. Health Psychol 1996 Mar;15(2):114-123. [Medline]
    24. Duncan LR, Hieftje KD, Culyba S, Fiellin LE. Game playbooks: Tools to guide multidisciplinary teams in developing videogame-based behavior change interventions. Transl Behav Med 2014 Mar;4(1):108-116 [FREE Full text] [CrossRef] [Medline]
    25. Hieftje K, Rosenthal MS, Camenga DR, Edelman EJ, Fiellin LE. A qualitative study to inform the development of a video game for adolescent HIV prevention. Games Health J 2012 Aug 10;1(4):294-298 [FREE Full text] [CrossRef] [Medline]
    26. Jemmott JB, Jemmott LS, Fong GT. Efficacy of a theory-based abstinence-only intervention over 24 months: A randomized controlled trial with young adolescents. Arch Pediatr Adolesc Med 2010 Feb;164(2):152-159 [FREE Full text] [CrossRef] [Medline]
    27. Kelly JA, St Lawrence JS, Hood HV, Brasfield TL. An objective test of AIDS risk behavior knowledge: scale development, validation, and norms. J Behav Ther Exp Psychiatry 1989 Sep;20(3):227-234. [Medline]
    28. Jemmott L, Jemmott J, McCaffree K. Making a Difference!: An Abstinence Approach to HIV/STD and Teen Pregnancy Prevention. New York: Select Media, Inc; 2002:1-263.
    29. Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010;12(1):e4 [FREE Full text] [CrossRef] [Medline]
    30. Peters E, Klein W, Kaufman A, Meilleur L, Dixon A. More is not always better: intuitions about effective public policy can lead to unintended consequences. Soc Issues Policy Rev 2013 Jan 1;7(1):114-148 [FREE Full text] [CrossRef] [Medline]
    31. Zhu SH, Stretch V, Balabanis M, Rosbrook B, Sadler G, Pierce JP. Telephone counseling for smoking cessation: Effects of single-session and multiple-session interventions. J Consult Clin Psychol 1996 Feb;64(1):202-211. [Medline]
    32. Jacobson NS, Dobson KS, Truax PA, Addis ME, Koerner K, Gollan JK, et al. A component analysis of cognitive-behavioral treatment for depression. J Consult Clin Psychol 1996 Apr;64(2):295-304. [Medline]
    33. Borkovec TD, Newman MG, Pincus AL, Lytle R. A component analysis of cognitive-behavioral therapy for generalized anxiety disorder and the role of interpersonal problems. J Consult Clin Psychol 2002 Apr;70(2):288-298. [Medline]

    Edited by G Eysenbach; submitted 24.02.15; peer-reviewed by L Ruggiero, W Nilsen; comments to author 23.07.15; revised version received 03.08.15; accepted 15.08.15; published 28.10.15

    Copyright

    ©Erika Montanaro, Lynn E Fiellin, Tamer Fakhouri, Tassos C Kyriakides, Lindsay R Duncan. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 28.10.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.