This paper is in the following e-collection/theme issue:

Consumer & Patient Education and Shared-Decision Making (679) Ethics, Privacy, and Legal Issues (479) Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (832) HIV/AIDS/STI Prevention and Care (873) LGBTQ Issues (448) Linguistics, Comprehension, and Learning (52)

Published on in Vol 19, No 3 (2017): March

Use of Videos Improves Informed Consent Comprehension in Web-Based Surveys Among Internet-Using Men Who Have Sex With Men: A Randomized Controlled Trial

Use of Videos Improves Informed Consent Comprehension in Web-Based Surveys Among Internet-Using Men Who Have Sex With Men: A Randomized Controlled Trial

Use of Videos Improves Informed Consent Comprehension in Web-Based Surveys Among Internet-Using Men Who Have Sex With Men: A Randomized Controlled Trial

Authors of this article:

Eric William Hall1 Author Orcid Image ;   Travis H Sanchez1 Author Orcid Image ;   Aryeh D Stein2 Author Orcid Image ;   Rob Stephenson3 Author Orcid Image ;   Maria Zlotorzynska1 Author Orcid Image ;   Robert Craig Sineath1 Author Orcid Image ;   Patrick S Sullivan1 Author Orcid Image
Article Authors Cited by (18) Tweetations (11) Metrics

    Original Paper

    1Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, United States

    2Hubert Department of Global Health, Rollins School of Public Health, Emory University, Atlanta, GA, United States

    3Department of Health Behavior and Biological Sciences, School of Nursing and The Center for Sexuality and Health Disparities, University of Michigan, Ann Arbor, GA, United States

    Corresponding Author:

    Eric William Hall, MPH

    Department of Epidemiology

    Rollins School of Public Health

    Emory University

    GCR 432

    1518 Clifton Road

    Atlanta, GA, 30322

    United States

    Phone: 1 404 727 8965

    Fax:1 404 727 8737

    Email: eric.w.hall@emory.edu


    Background: Web-based surveys are increasingly used to capture data essential for human immunodeficiency virus (HIV) prevention research. However, there are challenges in ensuring the informed consent of Web-based research participants.

    Objective: The aim of our study was to develop and assess the efficacy of alternative methods of administering informed consent in Web-based HIV research with men who have sex with men (MSM).

    Methods: From July to September 2014, paid advertisements on Facebook were used to recruit adult MSM living in the United States for a Web-based survey about risk and preventive behaviors. Participants were randomized to one of the 4 methods of delivering informed consent: a professionally produced video, a study staff-produced video, a frequently asked questions (FAQs) text page, and a standard informed consent text page. Following the behavior survey, participants answered 15 questions about comprehension of consent information. Correct responses to each question were given a score of 1, for a total possible scale score of 15. General linear regression and post-hoc Tukey comparisons were used to assess difference (P<.001) in mean consent comprehension scores. A mediation analysis was used to examine the relationship between time spent on consent page and consent comprehension.

    Results: Of the 665 MSM participants who completed the comprehension questions, 24.2% (161/665) received the standard consent, 27.1% (180/665) received the FAQ consent, 26.8% (178/665) received the professional consent video, and 22.0% (146/665) received the staff video. The overall average consent comprehension score was 6.28 (SD=2.89). The average consent comprehension score differed significantly across consent type (P<.001), age (P=.04), race or ethnicity (P<.001), and highest level of education (P=.001). Compared with those who received the standard consent, comprehension was significantly higher for participants who received the professional video consent (score increase=1.79; 95% CI 1.02-2.55) and participants who received the staff video consent (score increase=1.79; 95% CI 0.99-2.59). There was no significant difference in comprehension for those who received the FAQ consent. Participants spent more time on the 2 video consents (staff video median time=117 seconds; professional video median time=115 seconds) than the FAQ (median=21 seconds) and standard consents (median=37 seconds). Mediation analysis showed that though time spent on the consent page was partially responsible for some of the differences in comprehension, the direct effects of the professional video (score increase=0.93; 95% CI 0.39-1.48) and the staff-produced video (score increase=0.99; 95% CI 0.42-1.56) were still significant.

    Conclusions: Video-based consent methods improve consent comprehension of MSM participating in a Web-based HIV behavioral survey. This effect may be partially mediated through increased time spent reviewing the consent material; however, the video consent may still be superior to standard consent in improving participant comprehension of key study facts.

    Trail Registration: Clinicaltrials.gov NCT02139566; https://clinicaltrials.gov/ct2/show/NCT02139566 (Archived by WebCite at http://www.webcitation.org/6oRnL261N).

    J Med Internet Res 2017;19(3):e64

    doi:10.2196/jmir.6710

    Keywords

    informed consent; surveys and questionnaires; HIV 


    Men who have sex with men (MSM) continue to be the group most impacted by human immunodeficiency virus (HIV) in the United States. In 2014, 70% of all new infections in the United States occurred among MSM [1]. In many regions around the world, HIV incidence rates among MSM have been increasing [2-4]. This increase has been theorized, in part, to result from an increase of MSM using the Internet to facilitate sexual relationships through partner selection websites [4-7]. There is evidence that men who arrange Web-based sexual encounters may have increased odds of having unprotected anal intercourse [5,6,8], which presents an increased risk for HIV transmission. As a result, Web-based HIV research and prevention opportunities can be particularly valuable.

    Increasing Internet accessibility also can improve the delivery of health services among all MSM. Internet usage is highest among young Americans: 96% of 18-29 year olds used the Internet in 2015 [9]. This high coverage provides an opportunity to deliver Web-based HIV prevention and treatment information on a large scale [10,11]. However, HIV prevention research is first needed to establish the efficacy of prevention programs.

    The Internet is increasingly being used to recruit underserved MSM and engage them in HIV prevention research [7]. Compared with face-to-face interviews, Web-based research can collect data from a large number of people in a short period of time [12]. Web-based studies can anonymously include MSM who might be stigmatized if their sexual interests were known publicly [13]. Furthermore, Web-based research can provide access to men who might be at an elevated risk of HIV infection. Compared with participants recruited at physical venues, MSM recruited through the Web are more likely to self-report sexually transmitted infections (STIs), anal intercourse, and unprotected anal intercourse [14-18].

    However, there are challenges to using the Internet for HIV prevention research [19]. The data can be subject to selection bias because of demographic differences in Internet use and access [9]. Willingness to click on an advertisement, provide consent, and begin a Web-based survey differs by age, race, education, and urbanicity of residence [20]. Furthermore, there are ethical challenges that are unique to Web-based research [11,21]. When consent is administered on the Web, it is difficult to confirm the age, competency, and comprehension of a potential participant [22]. This is particularly concerning because many of the topics that are typically covered in HIV prevention research (sexual behavior, drug use, commercial sex work, and so on) contain confidential and sensitive information.

    Most Web-based consent processes involve the respondent reading a document and then indicating that they agree to participate by clicking a button. This is similar to the lengthy terms-of-service documents that many Internet and mobile phone services require, and participants may develop a habit of accepting documents without fully reviewing them. An ethical review of a Web-based HIV prevention study called for innovative ways to provide informed consent to participants [23]. Better alignment between a participant’s typical Web-based experience and the informed consent process may improve their consumption of Web-based consent information.

    Many Web-based social media interactions now involve the use of photos and videos [24]. Internet users may also be more acclimated to list-based or brief summaries when they do choose to consume textual information. Previous Web-based research with MSM has also shown that brief consent summaries may improve consent information comprehension over a standard consent process, but this research was done before Web-based interactions became more photo- or video-focused and therefore we did not examine this alternative [19,25]. The objective of our study was to assess if comprehension of key informed consent facts improved through the use of these new techniques versus a standard text-based informed consent form.


    Study Design

    A randomized trial (Clinicaltrials.gov NCT02139566) was conducted to assess alternative methods of administering informed consent in a Web-based HIV behavioral research study with young adult MSM residing in the United States. All survey data were collected through a Health Insurance Portability and Accountability Act (HIPAA)–compliant Web-based survey platform (SurveyGizmo, Boulder, CO). Participants were recruited from Facebook from July 28 to September 8, 2014. Banner advertisements were presented to men who indicated they were interested in media and services targeted toward gay men or interested in men on their Facebook profiles. Respondents who clicked on the banners were taken to an eligibility screener. To be eligible, respondents had to report that they lived in the United States, were 18 to 34 years old, were male at birth, and have had sex with another man in the past 12 months. Eligible respondents were randomized by the survey into one of the 4 consent groups: standard informed consent, frequently asked questions (FAQs) informed consent, a video informed consent produced by a professional company (Tom Coggia, iconAtomic), or a video informed consent produced by study staff (Figure 1). The 3 alternative consent methods were developed based on the results of 5 online focus groups which engaged MSM in discussions about how they considered and interacted with Web-based informed consent documents.

    The standard informed consent consisted of a consent document presented in a scrolling window. The FAQ informed consent consisted of 19 questions (e.g., “What is the purpose of the study?”,“Who is conducting the study?”) that revealed short paragraphs of information when the participant clicked on them (Multimedia Appendix 1). The 2 video consents were each about 3.5 minutes long and covered the same topics as the standard and FAQ consents, but used a different script written by the researchers and were presented verbally with bullets of key points highlighted on the screen (Multimedia Appendices 2 and 3). The level of reading comprehension was consistent between the video scripts, the standard consent, and the FAQ consent. All respondents also had the option to open or download the full standard informed consent document as a portable document format (PDF). This study was reviewed and approved by Emory University Institutional Review Board that approved a waiver of written informed consent. After providing consent, participants were directed to a 15-minute Web-based survey on demographic characteristics, sexual history, HIV and STI testing history, and use of HIV prevention services. At the end of the survey, we asked 15 questions about essential elements covered in the informed consent process (Multimedia Appendix 4). Participants did not know there would be questions about the informed consent process. Participants who completed the survey and answered the comprehension questions were given a US $20 Amazon gift card sent by email.

    ‎
    Figure 1. Consort diagram and study scheme for participant enrollment of men who have sex with men in a randomized trial of informed consent methods, United States, 2014.
    View this figure

    Measures

    The primary analytic outcome was total consent comprehension score, which was calculated by tabulating the number of consent comprehension questions that each participant got correct. Similar to previously used methods, 1 point was given for each correct question for a minimum total score of 0 and maximum of 15 [26]. The 15 consent comprehension questions were selected from a larger pool of 29 possible questions with multiple questions from each of the essential elements of informed consent [27]. Before the main study, consent comprehension questions were pilot-tested among MSM recruited in the same manner as the main study (banner advertisements on Facebook) and were presented with a standard consent form. The pilot study recruited 132 MSM who completed a short survey and the 29 comprehension questions. The data were analyzed for frequencies of correct responses to the 29 pilot consent comprehension questions. At least one question from each of the essential elements was retained. If there were multiple questions from a category, questions with very low or very high correct response rates were removed in order to allow for variability in responses to assess difference in comprehension during the main study.

    Time spent on consent was measured by amount of time the respondent stayed on the Web page with the consent information. In order to progress to the next page, respondents had to click on a button indicating they agree (or do not agree) to participate in the study. Demographic variables (race or ethnicity, highest level of education, and age) were summarized as frequencies and percentages in each consent group. Race or ethnicity was coded into 6 categories: Asian, black, Hispanic, multiracial, American Indian or Alaska Native, and white.

    Analyses

    All analyses were done in SAS version 9.4 (SAS Institute, Cary, NC). As recommended for Web-based participant recruitment, a deduplication protocol was determined before data collection to remove duplicate or artificial survey attempts [28]. While implementing the study, we noted a pattern of frequent responses from the same Internet protocol (IP) address, very short survey completion times, and unusual email addresses—all indicators of potential fraudulent responses from artificial hacking or bot programs possibly aimed at getting the monetary incentive. We immediately put measures in place to curtail this activity, including Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA) codes and verification of email addresses submitted for incentives [29]. For purposes of analyses, the following types of responses were excluded: a single IP address submitted 5 or more responses, total completed survey time was less than 5 minutes, or invalid email address as determined by staff member reviewing all submitted emails. All analysis was intent-to-treat and all participants were analyzed in the original randomization groups.

    Pearson chi-square tests were used to assess demographic differences by consent randomization group. Time spent on consent and consent comprehension score were analyzed as continuous variables and summarized using medians and interquartile ranges (IQRs) or means and standard deviations, where appropriate. Residuals for both continuous variables were gauged for normality by inspecting histograms, boxplots, and quantile-quantile probability plots. As the time spent on consent was not normally distributed, Kruskal-Wallis tests were used to assess differences in time spent on consent by demographic variables. Time spent on consent was log transformed to assess the correlation with consent comprehension score using Pearson correlation coefficient.

    Factors associated with consent comprehension score were assessed using general linear regression. Analyses were conducted for consent group, age, race, and highest level of education. Least-squares means and 95% CIs were reported. When overall significant differences were found, pairwise comparisons were considered using the Tukey-Kramer method [30]. A mediation analysis was conducted to determine if time spent on consent was a mediator in the relationship between consent group and consent comprehension score. Direct effects were calculated to determine the change in consent comprehension score when time on consent was held constant. Indirect effects were calculated to determine the change in consent comprehension score when consent group was held constant and time on consent differed by the amount it differed between consent groups. We used methods for mediation analysis with a multicategorical independent variable as described by Hayes et al [31]. Asymmetric bootstrap procedure with 10,000 resamples was used to calculate bias-corrected CIs around the relative indirect effect estimates. All statistical tests were assessed at alpha=.05.


    Of the 2849 survey responses, there were 665 (23.34%, 665/2849) eligible participants who consented to be part of the study and complete the survey and comprehension questions (Figure 1). There were 161 (24.2%, 161/665) participants in the standard consent group, 180 (27.1%, 180/665) in the FAQ group, 178 (26.8%, 178/665) in the professional video group, and 146 (22.0%, 146/665) in the staff video group (Table 1). When subsetting the data to only include survey responses submitted after the artificial hacking or bot was curtailed, the percentage of valid, eligible surveys didn’t substantially differ (data not presented).

    Table 1. Demographic characteristics of 665 men who have sex with men who participated in a randomized trial of informed consent methods, by informed consent group randomization, United States, 2014.
    Demographic characteristicsConsent group


    		n (%)Standard
    n (%)    		FAQa
    n (%)    		Professionally produced video
    n (%)    		Staff-produced video
    n (%)    		P valueb
    All

    		161 (24.2)180 (27.1)178 (26.8)146 (22.0)
    Race or ethnicity.13

    		Asian or Pacific Islander24 (3.6)3 (1.9)11 (6.2)4 (2.3)6 (4.1)

    		Black22 (3.3)5 (3.1)5 (2.8)10 (5.7)2 (1.4)

    		Hispanic147 (22.1)31 (19.4)45 (25.4)39 (22.3)32 (22.1)

    		Multiracial26 (3.9)6 (3.8)8 (4.5)8 (4.6)4 (2.8)

    		American Indian or Alaska Native8 (1.2)1 (0.6)5 (2.8)0 (0)2 (1.4)

    		White430 (64.7)114 (71.3)103 (58.2)114 (65.1)99 (68.3)

    		Missing8 (1.2)




    Highest level of schooling.73

    		≤ High school88 (13.3)18 (11.2)25 (14.0)29 (16.4)16 (11.0)

    		Some college288 (43.4)75 (46.6)73 (40.8)80 (45.2)60 (41.1)

    		Bachelor\'s degree183 (27.6)46 (28.6)52 (29.1)42 (23.7)43 (29.5)

    		Graduate or professional104 (15.7)22 (13.7)29 (16.2)26 (14.7)27 (18.5)

    		Missing2




    Age, in years.34

    		18-22164 (24.7)46 (28.6)45 (25.0)47 (26.4)26 (17.8)

    		23-27236 (35.5)57 (35.4)58 (32.2)65 (36.5)56 (38.4)

    		28-34265 (39.8)58 (36.0)77 (42.8)66 (37.1)64 (43.8)
    Age in years, median (IQRc)26 (23-29)25 (22-28)26 (23-30)26 (22-29)26 (24-30).10

    aFAQ: frequently asked questions.

    bCalculated by Pearson chi-square test for categorical variables and by Kruskal-Wallis test for age as a continuous variable. All tests were assessed with alpha=.05.

    cIQR: Interquartile range.

    Of the participants that reported a race or ethnicity, most identified as white (64.7%, 430/657) or Hispanic (22.1%, 147/657). The median age was 26 years (IQR: 23-39). There were 88 (13.3%, 88/663) participants who reported that they did not receive any schooling past high school. There was not a statistically significant difference in participant race, highest level of education, or age across the 4 randomized consent groups.

    Among all participants, the median time spent on the informed consent was 60.5 seconds (Table 2). The time spent on the consent page significantly differed by type of consent (P<.001), with participants in the professional video (median=115.0 seconds, IQR: 37.0-208.0) and staff video (median=117.0 seconds, IQR: 39.0-212.0) consent groups spending more time than participants in the standard (median=37.0 seconds, IQR: 14.5-88.0) and FAQ (median=20.5 seconds, IQR: 11.0-82.0) consent groups. About 36.5% (65/178) of participants in the professional video group and 19.9% (29/146) of participants in the staff video group stayed on the consent page for an amount of time that was longer than the duration of the video (professional video: 200 seconds; staff video: 218 seconds, Figure 2). The amount of time spent on the informed consent page differed by race (P=.02) and education (P=.01). Time on consent was significantly correlated with consent comprehension score in all consent groups (all P<.001; Figure 2).

    Table 2. Time spent on informed consent and consent comprehension scores among men who have sex with men in a randomized trial of informed consent methods, United States, 2014.
    CharacteristicTime spent on consentaConsent comprehension score






    		Post-hoc Tukey comparisonsf


    		nMedianIQRbP
    valuec    		nMean (SD)dANOVAe P
    value    		Mean
    difference    		95% CIP
    value
    All
    		66060.517.0-162.0
    		6656.3 (2.9)



    Consent group<.001

    		<.001



    		Standard16037.014.5-88.0
    		1615.5 (2.7)
    		Referenceh


    		FAQg17820.511.0-82.0
    		1805.2 (2.7)
    		−0.30−1.06 to 0.47.75

    		Professional video177115.037.0-208.0
    		1787.3 (2.8)
    		1.791.02 to 2.55<.001

    		Staff video145117.039.0-212.0
    		1467.3 (2.8)
    		1.790.99 to 2.59<.001
    Race or ethnicity.02

    		<.001



    		Asian2471.018.0-212.0
    		244.9 (2.8)
    		−1.77−3.47 to −0.07.04

    		Black2295.530.0-200.0
    		227.1 (3.0)
    		0.44−1.33 to 2.22.98

    		Hispanic14725.013.0-146.0
    		1475.5 (2.4)
    		−1.19−1.97 to −0.42<.001

    		Multiracial2667.024.0-201.0
    		266.4 (2.4)
    		−0.22−1.86 to 1.41.99

    		American Indian or Alaska Native827.517.0-64.5
    		83.1 (1.6)
    		−3.52−6.41 to −0.63.01

    		White42568.020.0-166.0
    		4306.7 (3.0)
    		Reference

    Highest level of education.008

    		.001



    		≤ High school8839.014.0-103.5
    		885.5 (2.0)
    		Reference


    		Some college28753.015.0-154.0
    		2886.2 (3.0)
    		0.63−0.26 to 1.53.27

    		Bachelor\'s degree18065.018.0-173.0
    		1836.4 (3.0)
    		0.85−0.10 to 1.81.10

    		Graduate or professional10393.037.0-205.0
    		1047.1 (3.0)
    		1.610.55 to 2.68.01
    Age, in years.14

    		.04



    		18-2216452.516.0-157.0
    		1646.1 (2.5)
    		Reference


    		23-2723471.021.0-168.0
    		2366.7 (3.0)
    		0.57−0.12 to 1.26.13

    		28-3426253.515.0-165.0 2656.1 (3.0) −0.03−0.70 to 0.64.99

    aIn seconds.

    bIQR: interquartile range.

    cCalculated using Kruskal-Wallis tests with alpha=.05.

    dSD: standard deviation.

    eANOVA: analysis of variance.

    fThe null value for mean difference is 0.0.

    gFAQ: frequently asked questions.

    hReference: Reference category for all mean difference comparisons within each characteristic.

    Participants in each consent group recorded the following mean comprehension scores: professional video, 7.3 (SD 2.8, range: 1-13); staff-produced video, 7.3 (SD 2.8, range: 1-13); FAQ, 5.2 (SD 2.7, range: 0-13); and standard consent, 5.5 (SD 2.7, range: 1-13; Figure 3). The average consent comprehension score differed significantly by consent group (P<.001), age (P=.04), race or ethnicity (P<.001), and highest level of education (P=.001; Table 2). On average, participants in the professional video group scored 1.79 (95% CI 1.02-2.55) points higher than participants in the standard consent group. Similarity, the average score of participants who were in the staff-produced video group was 1.79 (95% CI 0.99-2.59) points higher than the standard consent group. There was not a significant difference in the average scores of participants in the FAQ and standard consent groups (P=.75). The number of correct responses significantly differed by consent group for 11 of the 15 consent comprehension questions (Multimedia Appendix 5).

    From the mediation analysis, the direct effects of both the professional video (0.93, 95% CI 0.39-1.48) and the staff-produced video (0.99, 95% CI 0.42-1.56) were statistically significant when time spent on consent was held constant. Both video consent methods (relative to standard consent) also indirectly influence consent comprehension score through time spent on consent (professional video 95% CI 0.57-1.16, staff-produced video 95% CI 0.55-1.10; Table 3). This indicates that time on consent functions as a partial mediator of the effect of consent group on consent comprehension score.

    Table 3. Results of a mediation analysis of time spent on consent in the relationship between consent type and consent comprehension score among 665 men who have sex with men in a randomized trial of informed consent methods, United States, 2014.
    Consent groupTotal effectDirect effectRelative indirect effect through time spent on consent

    		CoefficientaSEe95% CICoefficientbSE95% CICoefficientcBoot SEBCd bootstrap 95% CI
    FAQf−0.320.30−0.91 to 0.26−0.270.27−0.80 to 0.25−0.050.11−0.27 to 0.16
    Professional video1.780.301.19 to 2.36.930.280.39 to 1.48.850.150.57 to 1.16
    Staff video1.800.311.19 to 2.42.990.290.42 to 1.56.810.140.55 to 1.10

    aTotal change in average consent comprehension score.

    bChange in consent comprehension score when time spent on consent is held constant.

    cChange in consent comprehension score when consent group is held constant and time spent on consent changes by the amount it differs by consent group.

    dBC: bias-corrected; number of bootstrap resamples=10,000.

    eSE: standard error.

    fFAQ: frequently asked questions.

    ‎
    Figure 2. Correlation of time spent on informed consent and consent comprehension score, by type of informed consent, among 665 MSM in a randomized trial of informed consent methods, United States, 2014.
    View this figure
    ‎
    Figure 3. Boxplot of consent comprehension scores, by type of informed consent, among 665 men who have sex with men in a randomized trial of informed consent methods, United States, 2014.
    View this figure

    Principal Findings

    The results of our study highlight the need to improve consent comprehension in Web-based HIV research among MSM. On average, participants were missing one-half (video consent group) to two-thirds (standard consent group) of the informed consent information deemed essential for understanding their participation in it. In particular, questions that focused on personal health information, contacts for questions about the study, or HIPAA authorization were answered correctly by less than half of the participants in every group (Multimedia Appendix 5).

    Our study also indicates that video-based methods of administering informed consent can lead to increased consent comprehension (compared with traditional consent forms) in Web-based research. Similar improvements in consent comprehension have been seen when video-based consent is used in clinical and surgical settings [32]. The 4 questions that did not see a difference in consent comprehension focused on benefits of participating (“What benefit can you or others reasonably expect from this research?”), voluntary participation (“True or false: Participation in this study is voluntary”), questions about the study (“Who can you contact if you have questions, concerns or complaints about the study?”), and revoking HIPAA authorization (“What will happen if you revoke your authorization?”). The latter 2 questions were incorrectly answered by the majority of participants in each consent group. Researchers may need to provide these questions early in the consent process or find other ways to highlight this information.

    Although the median amount of time spent on consent was much higher in the 2 video groups, many participants still passed through the consent page quickly. About half of the participants in the FAQ group spent less than 20.5 seconds on the consent page, indicating many people read very quickly or did not read every question. Whereas the videos may have held the attention of participants longer, half of the participants in the video group spent less than 2 minutes on the consent page, even though the videos were 3.5 minutes long. It is not clear why time spent on consent varied by race or ethnicity.

    Comprehension test scores were almost identical between the 2 video groups, indicating a professionally produced video does not provide additional understanding of informed consent over a staff-produced video. The FAQ group did not demonstrate higher consent comprehension compared with the standard group. This could be because the FAQ consent page still required participants to click on each question and read several lines of text, thus making it less accessible to participants who have lower levels of literacy or susceptible to respondents who click on each question but don’t read each one. In addition, the FAQ consent page did not prohibit users from “clicking through” to the survey, even if they did not open any of the FAQ topics.

    Time spent on the consent page was significantly correlated with consent comprehension across all groups. Although the correlation coefficient was largest among participants in the FAQ group, comprehension improved as time spent on the consent page (and, presumably, engagement with the consent materials) increased in all groups. The results from the mediation analysis indicate that time on consent does not entirely explain the association between consent group and consent comprehension, which means participants may understand the content in the video better than other methods. Our survey design did not prevent participants from clicking through to the next page before the entire video had finished, and a large number of participants in each group remained on the video page for a period of time that was shorter than the duration of the video. To ensure that all participants are presented the necessary consent information, researchers could design a survey that prevents the user from advancing to the next page until after the video has finished.

    Limitations

    There are several limitations to consider when interpreting these results. First, with all Web-based research, decisions made during the deduplication process can lead to biased results [28,33]. As a result of possible fraudulent activity, we had to exclude a large proportion of surveys from this analysis to be most conservative. Though other Web-based research studies that offer incentives have reported similar levels of potentially duplicate or fraudulent surveys, this was uncharacteristically high for many other Web-based research studies with MSM that use a similar deduplication process [33-35]. It is possible our process excluded legitimate responses or failed to exclude some fraudulent responses which may bias findings.

    Second, we tested comprehension of consent materials using a survey only among respondents who provided their consent to participate in the main study. We don’t know how many legitimate respondents clicked on the advertisement, encountered the consent materials, and then declined to participate. Although the data would allow us to determine the number of survey responses that landed on the consent page and the number that agreed to participate by each consent randomization group, those data include artificial and duplicate responses. It is possible that one of the consent methods was more likely to discourage participation in the study and if this behavior was also associated with consent, comprehension could lead to biased results. Similarly, all participants had the opportunity to download a PDF version of the written consent and we were not able to track which participants did that. It is possible, though unlikely, that participants in some groups were more likely to download the PDF version. Future implementation of alternative consent methods will also include this option in order to ensure that guidelines for the documentation of informed consent are met [27].

    Recruitment for this study targeted social media users and this convenience sample may not be representative of the general community of MSM. However, this type of recruitment is common in research among MSM and allows researchers to collect behavior data for a large number of MSM in a short period of time [36]. Finally, our sample is limited to MSM who report living within the United States, which limits the generalizability of these results to other groups or cultures affected by the HIV epidemic. Although some Web-based HIV risk and prevention research has been conducted in low or middle income countries [37,38], there are additional ethical concerns that must be considered when designing informed consent procedures in these settings [39,40].

    Despite these limitations, this research advances the development of innovative methods of administering informed consent in Web-based HIV prevention research among MSM. Although our study tested consent comprehension for a comparative analysis, researchers might also include a Web-based quiz of consent information and require correct answers before proceeding to the actual research [22]. Web-based HIV prevention researchers can also adapt the “teaching then testing” methods used in drug or alcohol abuse research [41].

    Future Research

    Future research should continue to improve on these innovative consent procedures and ensure they are optimized for use on mobile phone apps. About 64% of American adults now own a mobile phone that can access the Internet [42] and there is indication that MSM continue to adopt mobile phone technology faster than the general population [43] and are increasingly seeking sexual partners on mobile phone apps [7]. Social media mobile sites and mobile phone apps will continue to be a source of recruitment for HIV prevention research. In the current environment, using videos to administer informed consent are likely to be more effective than traditional, text-based approaches. People are currently used to interacting and receiving information through Web-based videos and matching research processes to these user experiences can improve how we conduct key activities. However, the continued use of studies similar to this one will help determine best practices as they arise and develop in this changing environment.

    Acknowledgments

    This work was funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R21HD074502) and the Emory University Center for AIDS Research (P30AI050409).

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Screenshot of frequently asked questions (FAQ) consent form used in a randomized trial of informed consent methods, United States, 2014.

    PNG File, 84KB
    ‎

    Multimedia Appendix 2

    Staff-produced video used in a randomized trial of informed consent methods, United States, 2014.

    MP4 File (MP4 Video), 52MB
    ‎

    Multimedia Appendix 3

    Professional video used in a randomized trial of informed consent methods, United States, 2014.

    MP4 File (MP4 Video), 40MB
    ‎

    Multimedia Appendix 4

    Consent comprehension questions used in a randomized trial of informed consent methods, United States, 2014.

    PDF File (Adobe PDF File), 306KB
    ‎

    Multimedia Appendix 5

    Number of correct responses to each consent comprehension question, by type of informed consent, in a randomized trial of informed consent methods, United States, 2014.

    PDF File (Adobe PDF File), 274KB
    1. CDC. HIV Surveillance Report   URL: http://www.cdc.gov/hiv/library/reports/surveillance/ [accessed 2016-09-22] [WebCite Cache]
    2. Beyrer C, Sullivan P, Sanchez J, Baral SD, Collins C, Wirtz AL, et al. The increase in global HIV epidemics in MSM. AIDS 2013 Nov 13;27(17):2665-2678. [CrossRef] [Medline]
    3. Beyrer C, Baral SD, van Griensven F, Goodreau SM, Chariyalertsak S, Wirtz AL, et al. Global epidemiology of HIV infection in men who have sex with men. Lancet 2012 Jul 28;380(9839):367-377 [FREE Full text] [CrossRef] [Medline]
    4. Sullivan PS, Hamouda O, Delpech V, Geduld JE, Prejean J, Semaille C, Annecy MSM Epidemiology Study Group. Reemergence of the HIV epidemic among men who have sex with men in North America, Western Europe, and Australia, 1996-2005. Ann Epidemiol 2009 Jun;19(6):423-431. [CrossRef] [Medline]
    5. Liau A, Millett G, Marks G. Meta-analytic examination of online sex-seeking and sexual risk behavior among men who have sex with men. Sex Transm Dis 2006 Sep;33(9):576-584. [CrossRef] [Medline]
    6. Garofalo R, Herrick A, Mustanski BS, Donenberg GR. Tip of the Iceberg: young men who have sex with men, the Internet, and HIV risk. Am J Public Health 2007 Jun;97(6):1113-1117. [CrossRef] [Medline]
    7. Grov C, Breslow AS, Newcomb ME, Rosenberger JG, Bauermeister JA. Gay and bisexual men's use of the Internet: research from the 1990s through 2013. J Sex Res 2014;51(4):390-409 [FREE Full text] [CrossRef] [Medline]
    8. Lewnard JA, Berrang-Ford L. Internet-based partner selection and risk for unprotected anal intercourse in sexual encounters among men who have sex with men: a meta-analysis of observational studies. Sex Transm Infect 2014 Jun;90(4):290-296. [CrossRef] [Medline]
    9. Perrin A, Duggan M. Pew Research Center. Americans' Internet Access: 2000-2015   URL: http://www.pewinternet.org/2015/06/26/americans-internet-access-2000-2015/ [accessed 2016-09-22] [WebCite Cache]
    10. Sullivan PS, Carballo-Diéguez A, Coates T, Goodreau SM, McGowan I, Sanders EJ, et al. Successes and challenges of HIV prevention in men who have sex with men. Lancet 2012 Jul 28;380(9839):388-399 [FREE Full text] [CrossRef] [Medline]
    11. Chiasson MA, Parsons JT, Tesoriero JM, Carballo-Dieguez A, Hirshfield S, Remien RH. HIV behavioral research online. J Urban Health 2006 Jan;83(1):73-85 [FREE Full text] [CrossRef] [Medline]
    12. Rhodes SD, Bowie DA, Hergenrather KC. Collecting behavioural data using the world wide web: considerations for researchers. J Epidemiol Community Health 2003 Jan;57(1):68-73 [FREE Full text] [Medline]
    13. Ross MW. Typing, doing, and being: sexuality and the internet. J Sex Res 2005 Nov;42(4):342-352. [CrossRef] [Medline]
    14. Sanchez T, Smith A, Denson D, Dinenno E, Lansky A. Internet-based methods may reach higher-risk men who have sex with men not reached through venue-based sampling. Open AIDS J 2012;6:83-89 [FREE Full text] [CrossRef] [Medline]
    15. Hernandez-Romieu AC, Sullivan PS, Sanchez TH, Kelley CF, Peterson JL, Del RC, et al. The comparability of men who have sex with men recruited from venue-time-space sampling and facebook: a cohort study. JMIR Res Protoc 2014;3(3):e37 [FREE Full text] [CrossRef] [Medline]
    16. Shi L, Wei C, McFarland W, Yan H, Li J, Raymond HF. Comparing samples of men who have sex with men recruited online and in venues, Jiangsu Province, China, 2013. LGBT Health 2016 Jun;3(3):238-242 [FREE Full text] [CrossRef] [Medline]
    17. Tsui HY, Lau JT. Comparison of risk behaviors and socio-cultural profile of men who have sex with men survey respondents recruited via venues and the internet. BMC Public Health 2010;10:232 [FREE Full text] [CrossRef] [Medline]
    18. Evans AR, Wiggins RD, Mercer CH, Bolding GJ, Elford J. Men who have sex with men in Great Britain: comparison of a self-selected internet sample with a national probability sample. Sex Transm Infect 2007 Jun;83(3):200-5; discussion 205 [FREE Full text] [CrossRef] [Medline]
    19. Pequegnat W, Rosser BR, Bowen AM, Bull SS, DiClemente RJ, Bockting WO, et al. Conducting Internet-based HIV/STD prevention survey research: considerations in design and evaluation. AIDS Behav 2007 Jul;11(4):505-521. [CrossRef] [Medline]
    20. Sullivan PS, Khosropour CM, Luisi N, Amsden M, Coggia T, Wingood GM, et al. Bias in online recruitment and retention of racial and ethnic minority men who have sex with men. J Med Internet Res 2011;13(2):e38 [FREE Full text] [CrossRef] [Medline]
    21. Sullivan PS, Grey JA, Simon RB. Emerging technologies for HIV prevention for MSM: what we have learned, and ways forward. J Acquir Immune Defic Syndr 2013 Jun 1;63 Suppl 1:S102-S107 [FREE Full text] [CrossRef] [Medline]
    22. Curtis BL. Social networking and online recruiting for HIV research: ethical challenges. J Empir Res Hum Res Ethics 2014 Feb;9(1):58-70 [FREE Full text] [CrossRef] [Medline]
    23. Bull SS, Breslin LT, Wright EE, Black SR, Levine D, Santelli JS. Case study: an ethics case study of HIV prevention research on Facebook: the Just/Us study. J Pediatr Psychol 2011;36(10):1082-1092 [FREE Full text] [CrossRef] [Medline]
    24. Purcell K. Pew Research Center. 2013. Online Video 2013   URL: http://www.pewinternet.org/2013/10/10/online-video-2013/ [accessed 2016-09-22] [WebCite Cache]
    25. Rosser BR, Gurak L, Horvath KJ, Oakes JM, Konstan J, Danilenko GP. The challenges of ensuring participant consent in internet-based sex studies: a case study of the Men's INTernet Sex (MINTS-I and II) studies. J Comput Mediat Commun 2009 Apr;14(3):602-626 [FREE Full text] [CrossRef] [Medline]
    26. Arora A, Rajagopalan S, Shafiq N, Pandhi P, Bhalla A, Dhibar DP, et al. Development of tool for the assessment of comprehension of informed consent form in healthy volunteers participating in first-in-human studies. Contemp Clin Trials 2011 Nov;32(6):814-817. [CrossRef] [Medline]
    27. US Department of Health Human Services. HHS. 2009. Code of Federal Regulations   URL: https://www.hhs.gov/ohrp/sites/default/files/ohrp/policy/ohrpregulations.pdf [accessed 2017-02-21] [WebCite Cache]
    28. Bauermeister J, Pingel E, Zimmerman M, Couper M, Carballo-Diéguez A, Strecher VJ. Data quality in web-based HIV/AIDS research: handling invalid and suspicious data. Field methods 2012 Aug 1;24(3):272-291 [FREE Full text] [CrossRef] [Medline]
    29. Von AL, Blum M, Hopper N, Langford J. CAPTCHA: Using hard AI problems for security. In: Advances in Cryptology — EUROCRYPT 2003. EUROCRYPT 2003. Lecture Notes in Computer Science, vol 2656. Berlin, Heidelberg: Springer; 2003:294-311.
    30. Tukey JW. Comparing individual means in the analysis of variance. Biometrics 1949 Jun;5(2):99-114. [Medline]
    31. Hayes AF, Preacher KJ. Statistical mediation analysis with a multicategorical independent variable. Br J Math Stat Psychol 2014 Nov;67(3):451-470. [CrossRef] [Medline]
    32. Schenker Y, Fernandez A, Sudore R, Schillinger D. Interventions to improve patient comprehension in informed consent for medical and surgical procedures: a systematic review. Med Decis Making 2011;31(1):151-173. [CrossRef] [Medline]
    33. Grey JA, Konstan J, Iantaffi A, Wilkerson JM, Galos D, Rosser BR. An updated protocol to detect invalid entries in an online survey of men who have sex with men (MSM): how do valid and invalid submissions compare? AIDS Behav 2015 Oct;19(10):1928-1937. [CrossRef] [Medline]
    34. Sanchez TH, Sineath RC, Kahle EM, Tregear SJ, Sullivan PS. The annual American men's internet survey of behaviors of men who have sex with men in the United States: protocol and key indicators report 2013. JMIR Public Health Surveill 2015;1(1):e3 [FREE Full text] [CrossRef] [Medline]
    35. Sanchez T, Zlotorzynska M, Sineath C, Kahle E, Sullivan P. The annual American men's internet survey of behaviors of men who have sex with men in the United States: 2014 key indicators report. JMIR Public Health Surveill 2016;2(1):e23 [FREE Full text] [CrossRef] [Medline]
    36. Sanchez T, Smith A, Denson D, Dinenno E, Lansky A. Developing a web-based HIV behavioral surveillance pilot project among men who have sex with men. Open AIDS J 2012;6:224-231 [FREE Full text] [CrossRef] [Medline]
    37. Stephenson R, Chard A, Finneran C, Sullivan P. Willingness to use couples voluntary counseling and testing services among men who have sex with men in seven countries. AIDS Care 2014 Feb;26(2):191-198 [FREE Full text] [CrossRef] [Medline]
    38. Chard AN, Finneran C, Sullivan PS, Stephenson R. Experiences of homophobia among gay and bisexual men: results from a cross-sectional study in seven countries. Cult Health Sex 2015;17(10):1174-1189 [FREE Full text] [CrossRef] [Medline]
    39. Chiu CJ, Menacho L, Fisher C, Young SD. Ethics issues in social media-based HIV prevention in low- and middle-income countries. Camb Q Healthc Ethics 2015 Jul;24(3):303-310. [CrossRef] [Medline]
    40. Krogstad DJ, Diop S, Diallo A, Mzayek F, Keating J, Koita OA, et al. Informed consent in international research: the rationale for different approaches. Am J Trop Med Hyg 2010 Oct;83(4):743-747 [FREE Full text] [CrossRef] [Medline]
    41. Aldridge J, Charles V. Researching the intoxicated: informed consent implications for alcohol and drug research. Drug Alcohol Depend 2008 Mar 1;93(3):191-196. [CrossRef] [Medline]
    42. Smith A. Pew Research Center. US smartphone use in 2015   URL: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/ [accessed 2016-09-22] [WebCite Cache]
    43. Grov C, Ventuneac A, Rendina HJ, Jimenez RH, Parsons JT. Perceived importance of five different health issues for gay and bisexual men: implications for new directions in health education and prevention. Am J Mens Health 2013 Jul;7(4):274-284 [FREE Full text] [CrossRef] [Medline]

    ‎
    CAPTCHA: Completely Automated Public Turing test to tell Computers and Humans Apart
    FAQ: frequently asked questions
    HIV: human immunodeficiency virus
    HIPAA: Health Insurance Portability and Accountability Act
    IP: internet protocol
    IQR: interquartile range
    MSM: men who have sex with men
    PDF: portable document format
    STI: sexually transmitted infection

    Edited by G Eysenbach; submitted 28.09.16; peer-reviewed by B Curtis, S Badawy, P Frew; comments to author 01.12.16; revised version received 07.12.16; accepted 08.02.17; published 06.03.17

    Copyright

    ©Eric William Hall, Travis H Sanchez, Aryeh D Stein, Rob Stephenson, Maria Zlotorzynska, Robert Craig Sineath, Patrick S Sullivan. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 06.03.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.