Bias in Online Recruitment and Retention of Racial and Ethnic Minority Men Who Have Sex With Men

Introduction

Men who have sex with men (MSM) are the most heavily impacted risk group for HIV in the United States [1]. Indeed, MSM is the only US risk group in which HIV incidence has been increasing since 2000 [2]. Similar to the United States, there is an international resurgence in HIV infections among MSM in industrialized countries in North America, Europe, and Australia [3]. Among MSM in the United States, men of color—especially younger men of color—are experiencing the most dramatic increases in HIV surveillance case reports [4,5]. At the same time, the number of HIV prevention interventions tested with MSM is low relative to the proportion of the US HIV epidemic represented by this vulnerable population [6].

The role of the Internet in the MSM epidemic—as a facilitator of transmission or as an opportunity for HIV prevention—is complex and unclear. In the results of most studies reported to date, men who meet sex partners on the Internet are also men who are more likely to engage in high risk sex [7-10] although the causality of this relationship is not clear. Regardless of causality, the Internet seems to hold promise as a way to reach men with significant sexual risk behaviors. We also know that there have been some early suggestions that Internet-based HIV prevention interventions hold promise as a strategy for accessing and increasing protective behaviors among MSM [11].

While the Internet represents a promising intervention strategy, significant gaps remain in our ability to optimally leverage the Internet to evaluate and implement HIV prevention interventions for MSM. One of the most significant barriers is that MSM of color, who bear the greatest risk for HIV infection in the United States, are systematically underrepresented in nearly all HIV prevention Internet studies to date (Table 1). Black men have been historically underenrolled by 29% to 84%, and Hispanic men underenrolled by 6% to 89%. Setting quotas for enrollment by race/ethnicity is one way to address the problem but raises additional concerns about bias among minority men who actually enroll (and how they are different from the unsurveyed men who do not enroll). Thus, underenrollment of MSM is a problem for Internet-based HIV prevention research and is ideally addressed by finding ways to obtain comparable rates of enrollment and retention for minority men.

Our study addressed two aspects of this problem: (1) challenges in recruiting MSM of color (ie, the extent to which MSM of color click through banner advertisements to enroll in research studies) and (2) challenges in retaining MSM of color (ie, the extent to which MSM of color complete an online survey once they begin taking it). Our goals were to quantify the extent of underrecruitment of MSM of color, to identify factors associated with recruitment, and to determine the extent of underretention of MSM of color. To address these questions, we conducted an Internet survey of MSM recruited through banner advertisements shown on MySpace.com, a popular online social networking site.

Methods

Recruitment and Participation

Internet-using MSM were recruited from March 19, 2009, through April 16, 2009, through selective placement of banner advertisements on MySpace.com. During the recruitment period, advertisements were displayed to MySpace members based on self-reported demographic MySpace profile information. Exposures were made to males 18 years of age and over logging into MySpace whose profile indicated a residence in the United States and who reported their sexual orientation as gay, bisexual, or unsure. Participants who clicked through the banner advertisements were taken to an eligibility screener for an Internet-based survey.

A total of six banner advertisements were used, all with similar text and graphical design (Figure 1). Of the six, two of the advertisements presented a white male model, two presented a black male model, and two presented an Asian male model. Asian male models were used as controls since our hypotheses centered primarily on black/white differences. Data on the number of advertisement exposures and “click-throughs” were collected for each combination of model race, respondent race/ethnicity, respondent sexual orientation (gay, bisexual, unsure), and level of education (less than high school, high school, some college, college graduate and higher). Participant data on race/ethnicity, sexual orientation, and level of education for the “click-through” analysis were obtained from participants’ MySpace.com profiles. Therefore, these categories, which were preset options on MySpace.com, are not identical to those that are presented in the analysis of survey completion, where we used the data from participants’ survey responses (and which contained different category options for sexual identity and education level).

‎View this figure

Participants referred to the survey site after clicking through the banner advertisement were first screened for eligibility. Participants were eligible for the survey if they were male, 18 years of age and over, and reported at least one male sex partner in the last 12 months. Eligible participants were provided an online informed consent document, and consenting participants were passed into an online survey. In the survey, participants were asked for relevant demographic information as well as questions about the use of the Internet to meet sex partners, recent sexual risk behaviors, use of technology, HIV testing history, and interest in specific, new HIV prevention interventions. Participants did not receive an incentive to participate in the baseline survey. The study was reviewed and approved by the institutional review board (IRB) of Emory University.

Analysis

Characteristics Associated With Click-Through of Advertisements

Internet advertisement exposures and click-throughs were totaled and stratified by race/ethnicity of the MySpace user. Within each stratum of race/ethnicity, data were summarized by race of the model displayed in the advertisement, education, and sexual orientation of the respondent as reported in the MySpace profiles. We utilized multivariable logistic regression to model factors associated with clicking through the banner advertisement, stratified by race/ethnicity of the MySpace user. With click-through as the outcome and race of the model, education, and sexual identity as the independent factors of interest, we calculated adjusted odds ratios (ORs) with 95% confidence intervals (CIs) for each racial/ethnic group.

Characteristics Associated With Survival in the Survey

To determine whether there was any bias in completion of the survey, we used survival analysis to model factors associated with survey dropout. We evaluated survey dropout by race, age group, sexual orientation, education, geographic region, and gender of sexual partners in the last 12 months (male only or male and female) as reported in the online survey. Participants were assigned a complete or incomplete status for each page of the survey. A complete status was assigned if the participant answered at least 50% of the questions on the survey page. Failure (dropout from the survey) was defined as at least two consecutive incomplete pages. Kaplan–Meier survival methods and log-rank tests were used to evaluate survey dropout by individual demographic factors. For multivariable analysis, Cox proportional hazards regression was used to identify variables associated with survey dropout. Models were constructed separately for men who had only male partners and those who had both male and female partners; the reason for this was that men with male and female partners were asked additional questions about their female partners, which resulted in a longer survey. Thus, it was not possible to equitably assess dropout in one model that included both groups. The multivariable model findings are reported as hazard ratios (HR) with 95% CIs. All data analysis was completed using SAS version 9.2 (SAS Institute, Cary, North Carolina, USA).

Results

The recruitment and enrollment of study participants is described in Figure 2. A total of 8,257,271 MySpace advertising impressions over a 29-day period resulted in 30,559 (0.37%) click-throughs to our eligibility screener. Of those potential participants, 35% (10,688 of 30,559) abandoned the page without attempting the survey. Of the 16,896 respondents who saved or submitted their survey responses, 98% (16,597 of 16,896) completed the three sex, age, and partner questions used to determine eligibility. Approximately 30% (4916 of 16,597) of respondents who completed the screening questions were ineligible. Ineligibility was most often due to reporting no male sex partners in the past year (3394 of 4916 potential participants or 69%), followed by reporting an age of less than18 years (876 of 4916 potential participants or 18%). Thus, a total of 11,681 men were eligible for participation and were referred to informed consent. Another 20% (2333 of 11,681) of participants discontinued the survey at this point, not completing the informed consent process. Of those who completed the consent process, the vast majority, 96% (9005 of 9348), consented to participate.

‎View this figure

Of the 9005 consenting participants, 39% (3473 of 9005) were white non-Hispanic, 14% (1293 or 9005) were black non-Hispanic, and 31% (2809 of 9005) were Hispanic. The median age of participants was 21, with 68% (6157 of 9005) of participants aged 18 to 24. Most reported having only male sex partners, and 12% (1117 of 9005) reported both male and female partners. In all, 69% (6176 of 8409 reporting) of the men self-identified as homosexual or gay, while 23% (2077 of 8409 reporting) self-identified as bisexual, and less than 1% (60 of 8409 reporting), as heterosexual or straight. Over half of participants (4815 of 8357 reporting) attended at least some college, and about a third (2914 of 8357) completed high school or a general equivalency diploma (GED). Over 60% of participants (4843 of 8063 reporting) had unprotected anal intercourse (UAI) with a male sex partner in the past 12 months, while over a quarter of participants (1775 of 6623 reporting) indicated they had never been tested for HIV.

Characteristics Associated With Click-Through of Advertisements

The overall click-through rate for the banner advertisements was 0.37% (30,599 clicks from 8,257,271 impressions); this varied by race, education, sexual identity, and race of model. In general, black, Hispanic, and men of other races were less likely to click through on the banner advertisements than were white men, with click-through rates of 0.36% (6474 clicks from 1,785,088 impressions) for black men, 0.35% (8873 clicks from 2,510,434 impressions) for Hispanic men, and 0.33% (8271 clicks from 2,497,487 impressions) for men of other races versus 0.48% (6995 clicks from 1,464,262 impressions) for white men. Men with more than a high school level of education were more likely to click through, as were men who self-identified as gay or bisexual. Click-through rates were higher when the models displayed were black (0.33% or 9006 clicks from 2,751,415 impressions) or Asian (0.48% or 13,241 clicks from 2,753,684 impressions) versus white (0.30% or 8312 clicks from 2,752,172 impressions) (data not shown).

Because our primary research question was focused on how to optimize banner advertisement to achieve comparable click-through rates for racial/ethnic minority men, we conducted multivariable analyses stratified by race/ethnicity (Table 2). Among all nonwhite racial/ethnic groups analyzed, men with greater than a high school education had higher odds of click-through compared with those with less education. Black, white, and Hispanic men that self-identified as gay or bisexual also had higher odds of click-through. Hispanic men had higher odds of click-through when exposed to Asian models; black men had higher odds of click-through when exposed to black models. White men had higher odds of click-through when shown an Asian model, but lower odds of click-though when shown a black model.

Characteristics Associated With Survival in the Survey

Of the 9005 participants, 69% (6258 of 9005) completed the entire survey, and 31% (2747 of 9005) dropped out of the survey after starting. Demographic characteristic of participants by survey completion status are provided in Table 3, and hazards of dropping out of the survey from the Cox proportional hazards model are described in Table 4. Figures 3 and 4 display the Kaplan-Meier curves for survival in the survey by participant race/ethnicity and sexual orientation, respectively, among participants reporting having only male partners in the past 12 months. Among white study participants, 77% (2670 of 3473) completed the entire survey compared with 66% (849 of 1293) of black participants and 71% (1987 of 2809) of Hispanic participants. Among those with only male sex partners in the last year, black men had about a 60% higher hazard of dropping out than white men, and Hispanic men had an increased hazard of about 30%. Among those with both male and female partners in the last year, both black men and Hispanic men had an increased hazard of dropout (30% and 50% increased hazard, respectively) compared with white men. Only 58% (35 of 60) of men who self-identified as heterosexual completed the survey compared with 75% (4653 of 6176) of men who self -identified as gay, resulting in heterosexually identified men having over twice the hazard of dropping out of the survey compared with men identifying as homosexual. Participants who reported less than a high school level of education had an increased hazard of dropout compared with those with greater than a high school level of education. There was no significant difference in hazard of dropout related to age or geographical region.

‎View this figure

‎View this figure

Discussion

Our results illustrate two levels at which bias may occur in online HIV prevention research with MSM. Based on differential click-through rates, our data suggest that online surveys recruited through banner advertisements similar to ours may underrepresent black and Hispanic MSM, MSM with less education, and MSM who do not identify as gay. Once engaged in the survey, further bias is introduced by differential dropout during the survey; in this case, our data suggest that we were more likely to lose as participants men who were black, Hispanic, or of other nonwhite races, as well as men who were less educated, and men who identify as heterosexual.

Our approach furthers knowledge in this field because we used an Internet venue for advertisement in which we could collect data on each exposure to the banner advertisement (each impression) along with the race, education, and sexual identity of the person who saw the advertisement (as reported in their profile). This allowed us, for the first time to our knowledge, to calculate race-, education-, and sexual identity-specific click-through rates and model how the characteristics of Internet users who saw the banner advertisements related to their likelihood of clicking through that advertisement. The results indicate that the black men who saw our advertisements were much more likely to click through an advertisement displaying a black model than one displaying a white model, whereas the white men who saw our advertisements were much less likely to click through an advertisement with a black model than a white model. All of the racial/ethnic groups were more likely to click through on the advertisements displaying the Asian models compared with the advertisements displaying the white models; however, it is unclear why the Asian model advertisements yielded the highest click-through.

These data suggest that it may be possible to mitigate the differentially low click-through of black and Hispanic MSM by changing the racial/ethnic composition of the banner advertisements that they are shown. For example, black Internet users may be shown, as a group, banner advertisements that predominantly depict black models. It is important to emphasize that, to our understanding, an important goal is to achieve comparable click-through rates by racial or ethnic group. It will always be possible to set quotas to ensure that equal numbers of men in each racial/ethnic group are included, but the central question must be what biases attend the inclusion of those specific men. In other words, if 25% fewer black men are clicking through than white men, how are those men who are not clicking through different from the ones who are? Striving for comparable click-through rates by race/ethnicity is one way to address this concern.

Although MSM of color were also more likely to drop out of our survey prematurely, our data are less definitive in identifying solutions to this source of bias. However, we propose several possible reasons for our findings and options to reduce differential dropout. First, it is well documented that black Americans are less likely to have access to private, high speed Internet than are white Americans [19]. We speculate that, if black non-Hispanic participants were taking the current survey either without high-speed Internet access or in a public place (for example, a library), they may have discontinued participation early because of frustration with page load times or because of privacy concerns as questions became more personal. Possible solutions include optimizing our Internet interfaces for nonbroadband use or allowing an option to complete surveys via text messaging, as black and Hispanic Americans exceed white Americans on mobile phone ownership and text message use [20]. Another possible explanation for our results is that participants with less education drop out sooner because their literacy skills are lower. Possible solutions are attempting to lower the reading level of the survey and adding an option for text-to-speech service to the survey website. An additional explanation may be that the sensitive nature of the questions (ie, sexual history and HIV testing information) may have led some participants to discontinue their participation. An informal analysis of the pages from which men dropped out of the survey did not indicate that dropouts were clustered on a certain page; therefore, if the content of questions was a cause for dropout it was more likely due to the general nature of the topics and not to a specific question. However, it is important to note that bisexual- and straight-identified participants, although they had already reported having male sex partners, tended to drop out when the series of questions about male partners began. In future versions, we may choose to ask questions about male sex partners later in the survey, after questions about female sex partners.

Although not the primary focus of our study, we also note that social networking sites may be fruitful places for HIV prevention research among MSM. Most previous studies have recruited from sites that were either more explicitly gay-identified [13,16] or that were sex-seeking sites [11]. Using MySpace, we found that we were able to recruit a large number of behaviorally eligible MSM quickly. Also, the demographic and behavioral characteristics of these men suggest that they were at least as high risk and in need of prevention services as men recruited in “‘real-world” venues. The greatest expansion in the US HIV epidemic among MSM is among MSM aged 13 to 24. The median age of our consenting participants recruited through a social networking site was 21; for comparison, the average age of participants in the 2008 National HIV Behavioral Surveillance System (NHBS) was 32 [21]. Also, our participants were relatively high risk: 60% reported unprotected anal intercourse with a male sex partner in the past 12 months compared with 47% of 2003-2005 NHBS respondents [22]. Use of prevention services among our participants was low, with 28% never having been tested for HIV versus 10% of 2008 NHBS respondents who had never been tested [21]. Thus, recruiting through a social networking site geared toward younger users yielded a young, high-risk sample with low levels of prior HIV testing—an ideal population for HIV prevention research and, eventually, Internet-delivered interventions.

There are a number of limitations to our study. First, our participants are not a representative sample, and our conclusions cannot be generalized to all MSM users of MySpace, to users of other social networking sites, or to US MSM more generally. Because we only showed our banner advertisements to men who identified themselves in their MySpace profiles as gay, bisexual, or unsure, we did not display banner advertisements to men who may have had male sex partners but who identified themselves as straight. Further, we could not verify that certain self-reported characteristics of our respondents—for example, male sex, or race/ethnicity—were correct. However, we recruited men based on their MySpace profile, and therefore we would only refer men to the eligibility screener if they had identified those characteristics in their profile, independent of the research study. Our data were subject to recall bias because of our 12-month recall period and to potential social desirability bias. There is also concern about participants taking the survey multiple times. In our case, we think that this was not common for two reasons. First, participants could only enter the survey site by being referred from a link in the banner advertisement. Therefore, a participant could only participate a second time if he was shown the banner advertisement a second time—a low probability event. Also, we did not allow multiple surveys to be completed from the same Internet protocol (IP) address, which would further require that a participant change his IP address, or take the survey a second time from a different computer.

Our study has several implications for those conducting online HIV prevention research with MSM. First, we have demonstrated that it is possible in some Internet settings to collect denominator data, characterize nonrespondents (ie, men who clicked on the advertisement but did not answer any survey questions), and model factors associated with participation. Second, our data suggest that social networking sites may offer an appropriate alternate recruitment venue to sex-seeking sites, especially for studies that seek to enroll younger, higher risk MSM. Finally, our results suggest the need for further work to attempt to reduce disparities in click-through rates for MSM of color and improve retention in surveys of MSM of color, of those with less education, and of those who are not gay-identified. These are difficult problems to address, but the epidemiology of the MSM epidemic in the United States demands that we address them, and increasing technological capacities of Web services where MSM congregate offer new opportunities to apply the highest standards of prevention research to online HIV prevention studies.