This paper is in the following e-collection/theme issue:

ISRII Theme Issue 2013 (14) Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (782) Recruitment of Research Participants (277) Depression and Mood Disorders; Suicide Prevention (1271) Perinatal Depression; Postpartum Depression; PPD (88) Perinatal Education for Parents (208)

Published on in Vol 16, No 1 (2014): January

Keywords to Recruit Spanish- and English-Speaking Participants: Evidence From an Online Postpartum Depression Randomized Controlled Trial

Keywords to Recruit Spanish- and English-Speaking Participants: Evidence From an Online Postpartum Depression Randomized Controlled Trial

Keywords to Recruit Spanish- and English-Speaking Participants: Evidence From an Online Postpartum Depression Randomized Controlled Trial

Authors of this article:

Alinne Z Barrera1 ;   Alex R Kelman1 ;   Ricardo F Muñoz1
Article Authors Cited by (27) Tweetations (4) Metrics

    Original Paper

    Palo Alto University, Palo Alto, CA, United States

    Corresponding Author:

    Alinne Z Barrera, PhD

    Palo Alto University

    1791 Arastradero Road

    Palo Alto, CA, 94304

    United States

    Phone: 1 650 433 3854

    Fax:1 650 433 3888

    Email: abarrera@paloaltou.edu


    Background: One of the advantages of Internet-based research is the ability to efficiently recruit large, diverse samples of international participants. Currently, there is a dearth of information on the behind-the-scenes process to setting up successful online recruitment tools.

    Objective: The objective of the study was to examine the comparative impact of Spanish- and English-language keywords for a Google AdWords campaign to recruit pregnant women to an Internet intervention and to describe the characteristics of those who enrolled in the trial.

    Methods: Spanish- and English-language Google AdWords campaigns were created to advertise and recruit pregnant women to a Web-based randomized controlled trial for the prevention of postpartum depression, the Mothers and Babies/Mamás y Bebés Internet Project. Search engine users who clicked on the ads in response to keyword queries (eg, pregnancy, depression and pregnancy) were directed to the fully automated study website. Data on the performance of keywords associated with each Google ad reflect Web user queries from February 2009 to June 2012. Demographic information, self-reported depression symptom scores, major depressive episode status, and Internet use data were collected from enrolled participants before randomization in the intervention study.

    Results: The Google ads received high exposure (12,983,196 impressions) and interest (176,295 clicks) from a global sample of Web users; 6745 pregnant women consented to participate and 2575 completed enrollment in the intervention study. Keywords that were descriptive of pregnancy and distress or pregnancy and health resulted in higher consent and enrollment rates (ie, high-performing ads). In both languages, broad keywords (eg, pregnancy) had the highest exposure, more consented participants, and greatest cost per consent (up to US $25.77 per consent). The online ads recruited a predominantly Spanish-speaking sample from Latin America of Mestizo racial identity. The English-speaking sample was also diverse with most participants residing in regions of Asia and Africa. Spanish-speaking participants were significantly more likely to be of Latino ethnic background, not married, completed fewer years of formal education, and were more likely to have accessed the Internet for depression information (P<.001).

    Conclusions: The Internet is an effective method for reaching an international sample of pregnant women interested in online interventions to manage changes in their mood during the perinatal period. To increase efficiency, Internet advertisements need to be monitored and tailored to reflect the target population’s conceptualization of health issues being studied.

    Trial Registration: ClinicalTrials.gov NCT00816725; http://clinicaltrials.gov/show/NCT00816725 (Archived by WebCite at http://www.webcitation.org/6LumonjZP).

    J Med Internet Res 2014;16(1):e6

    doi:10.2196/jmir.2999

    Keywords

    Internet intervention; prevention; depression, postpartum; research subject recruitment; women; Spanish speaking 


    The amount of Internet-based research in recent years has increased substantially [1-3] with Internet interventions showing significant promise in both alleviating symptoms and changing behavior [2]. Internet interventions are accessible at any time from any location, can be used anonymously, and may provide a much-needed service to users who feel marginalized or stigmatized [1,2,4]. For researchers, the Internet is an effective method of data collection [5] given that it is cost-effective, efficient at recruiting large sample sizes, and provides an opportunity for sensitive topics to be examined among hard-to-reach populations [6-9].

    Most Internet intervention studies have focused on depression and anxiety, but an emphasis on other health issues is gaining interest as further evidence of their effectiveness is demonstrated [2,10]. The characteristics of those who participate in Internet interventions are dependent on the target problems being addressed; however, online study samples repeatedly include a greater proportion of women than men. This trend is not surprising given that the percentage of women using the Internet has steadily increased in recent years [11], and women are reported to be the largest group of Web users searching for health information related to mental illness [12]. The higher incidence of depression among women relative to men is likely a contributing factor to the greater focus on mental health topics, such as depression, among female Internet users.

    Women and women-specific health issues are well suited for technology-based interventions with perinatal women garnering much of the attention by researchers (eg, [13-17]). Currently, there are countless websites that provide information on pregnancy and postpartum health. A recent review, however, revealed that the quality of the content exhibited on websites that focused on maternal mental health was partially correct or incomplete at best, and provided few self-help tools and resources to users [18]. Thus, there is a need to develop and empirically test Internet interventions specifically designed for women that address issues that uniquely affect women, such as postpartum depression (PPD). Computer- and Internet-based methods have been used successfully to increase awareness of PPD among adolescent parents [19], to recruit and screen women at risk for PPD [20-22], or to disseminate PPD psychoeducational materials to postpartum women and clinicians, all of whom have responded positively to this platform [22,23]. Recent studies have demonstrated support for behavioral activation [24,25] and cognitive behavioral [26] Internet interventions that aim to reduce depressive symptoms among depressed postpartum women. Two recent reports describe the design, feasibility, and acceptability of Internet interventions to prevent PPD [27,28]. The study presented in this report is based on data collected as part of the Mothers and Babies/Mamás y Bebés Internet Project, an Internet-based, 2-condition, pilot randomized controlled trial (RCT) designed to examine the efficacy of a Web-adapted mood management prevention intervention (Mothers and Babies Course/Curso de Mamás y Bebés) [29]. Participants were recruited, screened, and randomized to either the mood management Internet intervention or to an information brochure [23]. Preliminary analyses from the prevention trial are currently underway.

    Although the Internet offers added benefits and opportunities, there are significant concerns and barriers with recruitment to Internet-based research investigations [30]. Many of these investigations rely on traditional forms of recruitment such as face-to-face recruitment, flyers, community messages, and mass media announcements [31-33]. Earlier studies that used online recruitment strategies struggled with identifying and reaching the target population (eg, [6,34]), low rates of response [35], and possible ethical concerns with recruiting over the Internet [3]. The more common methods of online recruitment are search engine advertisements, social media sites, online forums, Web links, banners, and email lists. The chosen method of recruitment depends on the population being targeted and where their online presence is greatest. However, recent reports suggest that targeted search engine advertisements are the most successful at reaching a wide range of potential participants for Internet research and are cost-effective methods of online recruitment [6,8].

    What remains unclear in the literature is how researchers should frame their Internet advertisements so that they are time-efficient and cost-effective (ie, targeted to a specific population who are most likely to enroll in the study). Graham and colleagues [36] provide a good example of delineating the process of tailoring online advertisements to recruit Latino smokers to an online smoking cessation website. As part of their process, the researchers convened a multicultural panel of experts who informed the content of banner ads that were then empirically tested among Latino smokers. The researchers found that tailoring banner ads to portray the cultural value of familismo (emphasis on the importance of family influences) was preferable among Spanish-speaking Latino smokers who indicated that the issue of smoking was not just about them, but also about the impact on the relationships they valued [36]. Perhaps more importantly, this study demonstrated that a behind-the-scenes exploration of key factors that influence the success of an online recruitment method is valuable and needs to be empirically tested to maximize the strength of online recruitment. Furthermore, it is recommended that researchers continually monitor the behaviors of users once they are actively engaged in the study website [37].

    The current study adds to the growing body of literature describing online recruitment methods for Internet intervention studies. The primary goal was to examine the impact of Google AdWords campaigns to recruit Spanish- and English-speaking pregnant women to an Internet RCT to prevent PPD. Both the Spanish and English ad campaigns used identical keywords and were minimally modified throughout the recruitment period. Given that few Internet intervention studies have focused exclusively on pregnant women, we describe the demographic characteristics of participants who enrolled in the trial.


    Internet Search Ads

    Google AdWords sponsored link campaigns were created in Spanish and English to advertise the Mothers and Babies/Mamás y Bebés Internet Project, a RCT to test the efficacy of an Internet intervention for the prevention of PPD. Google ads consist of a 25-character headline and 2 additional 35-character text lines. In this study, the headlines were descriptive of perinatal depression, whereas the text lines described the study and provided the website link. There were 3 ad campaigns (sad, depressed, and postpartum depression) per language, each containing 2 distinctive ads for a total of 6 ads per language (see Figure 1). Ads were distributed worldwide, and the content and associated keywords of each ad remained unchanged throughout the course of the study recruitment period.

    The Google AdWords account management website allows the owner of the ads to manipulate the bidding cost and content of each ad. A Google AdWords grant awarded to our team allowed for a maximum bid of US $1.00 per click. Ad owners can view the number of times the ad is shown (impressions) and the number of Web users who click on the ad (clicks); owners can also use HyperText Markup Language (HTML) code to track the impact of the ads on obtaining the desired action by the target audience (conversion). In this study, the HTML code to detect a conversion was integrated into the study website’s consent page (ie, 1 conversion = 1 consent to participate). For the purposes of this paper, we will refer to conversions as “consents” given that it is the desired outcome of online recruitment. Additional information on Google AdWords procedures and metrics can be found elsewhere [38].

    ‎
    Figure 1. Google ads in Spanish and English for a prevention of postpartum depression trial.
    View this figure

    Participant Recruitment

    Web users who searched the Internet between February 3, 2009 and June 15, 2012 with keyword queries associated with the ads were able to click on them and link to the Mothers and Babies/Mamás y Bebés Internet Project’s website landing page (Figure 2); this initial page briefly described the study and invited those interested to complete the eligibility screener. The study website was fully automated with items that contained logic to determine participant flow through the study. The eligibility screener contained items to assess for eligibility criteria for the RCT: being female, pregnant, age 18 years or older, and interested in the study website for themselves. Eligible women were directed to the baseline assessment, which contained the University of California institutional review board–approved consent form. Participants “signed” the form online by clicking “Yes, I am interested in participating in this study” and by entering a unique password that was generated in real time and provided online to the user. Eligible participants who consented and completed the baseline assessment were considered enrolled in the study regardless of their depression status or stage of pregnancy. Enrolled participants received access to the intervention sites being evaluated and were invited to complete monthly follow-up assessments up to 6 months postpartum.

    ‎
    Figure 2. Screenshot of the study landing page.
    View this figure

    Measures

    Eligibility Screener

    Upon entering the study website, participants indicated their country of residence and preferred language and completed items to determine their eligibility to participate (age, gender, pregnancy status, how they planned to use the website materials). Eligible participants who entered a valid email address were directed to the baseline assessment.

    Baseline Assessment

    The initial page of the survey was the informed consent that participants were required to complete before proceeding through the study website. Consenting participants were asked to complete questionnaire items on demographic characteristics (eg, country of birth, ethnicity, race, education), Internet use (eg, previous use for health information), pregnancy history (eg, weeks pregnant, pregnancy history), and depression (eg, current symptoms).

    Depression status was determined by the Center for Epidemiologic Studies-Depression Scale (CES-D) [39] and the Major Depressive Episode (MDE) Screener-Current/Lifetime version [40]. The CES-D is a 20-item self-report instrument that assesses for the presence of depressive symptoms during the past week. Total scores range from 0-60, with higher scores indicating more severe depressive symptoms. The MDE Screener is an 18-item self-report questionnaire that assesses for the presence of 5 or more MDE symptoms experienced within a 2-week or longer period of time during the past 2 weeks (current MDE) or during any period (other than the past 2 weeks) in their lifetime (past MDE). To screen positive for a MDE, significant impairment, as defined by Criterion C of the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) must also be present [41]. The MDE Screener is a screening tool that has demonstrated good psychometric properties with diagnostic screeners and clinical interviews [42].

    Data Analysis

    Data on the performance of search engine ads were extracted from the Google AdWords management website [38]. Participant data for those who visited (eligibility), consented (started baseline), and enrolled (completed baseline) in the RCT were analyzed using SPSS for Windows 20.0 (IBM Corp, Armonk, NY, USA). Descriptive and chi-square analyses were conducted to examine group differences.


    Participant Enrollment

    Web users interested in the Spanish- and English-language Google ads, as defined by clicking on the ad (ie, clicks), were located in 183 countries and territories; 6745 pregnant women consented to participate with 2517 (37.32%) failing to enter any data in the baseline assessment. Data collected in the eligibility screener indicated that those who consented but did not provide baseline information were older (P<.001) and more likely to be English speakers (P<.001). Of the remaining 4228 participants, 1653 (24.51% of those who consented) failed to enroll in the intervention study because they did not complete the baseline assessment and, therefore, were excluded from further analyses. The enrolled sample consisted of 2575 participants who met the eligibility criteria, consented to participate, and completed the baseline assessment. Compared to those who consented to participate but did not enroll in the study, participants who enrolled were more likely to be Latino (P=.02), older (P<.001), employed (P=.007), and have attained higher levels of education (P<.001). Higher rates of past (11.60% vs 9.51%) and current MDE (19.06% vs 9.69%) were reported for enrolled participants relative to those who did not complete the baseline assessment (P<.001).

    Impact of Internet Search Ads

    The Google ads were active for approximately 40 months during which 12,983,196 impressions and 176,295 clicks were made in response to Web users’ search entries (see Table 1). In all, 25.09% of those who clicked on the ad entered the study website after reviewing the brief description provided on the study’s landing page. Over 60% of the total impressions (62.27%) were in response to Web users who searched in English, whereas 66.10% of the total clicks were made by those who searched in Spanish. In the Spanish campaign, 116,531 Web users of 4,898,063 who were presented with an ad (ie, an impression) clicked the study ad (2.38%). In the English campaign, there were 59,764 clicks in response to 8,085,133 impressions (0.74%). See Table 2 for detailed data on the performance of each ad campaign by language.

    The keywords with the greatest exposure (ie, highest clicks and impressions) and which generated the highest traffic in both languages were pregnancy/embarazo and pregnant/embarazada (Table 3). For all campaigns, 113,525 clicks of 4,819,662 impressions (2.35%) and 57,197 clicks of 7,836,146 impressions (0.73%) were generated by these keywords in the Spanish and English campaigns, respectively. In addition, those who queried with these keywords were the majority of those who consented to participate—94.53% (4718/4992) and 92.87% (1823/1963), respectively. In contrast, the rate of consent was highest for English-language speakers who searched with keywords that included a reference to both pregnancy and emotions (eg, mental health during pregnancy). A similar pattern was demonstrated in the Spanish campaign with keyword phrases such as “depresión despues del parto/depression after birth” or “embarazada y deprimida/pregnant and depressed” resulting in a greater proportion of consenting participants. Keywords that were descriptive of pregnancy without a reference to emotions (eg, months pregnant) also resulted in a higher rate of consent. This occurred with more frequency in the Spanish campaign (nueva mama/new mother, meses embarazo/months pregnant, semanas embarazo/weeks pregnant) than in the English campaign (months pregnant).

    Table 1. Participant online recruitment for the Mothers and Babies/Mamás y Bebés Internet Project.
    Ad or user behaviorLanguage, nTotal, n

    		SpanishEnglish
    Impressions (ads presented)4,898,0638,085,13312,983,196
    Clicks (clicks on ads)116,53159,764176,295
    Entered site (user proceeded beyond initial page)28,07416,15744,231
    Screened (user answered ≥1 eligibility items)11,620534916,969
    Eligible (user met eligibility criteria)8728373812,466
    Consented (user agreed to consent)477319726745
    Enrolled (user consented and completed baseline assessment)20125632575
    Table 2. Performance of Google AdWords campaigns from February 3, 2009 to June 15, 2012.
    Ad or user behaviorAd performance, n

    		SadDepressedPostpartum depression

    		SpanishEnglishSpanishEnglishSpanishEnglish
    Consent4855178213617912
    Clicks114,90253,6671574598755110
    Impressions4,794,4707,184,38492,280856,63411,31344,115
    Table 3. Spanish- and English-language ads with the highest-performing keyword(s).
    Ad character headlineKeyword(s)ConsentsCost per consent (US $)ClicksImpressions
    Spanish





    		Embarazada y tristeEmbarazo/embarazada (pregnancy/pregnant)4718$15.98113,5254,819,662

    		Triste durante embarazoMeses de embarazo (months pregnant)156$8.55213054,987

    		¿Embarazada y deprimida?Depresión en el embarazo (depression during pregnancy)109$2.8973911,716
    English





    		Sadness during pregnancyPregnancy/pregnant1823$25.7757,1977,836,146

    		Depression and pregnancyPregnancy depression82$11.99130197,488

    		Pregnancy and sadnessSad pregnant17$12.5229018,095

    		Pregnant and depressed?Depressed pregnant16$9.1219316,457

    Enrolled Participant Characteristics

    The final sample of enrolled participants (N=2575) consisted of pregnant women with a mean age of 28.16 years (SD 5.47) (see Table 4). Most completed study materials in Spanish (78.13%), were of Latino/Hispanic ethnic identity (77.89%), and self-identified their racial background as Mestizo descent (person of mixed Spanish and Indigenous ancestry; 35.16%) or European descent (30.43%). Most participants were married or living with a partner (66.99%), employed (59.90%), and college educated (70.37%).

    There were no differences in pregnancy status or depression history between the Spanish- and English-speaking participants. Participants were mostly in the second trimester of their pregnancy (mean 16.56 weeks, SD 9.59) and most (69.34%) did not meet DSM-IV criteria for a MDE. However, more English speakers met the criteria for a current MDE relative to Spanish speakers (20.04% vs 18.81%, respectively), whereas a higher percentage of Spanish speakers met criteria for a past MDE when compared to the English-speaking participants (11.88% vs 10.50%, respectively), although these differences were not statistically significant (P=.61). The mean CES-D score was elevated for both samples (mean 27.51, SD 13.77) with Spanish speakers endorsing slightly higher levels of depressive symptoms (P=.01). Participants reported accessing the Internet for information related to depression when not pregnant (29.31%) and during the perinatal period (42.64%). Spanish speakers accessed the Internet for depression information at greater proportions than English speakers did, especially when not pregnant or postpartum (32.05% vs 19.35%, P<.001).

    Group comparisons revealed that the Spanish-speaking women were more likely than English-speaking women to self-identify their ethnic background as Latino or Hispanic (91.45 vs 9.85%, P<.001). The racial descent varied significantly (P<.001) by language with the Spanish speakers mostly identifying as Mestizo (44.10%), European/Caucasian (32.74%), and other (16.42%), whereas the English speakers were of Asian (41.97%), European/Caucasian (21.60%), and African descent (19.75%). Similarly, Spanish speakers resided in Latin America and Spain whereas English speakers were primarily from India (Table 5). A greater proportion of English speakers were married or living with a partner (88.63%) compared to Spanish speakers (60.92%, P<.001). English speakers were more likely to have earned advanced educational degrees (28.08% vs 8.75%) whereas Spanish speakers were mostly comprised of college educated (73.38% vs 59.60%, P<.001) women.

    Table 4. Baseline characteristics of enrolled participants.
    Demographic itemsSpanish n=2012English n=563Total N=2575P
    Age, mean (SD)28.18 (5.68)28.07 (4.60)28.16 (5.47).69
    Latino/Hispanic (n=2343,a), n (%)1818 (91.45)39 (9.85)1857 (77.89)<.001
    Race (n=2343), n (%)


    		<.001

    		Mestizob819 (44.10)5 (1.03)824 (35.17)

    		European/Caucasian descent608 (32.74)105 (21.60)713 (30.43)

    		Other305 (16.42)73 (15.02)378 (16.13)

    		Asian descent11 (0.59)204 (41.97)215 (9.18)

    		African descent12 (0.65)96 (19.75)108 (4.61)

    		American Indian/Alaska Native102 (5.49)3 (0.62)105 (4.48)
    Married/live with partner (n=2572), n (%)1224 (60.92)499 (88.63)1723 (67.99)<.001
    Education (n=2528), n (%)


    		<.001

    		12 years or less353 (17.86)68 (12.32)421 (16.65)

    		University level/degree1450 (73.38)329 (59.60)1779 (70.37)

    		Advanced degree173 (8.75)155 (28.08)328 (12.97)
    Employed (n=2561), n (%)1217 (60.76)317 (56.81)1534 (59.90).09
    Weeks pregnant (n=2564), mean (SD)16.71 (9.62)16.01 (9.47)16.56 (9.59).13
    MDE history (n=2492), n (%)


    		.61

    		None1371 (69.31)357 (69.45)1728 (69.34)

    		Current MDE372 (18.81)103 (20.04)475 (19.06)

    		Past MDE235 (11.88)54 (10.50)289 (11.60)
    CES-D score (n=2475), mean (SD)27.86 (13.89)26.07 (13.22)27.51 (13.77).01
    Use Internet for depression information, n (%)




    		During perinatal period873 (43.39)225 (39.96)1098 (42.64).15

    		When not pregnant (n=2591)643 (32.05)107 (19.35)750 (29.31)<.001

    aValid percent reflects participants who completed the item.

    bMestizo: person of mixed Spanish and Indigenous ancestry.

    Table 5. Country of residence of enrolled participants.
    Country%a
    Spanish-speaking participants (2008/2012)

    		Chile16.30

    		Mexico15.24

    		Spain12.00

    		Argentina11.35

    		Colombia11.20

    		Venezuela11.00

    		Peru5.23

    		Bolivia3.43

    		Ecuador2.79

    		Dominican Republic2.54

    		Uruguay1.34

    		Paraguay1.14

    		29 Countries/territories with less than 1% each6.44
    English-speaking participants (562/563)

    		India36.30

    		South Africa12.81

    		Pakistan7.47

    		United Kingdom7.12

    		Iran2.67

    		Nigeria2.31

    		United States2.13

    		Ghana1.96

    		Kenya1.78

    		United Arab Emirates1.60

    		Saudi Arabia, Maldives, Uganda, Ireland (each)1.07

    		54 Countries/territories with less than 1% each19.57

    aBased on self-reported country of residence data.


    Principal Findings

    This study examined the impact of Spanish- and English-language Google AdWords to recruit an online sample of participants to an Internet RCT for the prevention of PPD. This method of recruitment was effective at exposing a large number of female Web users to the opportunities offered by the study website, such as learning skills to manage changes in mood during the transition to motherhood. During the 3 years that the ads were active, 176,295 Web users clicked on the advertised link and 12,466 pregnant women met eligibility criteria; 54% agreed to participate and 2575 (20.66%) completed sufficient baseline information to enroll in the study. These data suggest that pregnant women from around the world are interested in learning skills to manage their mood during and after pregnancy, are willing to use Web-based resources, and that the Internet is a viable means to reach them. Although a large number of women showed interest in joining the study, a large proportion exited the site immediately after clicking the ad or failed to continue once they were informed of participation details. A possible explanation for the former is the mismatch between what Web users queried and what they eventually found on the study’s landing page. That is, the information listed on the site did not correspond with what they were looking for or hoping to find when they searched, clicked, and initially visited the study website. The burden of participation may have been a factor to dissuade eligible participants from continuing their engagement in the study once they learned the details of participation from the consent form. Clearly, we need to make such Internet research sites more interesting and less burdensome so that more of those who are eligible not only consent to participate but also complete all facets of the study.

    High Performing Keywords

    Our examination of the highest-performing keywords for each of the Google AdWords campaigns revealed 3 primary findings. First, keywords that were more descriptive of the website content (eg, pregnant and depressed) resulted in a higher rate of consent to participate. This pattern of user behavior was evident in both the Spanish and English ads that referenced pregnancy and emotions, and suggests that the biggest gains occur when advertisements closely relate to the product being promoted. This approach is especially important online where there is heavy competition for the user’s attention and sustaining interest in one site over another is challenging. The specificity of pairing a reference to pregnancy and emotions in the keywords resulted in a higher consent rate among Spanish versus English users. This keyword type was also the most cost-effective in both languages relative to other high-performing keyword types (eg, pregnancy without referencing emotions). Cost-effectiveness by itself is not the most important measure of usefulness. The most cost-effective keywords yielded the least number of participants. To recruit the most participants in a reasonable period of time, we must be ready to pay for keywords that yield higher cost per consent. Given that most researchers are working within budgetary constraints, being able to maximize the cost of each consented individual is of high priority and importance. Furthermore, for the English ads there were more variations of this keyword type and, therefore, more related activity. Given that both the Spanish and English ads were identical in their content and associated keywords, this finding may suggest that Spanish-speaking women are less likely to search for information related to emotional health during pregnancy than English-speaking women or that the associated keywords do not reflect how Spanish-speaking women conceptualize perinatal distress. In fact, keywords that did not reference emotions or feelings but described pregnancy characteristics (eg, months pregnant) were very high activity ads and yielded a much larger number of impressions, clicks, consents, and enrolled participants. This suggests that women may be initially attracted by information about pregnancy that does not include information on emotional health during pregnancy. Thus, ads that are more generic may result in better recruitment outcomes for more women, of whom a subset may be subsequently intrigued by the focus on such issues as mood.

    Keyword Language Differences

    Previous reports have suggested that Latinos and Spanish speakers manifest psychological distress through physical complaints that can stem from stigma, cultural barriers, and differences in how psychological issues are conceptualized and experienced [43,44]. This study found that ads associated with broad search terms (eg, pregnancy) had greater reach and enrollment impact. This pattern of user behavior was evident in both languages and is consistent with female patterns of online search behavior which shows that women, more than men, search for health-related issues and that pregnancy-related queries are a major topic of inquiry among childbearing-aged female Web users [45,46]. There is a high likelihood that ads that were linked to broad keywords appeared in a large number of searches, many of which did not result in a visit to the study website. However, Spanish-speaking women who used broad keywords in their queries appeared to be more interested in the ads than English-speaking women as evidenced by the higher number of clicks and consents relative to the number of impressions which were almost twice as high in the English ads. We speculate that there may be fewer such Internet resources in Spanish than in English, therefore drawing a higher number of Spanish speakers to our website. These data indicate that broad, nonspecific keywords were effective at recruiting a large, diverse sample of pregnant women. However, this approach was more costly in the long term than more-specific keywords, which were more cost-effective per consent but resulted in fewer consenting participants. Researchers need to consider what their recruitment goals are and adjust their ad campaigns accordingly. To the best of our knowledge, this is the first fully automated prevention of PPD trial to recruit a large worldwide sample of Spanish- and English-speaking pregnant women. At the start of our recruitment period, we were uncertain of the potential interest of the study among pregnant women. Thus, our initial recruitment goal, which was to reach a large number of potential female Web users across the world, was accomplished.

    Participant Characteristic Outcomes

    A secondary aim of this study was to examine the characteristics of individuals who consented to participate in a Web-based trial. The ads attracted a mostly Spanish-speaking sample of women who resided in Latin American countries. This was an unexpected outcome of the recruitment efforts given the lower Internet penetration rate in this region of the world relative to Oceania/Australia, Europe, and North America [47]. Although the English-speaking sample was smaller, participants were equally diverse with the largest number of participants indicating that they resided in India and across different regions of Africa. The geographic location of participants mirrors the recent gender shift in Internet access by women in developed and developing nations. Furthermore, it highlights the potential need to develop new technology-based interventions targeting women from all around the world, especially those with current limited access to online resources but whose access is growing at a rapid rate (eg, 1310.8% growth in Latin American and the Caribbean in the past 10 years) [47].

    Spanish- and English-speaking participants were relatively similar on demographic characteristics with a few exceptions. The breakdown of the origin of our participants is important within a global mental health perspective, and especially critical when considering maternal mental health needs worldwide. To date, few resources exist in these regions of the world to address the day-to-day needs of pregnant women [48,49], especially as they relate to mental health issues during pregnancy. Furthermore, these women are accessing the Internet in regions of the world where the overall Internet penetration is lower and men significantly outnumber women in their use of the Internet [11,50]. This gender divide is likely to change as the global use of technology continues to grow.

    Participants in this study were well educated, with a greater proportion of English-language speakers possessing advanced degrees, whereas Spanish-language speakers had mostly attended university level or fewer years of formal education. It is clear that women with varied levels of education are using the Internet to obtain pregnancy-related information. Spanish- and English-speaking participants differed in their marital status with Spanish speakers more likely to indicate that they were single or without a current partner, thus potentially raising their risk for the onset of postpartum mood disorders [51]. Finally, the high rate of major depression found in this sample is of clinical concern and echoes a recent call-to-action to make maternal mental health a top priority [52]. Nearly one-third (30.66%) of our participants screened positive for a MDE sometime in their lifetime, yet our recruitment efforts did not seek or select participants based on their depression status. This indicates that the Internet is an effective tool to reach pregnant women who are depressed or at high risk for depression. Rates of perinatal depression range from 10%-25% [53,54], with point prevalence rates during pregnancy hovering around 12%-15% [51,55]. The higher rate of MDE found in this study may be attributed to measurement differences, the use of a self-report screening measure and not on clinician diagnostic information, the symptom overlap between pregnancy and depression, or to self-selection. Regardless, high rates of depressive symptoms were reported which is consistent with online samples of postpartum women [22]. These data highlight the need to develop multilingual, culturally considerate Internet resources for pregnant women regardless of depression risk. Many websites provide pregnancy information and many pregnant women are accessing the Internet to help guide their prenatal and postpartum health care [56]. However, the quality and reliability of the sites that include perinatal mental health topics is variable with a majority falling short of relaying complete and accurate information and even fewer providing practical skills to reduce symptoms [18]. Current studies that are in progress, such as the Mothers and Babies/Mamás y Bebés Internet Project directed by our team, as well as other prevention of PPD trials [27,28], stand to make a significant contribution to the availability of empirically tested Internet interventions to prevent PPD.

    Limitations

    Our findings were limited in several ways. First, the Google AdWords grant awarded to our team capped our bids at US $1.00. Second, this study solely reports on the use of minimally modified Google AdWords campaigns. We do not know how the ads would have performed if we had managed the campaigns based on their performance by adjusting the keywords, daily cost, or regional exposure. Third, given how the ads were set up and linked to the study website, we are unable to connect keyword types to participant characteristics. In order to understand how to target the ads to participants in different regions of the world who speak different languages and who conceptualize emotional distress differently, it would be beneficial to explore this further by recording the keyword entered by the user in their online search and examining their characteristics and behaviors on the study website. Finally, our data are only generalizable to Spanish and English speakers who use the Web to obtain information on the perinatal period.

    Conclusions

    This study contributes to the growing understanding of online recruitment for intervention trials. We provide evidence that pregnant women in many regions of the world are already seeking this type of health information and choosing to engage in Internet interventions by virtue of their participation in this study. Nonconsumable Internet interventions or those that can be reused with minimal added cost have the potential to reduce health disparities globally because they can be used an unlimited number of times from any region of the world without significant increases in cost [1]. The opportunity to share these resources with a wide range of individuals who may lack local resources requires the ability to design and test these interventions with global samples of individuals from diverse ethnic and cultural backgrounds. The health field would do well to use the potential of Internet interventions to increase health resources focused on maternal mental health and to reduce health disparities where there are few resources to protect childbearing women and advance the well-being of mothers and their babies.

    Acknowledgments

    The original RCT was funded by the National Institute of Mental Health (F32MH077371). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. Additional funding was provided by a Robert Wood Johnson Health Disparities Seed Grant (Nancy Adler, PI) to Dr Barrera, funding from the University of California Committee on Latino Research which funded the UCSF/SFGH Latino Mental Health Research Program (Muñoz, PI), and funds granted to the UCSF Internet World Health Research Center (Muñoz, Director) from the Department of Psychiatry at SFGH while under the direction of Robert Okin, MD. A very special thanks to Google, Inc for awarding the team (Muñoz, PI) an AdWords grant, which provided the ability to recruit worldwide using Google-sponsored links. Online data were collected by using the Illume survey system developed by DatStat, Inc. We thank Drs Lynn Abeita and Eunice Kim for feedback on earlier drafts of this paper. Lastly, we are thankful to all the women who participated in this research.

    Conflicts of Interest

    None declared.

    1. Muñoz RF. Using evidence-based Internet interventions to reduce health disparities worldwide. J Med Internet Res 2010;12(5):e60 [FREE Full text] [CrossRef] [Medline]
    2. Griffiths KM, Christensen H. Review of randomised controlled trials of Internet interventions for mental disorders and related conditions. Clinical Psychologist 2006;10(1):16-29. [CrossRef]
    3. Im EO, Chee W. Recruitment of research participants through the Internet. Comput Inform Nurs 2004;22(5):289-297. [Medline]
    4. Proudfoot J, Klein B, Barak A, Carlbring P, Cuijpers P, Lange A, et al. Establishing guidelines for executing and reporting Internet intervention research. Cognitive Behaviour Therapy 2011 Jun;40(2):82-97. [CrossRef]
    5. Temple EC, Brown RF. A comparison of Internet-based participant recruitment methods: Experiences engaging the hidden population of cannabis users in research. Journal of Research Practice 2011;7(2):1-20.
    6. Jones RB, Goldsmith L, Williams CJ, Kamel Boulos MN. Accuracy of geographically targeted Internet advertisements on Google AdWords for recruitment in a randomized trial. J Med Internet Res 2012;14(3):e84 [FREE Full text] [CrossRef] [Medline]
    7. Morgan AJ, Jorm AF, Mackinnon AJ. Internet-based recruitment to a depression prevention intervention: lessons from the Mood Memos study. J Med Internet Res 2013;15(2):e31 [FREE Full text] [CrossRef] [Medline]
    8. Ramo DE, Hall SM, Prochaska JJ. Reaching young adult smokers through the Internet: comparison of three recruitment mechanisms. Nicotine Tob Res 2010 Jul;12(7):768-775 [FREE Full text] [CrossRef] [Medline]
    9. Reips UD, Lengler R. The Web Experiment List: a Web service for the recruitment of participants and archiving of Internet-based experiments. Behav Res Methods 2005 May;37(2):287-292. [Medline]
    10. Cuijpers P, van Straten A, Andersson G. Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med 2008 Apr;31(2):169-177 [FREE Full text] [CrossRef] [Medline]
    11. Boland AL, Pauline MM, Vollman A. comScore. 2010 Jun 30. Women on the Web: How women are shaping the Internet   URL: http:/​/www.​comscore.com/​Insights/​Presentations_and_Whitepapers/​2010/​Women_on_the_Web_How_Women_are_Shaping_the_Internet [accessed 2013-12-17] [WebCite Cache]
    12. Lamberg L. Online empathy for mood disorders: patients turn to Internet support groups. JAMA 2003 Jun 18;289(23):3073-3077. [CrossRef] [Medline]
    13. Anderson GD, Nelson-Becker C, Hannigan EV, Berenson AB, Hankins GD. A patient-centered health care delivery system by a university obstetrics and gynecology department. Obstet Gynecol 2005 Jan;105(1):205-210. [CrossRef] [Medline]
    14. Britt DW, Norton JD, Hubanks AS, Navidad SA, Perkins RJ, Lowery CL. A two-period assessment of changes in specialist contact in a high-risk pregnancy telemedical program. Telemed J E Health 2006 Feb;12(1):35-41. [CrossRef] [Medline]
    15. Dennis CL, Kingston D. A systematic review of telephone support for women during pregnancy and the early postpartum period. J Obstet Gynecol Neonatal Nurs 2008;37(3):301-314. [CrossRef] [Medline]
    16. Hod M, Kerner R. Telemedicine for antenatal surveillance of high-risk pregnancies with ambulatory and home fetal heart rate monitoring--an update. J Perinat Med 2003;31(3):195-200. [CrossRef] [Medline]
    17. Homko CJ, Santamore WP, Whiteman V, Bower M, Berger P, Geifman-Holtzman O, et al. Use of an Internet-based telemedicine system to manage underserved women with gestational diabetes mellitus. Diabetes Technol Ther 2007 Jun;9(3):297-306. [CrossRef] [Medline]
    18. Moore D, Ayers S. A review of postnatal mental health websites: help for healthcare professionals and patients. Arch Womens Ment Health 2011 Dec;14(6):443-452. [CrossRef] [Medline]
    19. Logsdon MC, Barone M, Lynch T, Robertson A, Myers J, Morrison D, et al. Testing of a prototype Web based intervention for adolescent mothers on postpartum depression. Appl Nurs Res 2013 Aug;26(3):143-145. [CrossRef] [Medline]
    20. Le HN, Perry DF, Sheng X. Using the Internet to screen for postpartum depression. Matern Child Health J 2009 Mar;13(2):213-221. [CrossRef] [Medline]
    21. Kim H, Bracha Y, Tipnis A. Automated depression screening in disadvantaged pregnant women in an urban obstetric clinic. Arch Womens Ment Health 2007;10(4):163-169. [CrossRef] [Medline]
    22. Maloni JA, Przeworski A, Damato EG. Web recruitment and Internet use and preferences reported by women with postpartum depression after pregnancy complications. Arch Psychiatr Nurs 2013 Apr;27(2):90-95. [CrossRef] [Medline]
    23. Wisner KL, Logsdon MC, Shanahan BR. Web-based education for postpartum depression: conceptual development and impact. Arch Womens Ment Health 2008 Dec;11(5-6):377-385. [CrossRef] [Medline]
    24. O'Mahen HA, Richards DA, Woodford J, Wilkinson E, McGinley J, Taylor RS, et al. Netmums: a phase II randomized controlled trial of a guided Internet behavioural activation treatment for postpartum depression. Psychol Med 2013 Oct 23:1-15. [CrossRef] [Medline]
    25. O'Mahen HA, Woodford J, McGinley J, Warren FC, Richards DA, Lynch TR, et al. Internet-based behavioral activation--treatment for postnatal depression (Netmums): a randomized controlled trial. J Affect Disord 2013 Sep 25;150(3):814-822. [CrossRef] [Medline]
    26. Danaher BG, Milgrom J, Seeley JR, Stuart S, Schembri C, Tyler MS, et al. Web-based intervention for postpartum depression: Formative research and design of the MomMoodBooster program. JMIR Res Protoc 2012;1(2):e18 [FREE Full text] [CrossRef] [Medline]
    27. Haga SM, Drozd F, Brendryen H, Slinning K. Mamma mia: a feasibility study of a Web-based intervention to reduce the risk of postpartum depression and enhance subjective well-being. JMIR Res Protoc 2013;2(2):e29 [FREE Full text] [CrossRef] [Medline]
    28. Jones BA, Griffiths KM, Christensen H, Ellwood D, Bennett K, Bennett A. Online cognitive behaviour training for the prevention of postnatal depression in at-risk mothers: a randomised controlled trial protocol. BMC Psychiatry 2013 Oct 17;13(1):265 [FREE Full text] [CrossRef] [Medline]
    29. Muñoz RF, Le H, Ippen CG, Diaz MA, Urizar GG, Soto J, et al. Prevention of postpartum depression in low-income women: Development of the Mamás y Bebés/Mothers and Babies course. Cognitive and Behavioral Practice 2007 Feb;14(1):70-83. [CrossRef]
    30. Kraut R, Olson J, Banaji M, Bruckman A, Cohen J, Couper M. Psychological research online: report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. Am Psychol 2004;59(2):105-117. [CrossRef] [Medline]
    31. Alexander GL, Divine GW, Couper MP, McClure JB, Stopponi MA, Fortman KK, et al. Effect of incentives and mailing features on online health program enrollment. Am J Prev Med 2008 May;34(5):382-388 [FREE Full text] [CrossRef] [Medline]
    32. Bowen A, Williams M, Horvath K. Using the Internet to recruit rural MSM for HIV risk assessment: sampling issues. AIDS Behav 2004 Sep;8(3):311-319 [FREE Full text] [CrossRef] [Medline]
    33. Hedman E, Ljótsson B, Rück C, Furmark T, Carlbring P, Lindefors N, et al. Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation. Computers in Human Behavior 2010 Jul;26(4):736-740. [CrossRef]
    34. Jones RB, Goldsmith L, Hewson P, Williams CJ. Recruitment to online therapies for depression: pilot cluster randomized controlled trial. J Med Internet Res 2013;15(3):e45 [FREE Full text] [CrossRef] [Medline]
    35. Woodford J, Farrand P, Bessant M, Williams C. Recruitment into a guided Internet based CBT (iCBT) intervention for depression: lesson learnt from the failure of a prevalence recruitment strategy. Contemp Clin Trials 2011 Sep;32(5):641-648. [CrossRef] [Medline]
    36. Graham AL, Fang Y, Moreno JL, Streiff SL, Villegas J, Muñoz RF, et al. Online advertising to reach and recruit Latino smokers to an Internet cessation program: impact and costs. J Med Internet Res 2012;14(4):e116 [FREE Full text] [CrossRef] [Medline]
    37. Crutzen R, Roosjen JL, Poelman J. Using Google Analytics as a process evaluation method for Internet-delivered interventions: an example on sexual health. Health Promot Int 2013 Mar;28(1):36-42. [CrossRef] [Medline]
    38. Google. Google AdWords: Online advertising by Google   URL: http://www.google.com/adwords/?sourceid=awo&subid=ww-et-awhp_nelsontest3_nel_p&clickid [accessed 2013-12-17] [WebCite Cache]
    39. Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1977 Jun 01;1(3):385-401. [CrossRef]
    40. Muñoz RF. Preventing major depression by promoting emotion regulation: A conceptual framework and some practical tools. Int J Ment Health Promot 1998 Sep(Inaugural Issue):23-40 [FREE Full text]
    41. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition). Washington, DC: American Psychiatric Association; 2000.
    42. Vázquez FL, Muñoz RF, Blanco V, López M. Validation of Muñoz's mood screener in a nonclinical Spanish population. European Journal of Psychological Assessment 2008 Jan 21;24(1):57-64. [CrossRef]
    43. Nadeem E, Lange JM, Miranda J. Perceived need for care among low-income immigrant and US-born black and Latina women with depression. J Womens Health (Larchmt) 2009 Mar;18(3):369-375 [FREE Full text] [CrossRef] [Medline]
    44. Hulme PA. Somatization in Hispanics. J Psychosoc Nurs Ment Health Serv 1996 Mar;34(3):33-37. [Medline]
    45. Fox S. Health topics: 80% of Internet users look for health information online. Washington, DC: Pew Internet & American Life Project; 2011 Feb 01.   URL: http://www.pewInternet.org/~/media//Files/Reports/2011/PIP_Health_Topics.pdf [accessed 2013-12-03] [WebCite Cache]
    46. Pogorelc D. Google’s top health searches for: cancer, pregnancy symptoms and…hemorrhoid?. 2012.   URL: http:/​/medcitynews.​com/​2012/​12/​googles-top-health-searches-for-2012-cancer-pregnancy-symptoms-and-hemorrhoid/​ [accessed 2013-09-27] [WebCite Cache]
    47. Internet World Stats. 2012. World Internet Users Statistics Usage and World Population Stats   URL: http://www.Internetworldstats.com/stats.htm [accessed 2006-01-15] [WebCite Cache]
    48. Nour NM. An introduction to maternal mortality. Rev Obstet Gynecol 2008;1(2):77-81. [Medline]
    49. Belizán JM, Cafferata ML, Belizán M, Tomasso G, Chalmers B. Goals in maternal and perinatal care in Latin America and the Caribbean. Birth 2005 Sep;32(3):210-218. [CrossRef] [Medline]
    50. Dalberg Global Development Advisors. Women and the Web: Bridging the Internet gap and creating new global opportunities in low and middle-income countries.: Intel Corporation; 2012.   URL: http://www.intel.com/content/dam/www/public/us/en/documents/pdf/women-and-the-web.pdf [accessed 2013-12-17] [WebCite Cache]
    51. Le Strat Y, Dubertret C, Le Foll B. Prevalence and correlates of major depressive episode in pregnant and postpartum women in the United States. J Affect Disord 2011 Dec;135(1-3):128-138. [CrossRef] [Medline]
    52. Rahman A, Surkan PJ, Cayetano CE, Rwagatare P, Dickson KE. Grand challenges: integrating maternal mental health into maternal and child health programmes. PLoS Med 2013;10(5):e1001442 [FREE Full text] [CrossRef] [Medline]
    53. O'hara MW, Swain AM. Rates and risk of postpartum depression—a meta-analysis. Int Rev Psychiatry 1996 Jan;8(1):37-54. [CrossRef]
    54. Fisher J, Cabral de Mello M, Patel V, Rahman A, Tran T, Holton S, et al. Prevalence and determinants of common perinatal mental disorders in women in low- and lower-middle-income countries: a systematic review. Bull World Health Organ 2012 Feb 1;90(2):139G-149G [FREE Full text] [CrossRef] [Medline]
    55. Gavin NI, Gaynes BN, Lohr KN, Meltzer-Brody S, Gartlehner G, Swinson T. Perinatal depression: a systematic review of prevalence and incidence. Obstet Gynecol 2005 Nov;106(5 Pt 1):1071-1083. [CrossRef] [Medline]
    56. Romano AM. A changing landscape: Implications of pregnant women's Internet use for childbirth educators. J Perinat Educ 2007;16(4):18-24 [FREE Full text] [CrossRef] [Medline]

    ‎
    CES-D: Center for Epidemiological Studies-Depression
    HTML: HyperText Markup Language
    MDE: major depressive episode
    PPD: postpartum depression
    RCT: randomized controlled trial

    Edited by D Mohr; submitted 30.09.13; peer-reviewed by T Donker, B Danaher; comments to author 04.11.13; revised version received 06.12.13; accepted 07.12.13; published 09.01.14

    Copyright

    ©Alinne Z Barrera, Alex R Kelman, Ricardo F Muñoz. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.01.2014.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.