This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289) Quality/Credibility of eHealth Information and Trust Issues (531) Infodemiology and Infoveillance (1349) Infoveillance, Infodemiology, Digital Disease Surveillance, Infodemic Management (1041) Pregnancy Information, Education and Lifestyle Interventions (300) Public (e)Health, Digital Epidemiology and Public Health Informatics (725) Medicine 2.0: Social Media, Open, Participatory, Collaborative Medicine (1641)

Published on in Vol 25 (2023)

This is a member publication of University of Bristol (Jisc)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/44912, first published .
Evaluating Messaging on Prenatal Health Behaviors Using Social Media Data: Systematic Review

Evaluating Messaging on Prenatal Health Behaviors Using Social Media Data: Systematic Review

Evaluating Messaging on Prenatal Health Behaviors Using Social Media Data: Systematic Review

Authors of this article:

Nessie Felicia Frennesson1 Author Orcid Image ;   Cheryl McQuire2, 3 Author Orcid Image ;   Saher Aijaz Khan4 Author Orcid Image ;   Julie Barnett5 Author Orcid Image ;   Luisa Zuccolo4, 6, 7 Author Orcid Image
Article Authors Cited by Tweetations (8) Metrics

    Review

    1Tobacco and Alcohol Research Group, School of Psychological Science, University of Bristol, Bristol, United Kingdom

    2Centre for Public Health, Bristol Medical School, University of Bristol, Bristol, United Kingdom

    3National Institute for Health and Care Research, School for Public Health Research, Newcastle, United Kingdom

    4Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom

    5Department of Psychology, University of Bath, Bath, United Kingdom

    6Health Data Science Centre, Human Technopole, Milan, Italy

    7Medical Research Council Integrative Epidemiology Unit, University of Bristol, Bristol, United Kingdom

    Corresponding Author:

    Nessie Felicia Frennesson, MSc

    Tobacco and Alcohol Research Group

    School of Psychological Science

    University of Bristol

    12a Priory Road

    Bristol, BS8 1TU

    United Kingdom

    Phone: 44 1173746633

    Email: felicia.frennesson@bristol.ac.uk


    Background: Social media platforms are increasingly being used to disseminate messages about prenatal health. However, to date, we lack a systematic assessment of how to evaluate the impact of official prenatal health messaging and campaigns using social media data.

    Objective: This study aims to review both the published and gray literature on how official prenatal health messaging and campaigns have been evaluated to date in terms of impact, acceptability, effectiveness, and unintended consequences, using social media data.

    Methods: A total of 6 electronic databases were searched and supplemented with the hand-searching of reference lists. Both published and gray literature were eligible for review. Data were analyzed using content analysis for descriptive data and a thematic synthesis approach to summarize qualitative evidence. A quality appraisal tool, designed especially for use with social media data, was used to assess the quality of the included articles.

    Results: A total of 11 studies were eligible for the review. The results showed that the most common prenatal health behavior targeted was alcohol consumption, and Facebook was the most commonly used source of social media data. The majority (n=6) of articles used social media data for descriptive purposes only. The results also showed that there was a lack of evaluation of the effectiveness, acceptability, and unintended consequences of the prenatal health message or campaign.

    Conclusions: Social media is a widely used and potentially valuable resource for communicating and evaluating prenatal health messaging. However, this review suggests that there is a need to develop and adopt sound methodology on how to evaluate prenatal health messaging using social media data, for the benefit of future research and to inform public health practice.

    J Med Internet Res 2023;25:e44912

    doi:10.2196/44912

    Keywords

    acceptability; design; development; effectiveness; health behavior; health messaging; messaging; prenatal health; prenatal; social media data; social media; tool 


    Background

    Accurate and easily understandable information on prenatal health is important for maternal and child health. There are several poor health outcomes among newborns that are affected by prenatal health behaviors. For example, alcohol consumption during pregnancy can result in fetal alcohol spectrum disorders, leading to lifelong developmental disabilities and multiple comorbidities [1,2], and smoking during pregnancy can, among other outcomes, lead to low birthweight [3].

    The internet is used by people all around the world. As many as 5.48 billion of the world’s population are reported to be using the internet [4]. In terms of social media use, Facebook alone had 2.4 billion users in 2019 [5]. Facebook is not the only social media platform that is widely used. In 2021, 71% of Americans aged between 18 and 29 years reported using Instagram, with 65% reporting using Snapchat [6]. Gender differences are also noticeable in social media usage. Men are said to use social media to gain information, while women are reported to use social media to keep up with relationships [7]. This shows the importance of ensuring that information targeting women online reaches the intended audience.

    Moreover, there is a growing recognition that social media plays a key role in society today, not only being an important means of communication but at the same time being influential in health decisions [8]. Social media has been proven to spread information and create engagement quickly [9]. Social media can also be used by researchers and public health agencies to gain real-time insight into people’s attitudes and behaviors, as well as identify how people perceive public health messages [10]. It is, therefore, timely and important to investigate how and to what extent social media data have been used to evaluate prenatal health messaging. Zhu et al [11] reported that the majority of their participants, expecting mothers, had a good experience seeking information related to pregnancy on social media, and that had a positive impact on them. They reported, for example, that the participants could find support on social media and that it reduced anxiety and loneliness. Moreover, Baker and Yang [12] reported that 84% of their respondents, new mothers, thought of friends on social media as being supportive during pregnancy. Not only can social media be a source of information, but it can also help pregnant women feel more in control over their pregnancies and the health decisions they are making [13]. It has been shown that most pregnant women use the internet to search for information at least once a month during pregnancy [14]. Ford and Alwan [15] showed that around 20% of the pregnant women who responded to the survey used social media to find information on vaccinations during pregnancy. Other information that is searched for related to pregnancy is, for example, nutrition, medication, and antenatal care [14]. It is also important to note that there are concerns about searching for health-related information online. According to the World Health Organization (WHO) [16], an infodemic, meaning too much information or false information, could lead to worsening health outcomes. Misinformation about health is present on all different types of social media platforms [17].

    With the recent COVID-19 pandemic, pregnant women were advised to minimize their in-clinic visits and instead meet with their health care providers over the phone or through video calls [18]. In the absence of in-person consultations, social media could have been the initial and only source of information pertaining to health and well-being; therefore, it can play a substantial role in influencing health behaviors during pregnancy.

    Since social media is commonly used as a source of information, it is important that the information available on social media is both accepted by the intended audience and effective in changing the targeted health behavior. Moreover, it is important that the information reaches the audience that it intends to. Others have investigated how health messages delivered on social media have been evaluated [19]. However, this did not include a specific investigation of messages relating to prenatal health. Given the variability in the quality of information and the misinformation known to occur on social media, and related concerns that health messaging can potentially lead to increased anxiety among pregnant people, there is a need to review the impact, acceptability, effectiveness, and unintended consequences of prenatal health messaging and campaigns. Therefore, this systematic review aims to address this important gap in research by reviewing the current research on how prenatal health messaging and campaigns have been evaluated to date using social media data.

    Aims

    The specific research questions in this systematic review were:

    1. What research has been carried out on evaluating official messaging and campaigns on prenatal health behavior using social media data?
    2. Which methods have been used to evaluate the impact of official messaging and campaigns on prenatal health behavior using social media data?
    3. To what extent has previous research looked at the impact, effectiveness, acceptability, and unintended consequences of official prenatal health messaging and campaigns?
    4. What is the methodological quality of previous research that has evaluated official messaging and campaigns related to prenatal health?

    Search Strategy

    This systematic review was registered with PROSPERO (International Prospective Register of Systematic Reviews; CRD42022315743). The search strategy involved multiple steps. First, search terms were developed by the review team based on keywords from the relevant literature, expert knowledge, and consultation with library service colleagues. The following words were identified as suitable for the search: “Pregnan*,” “Prenatal,” “F?etal,” “Antenatal,” “Gestation*,” “Matern*,” “Mother,” “Health,” “Behavi*,” “Evaluati*,” “Efficac*,” “Effective*,” “Messag*,” “Campaign,” “Communication,” “Media,” “Social Media,” “Social Network Sites.”

    Searches were conducted in the following databases: PubMed (MEDLINE), PsycINFO, Web of Science, the Cochrane Library, and Scopus. The search string was adjusted depending on the database, together with relevant Medical Subject Headings terms and Boolean operators. The final search string for PubMed can be found in Multimedia Appendix 1. The search was carried out on March 23, 2022, with a final search conducted on June 14, 2022. Restrictions were made to only include articles in English or Swedish. Second, Google Scholar was searched using the same sets of search terms, screening the first 100 results and including the results if relevant. Third, manual searches of reference lists in the relevant literature were carried out. Final, Google Scholar was used to forward search for further relevant articles. When necessary, the article author or authors were contacted when full texts were not available.

    Inclusion and Exclusion Criteria

    Studies were included if they fit the following criteria: (1) the study had to describe an official message or campaign (for example, from a recognized health organization) targeting prenatal health behaviors. Evaluation of peer advice was not included, (2) the study had to target pregnant women or the general public, (3) the study had to use social media data (eg, likes, comments, and shares) to evaluate the message or campaign, and (4) the study had to be available in English or Swedish. Online as well as offline messaging or campaigns were included in the review, as long as they were evaluated using social media data. Studies with interventions related to preconception behavior were excluded.

    Both quantitative and qualitative studies were considered, and eligible studies included systematic reviews, randomized controlled trials, nonrandomized controlled studies, observational studies, natural experiments, and qualitative studies. Moreover, gray literature sources, such as reports, blogs, and conference proceedings, were considered.

    Overlapping Studies

    A total of 2 included studies analyzed the same campaign [20,21]. After careful consideration among reviewers, the decision was made to keep them both in the review since they used different methods for analysis.

    Data Extraction and Analysis

    The identified articles were assessed according to the inclusion criteria listed above. The first reviewer (NFF) screened the study titles and abstracts. A random selection of 10% (2/13) of the full texts was assessed by the third reviewer (SAK) for eligibility as agreed on in the review protocol. Reasons for exclusion were documented throughout the full text screening process. There was no discrepancy, and consensus was reached between the reviewers, so there was no need for consultation by the other reviewers at this stage.

    The first reviewer (NFF) independently extracted the data agreed on in the preregistered review protocol (PROSPERO CRD42022315743), including sample population characteristics (eg, geographical location) and message or campaign characteristics (eg, message or campaign topic, objectives of the message or campaign, and message or campaign delivery). The full list of the data extracted can be found in Multimedia Appendix 2. The third reviewer (SAK) repeated the data extraction for 10% of the articles to ensure accuracy. In the case of full texts missing or when clarification was needed, the article author or authors were contacted. The studies were summarized narratively.

    Articles described and measured outcomes differently. However, the following definitions were applied to all articles: Effectiveness was defined as a change of attitudes, behavior, or knowledge relating to the behavior targeted. Acceptability was defined as either qualitatively to what level the campaign or message was agreed with or rejection of the campaign or message, or quantitatively as the click-through rate as previously used in social media data analysis [22]. Unintended consequences were defined as any consequence or effect that was not intended for the messaging or campaign (eg, stigmatizing certain behaviors or individuals). Impact was defined as each paper’s social media metrics (eg, likes and shares). Only outcomes measured using social media data were evaluated in this systematic review.

    A content analysis with a thematic synthesis approach was used to summarize qualitative evidence. Quantitative summary statistics were used when applicable. No subgroup analysis could be performed as there were not a sufficient number of studies identified.

    Quality Assessment of the Studies

    Despite the increasing number of studies using social media data, no relevant validated quality assessment tool could be identified. There are several important factors to take into account when assessing the quality of social media data that have been discussed by several scholars [23-25]. Therefore, after considering previous discussions around social media data quality, we make use of the assessment tool developed by Golder et al [26] and further refine it to the following criteria:

    1. Bias: for example, if applicable, has sampling bias been accounted for? If intervention or campaign, has it been reported whether the intended population has been reached?
    2. Choice of data: for example, can social media data help answer the study objectives? How well has the selection of specific social media channels been argued for?
    3. Data extraction: for example, are the methods that have been used to extract the social media data reported? If there is an extraction of comments or tweets, has there been a double extraction of the data?
    4. Statistical analysis: for example, is the analysis method appropriate for the objective of the study?

    The studies were rated as either low quality, high quality, or cannot tell. The studies were assessed based on their social media data and methodology; no other eventual data source or methodology was used. The first reviewer (NFF) assessed all included studies, with the third reviewer (SAK) assessing 3 of the studies. Any disagreement was discussed between the reviewers.


    Result of the Search

    The initial screening identified 1404 records, with another 6 identified by searching the first 100 Google Scholar results, as well as with backward and forward reference searches. This process resulted in a total of 11 articles meeting all inclusion criteria for this review. Figure 1 illustrates a flow diagram of the selection process. After full text screening, 1 article was excluded due to not targeting prenatal health specifically [27], and another article was excluded due to not using social media data [28].

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analysis) flowchart depicting the selection of studies.

    Characteristics of Included Studies

    A total of 3 eligible studies targeted alcohol consumption during pregnancy [20,21,29], and 2 of the studies targeted maternal vaccination [30,31]. Another 2 studies were on the topics of weight gain, nutrition, and the importance of seeking prenatal care [32,33]. The remaining studies targeted seeking prenatal care if experiencing decreased fetal movements [34], healthy weight gain during pregnancy [35], maternal smoking [36], and nutrition during pregnancy [37]. All the included articles were published between 2016 and 2022. A majority (n=8) of the articles aimed to describe or evaluate campaigns and messaging [29,30,32-37], with the others aiming to understand the discourse around maternal vaccination [31], to understand how Facebook advertisements can be used to communicate public health messages [20], and to investigate the suitability of the dynamic transactional model to communicate public health [21].

    Further characteristics of the included studies are shown in Table 1.

    Table 1. Characteristics of included studies.
    Study, yearHealth behaviorAim of the paperCampaign or messaging populationGeographical location
    Bazzo et al [29], 2016Alcohol consumption during pregnancyDescribe a campaign intended to raise awareness about FASDaGeneral populationWorldwide
    Bonnevie et al [33], 2021aWeight gain, nutrition, and seeking prenatal careDescribe the campaign “Growing and Glowing” and its effectivenessBlack pregnant womenUnited Statesb
    Bonnevie et al [32], 2021bWeight gain, nutrition, and seeking prenatal careDescribe the campaign “Strong Beautiful Future” and its feasibilityBlack pregnant womenUnited Statesc
    Carlson et al [30], 2019Maternal influenza vaccinationEvaluate a campaign intended to increase awareness of maternal influenza vaccination and determine sources of information and attitudesPregnant womenAustralia
    Chan et al [34], 2021Seeking prenatal care if experiencing decreased fetal movementsDescribe the campaign “Movement Matter” and its effectivenessWomenAustralia
    Graham et al [35], 2019Healthy weight gain during pregnancyDescribe the implementation of a campaign intended to promote healthy weight gain during pregnancyWomenNot country specific
    Martin et al [31], 2020Maternal vaccinationUnderstand the discourse around maternal vaccination on social mediaPregnant women15 countriesd
    Miller et al [36], 2022Smoking during pregnancyEvaluate dissemination efficacy and examine reach and engagement with a Facebook campaign about risks of smoking during pregnancyPregnant womenUnited Statese
    Parackal et al [20], 2017Alcohol consumption during pregnancyUnderstand the communication process of Facebook advertisement communicating public health messagesWomenNew Zealand
    Parackal et al [21], 2021Alcohol consumption during pregnancyInvestigate the suitability of the Dynamic Transactional Model to communicate public health messagesWomenNew Zealand
    Verduci et al [37], 2021Nutrition during pregnancy and early lifeDescribe the introduction of a new tool of eHealth communication to communicate health messagesGeneral populationItaly

    aFASD: fetal alcohol spectrum disorders.

    bHillsborough County, Florida.

    cOrange County, Florida.

    dAustralia, Brazil, Canada, France, Germany, India, Italy, Mexico, Panama, South Africa, South Korea, Spain, Taiwan, the United Kingdom, and the United States.

    eNew Jersey, Massachusetts, Georgia, North Carolina, Louisiana, and Kentucky.

    Methodology of Included Studies

    Table 2 presents the included studies’ methodology. Each of the included articles used data obtained from social media channels. A wide range of metrics were used to evaluate the messaging and campaigns, making comparisons between them difficult. Around 6 of the articles used social media data mainly to present social media metrics and metadata. A majority of the studies (n=7) [29-34,37] used more than 1 social media channel as a source of data. The following social media channels were used:

    • Facebook (n=11) [20,21,29-37].
    • Twitter (subsequently rebranded X; n=6) [29,31-34,37].
    • Instagram (n=6) [30-34,37].
    • YouTube (n=1) [32].
    Table 2. Methodology and data source of included studies.
    StudySocial media channelSocial media data analysisAdditional data source used in paperSurvey population
    Bazzo et al [29], 2016Facebook and TwitterSocial media metricsaSurveySurvey to the partner organizations that distributed the campaign
    Bonnevie et al [33], 2021aInstagram, Facebook, and TwitterSocial media metricsSurvey and website dataWomen, aged 18-65 years living in Hillsborough Country Florida, United States
    Bonnevie et al [32], 2021bInstagram, Facebook, Twitter, and YouTubeSocial media metricsSurvey and website dataWomen, aged 18-65 years, living in Orange County Florida, United Statesb
    Carlson et al [30], 2019Instagram and FacebookSocial media metricsNews articles and surveyEnglish speaking pregnant women, aged 18 years or older attending an antenatal clinic in western Sydney, Australia
    Chan et al [34], 2021Instagram, Facebook, and TwitterSocial media metricsSurveyClinicians working from one of the designated clinics in Victoria, Australia. Pregnant women at ≥28 weeks gestation receiving care at one of the same clinics
    Graham et al [35], 2019FacebookSocial media metricsN/AcN/A
    Martin et al [31], 2020Instagram, Twitter, blogs, and forumsSocial media metrics, discourse analysis, topic analysis, and stance analysisNoneN/A
    Miller et al [36], 2022FacebookSocial media metrics and content analysisNoneN/A
    Parackal et al [20], 2017FacebookSocial media metrics, thematic analysis, logistic regression, and sentiment analysisNoneN/A
    Parackal et al [21], 2021FacebookCluster analysis and regression analysisNoneN/A
    Verduci et al [37], 2021Instagram, Facebook, and TwitterSocial media metricsWebsite and app dataN/A

    aSocial media metrics includes number of impressions, users, interactions, likes, tweets, reach, engagement, views, clicks, followers, reactions, and shares.

    bOnly Black women were included in the analysis.

    cN/A: not applicable.

    A total of 2 articles reported studies that used paid influencers to further spread their message and campaigns on social media platforms [32,33]. Martin et al [31] extracted social media data to analyze the discourse around maternal vaccination using stance, discourse, and topic analysis. Miller et al [36] conducted a content analysis. Parackal et al [20] performed a cluster analysis by using text mining techniques on comments extracted from a campaign against drinking alcohol during pregnancy as well as performing a logistic regression to find relationships between meaning-making themes and the message to abstain from alcohol. Parackal et al [21] used the same campaign to investigate the suitability of the dynamic transaction model when using social media to communicate health messages.

    Effectiveness, Acceptability, Impact, and Unintended Consequences

    Table 3 presents the study’s main results in terms of effectiveness, acceptability, impact, and unintended consequences.

    Table 3. Summary of effectiveness, acceptability, impact, and unintended consequences of included studies.
    StudyMeasurement of effectivenessMeasurement of acceptabilityMeasurement of impactMeasurement of unintended consequences
    Bazzo et al [29], 2016Not measuredNot measuredImpressionsa, usersb, interactionsb, likes, and tweets during launch weekNot measured
    Bonnevie et al [33], 2021aNot measuredNot measuredAverage monthly impressionsa, average daily reach (number of people who have seen the content), average monthly engagement (number of likes, comments, shares, video views, and clicks on posts)Not measured
    Bonnevie et al [32], 2021bNot measuredNot measuredAverage monthly impressionsa, average daily reach (number of unique people who have seen the content), average monthly engagement (number of likes, comments, shares, video views, and clicks on posts)Not measured
    Carlson et al [30], 2019Not measuredClick-through rate from Facebook to the NSWc Health websiteAverage monthly impressionsa, average daily reach (number of unique people who have seen the content), average monthly engagement (number of likes, comments, shares, video views, and clicks on posts)Not measured
    Chan et al [34], 2021Not measuredNot measuredImpressionsa, estimated combined reachb., likes, and followersNot measured
    Graham et al [35], 2019Not measuredClick-through rate on Google Ads to HPHCd website and click-through rate on Facebook ads to HPHC websiteImpressionsa, reactions, shares, and commentsNot measured
    Martin et al [31], 2020Not measuredStance analysisNot measuredSemantic network, topic analysis, and stance analysis
    Miller et al [36], 2022Not measuredContent analysisImpressionsa, reach (Number of unique individuals), likes, reactions, shares, comments, and video viewsNot measured
    Parackal et al [20], 2017Not measuredThematic analysisLikes, comments, shares, and viewsNot measured
    Parackal et al [21], 2021Not measuredNot measuredNot measuredNot measured
    Verduci et al [37], 2021Not measuredParents’ comments on social media channels and blogsViews, reachb., interactionsb, likes, and sharesNot measured

    aFacebook impressions are measured as to how often the content is displayed to the audience [38].

    bNot specified how it was measured.

    cNSW: New South Wales.

    dHPHC: Healthy Parents, Healthy Children.

    Effectiveness

    None of the included studies measured the effectiveness of the messaging or campaign.

    Acceptability

    A total of 6 studies provided data on acceptability, which was heterogeneously defined. A total of 2 studies used click-through rates (defined as the number of clicks per the number of impressions) as a measure of acceptability, showing rates ranging from 0.4% to 5.8% [30,35].

    A total of 3 more studies used a variety of other methods for assessing acceptability, including thematic analysis and stance analysis. In their study of maternal vaccination discourse, Martin et al [31] showed that the messaging of around 42% of the tweets could be classified as having a promotional stance toward maternal vaccination, with the rest being either neutral, ambiguous, or discouraging in their stance. Miller et al [36] in their study about informing women about the risks of maternal smoking reported more comments showing skepticism or disbelief (n=44) toward the message than comments showing support or belief (n=25), some of which said that the message was not believed or that there was no proof for it. By looking at comments posted on their social media platforms and blogs, Verduci et al [37] claimed that the website was helpful in increasing knowledge and awareness of the importance of nutrition during pregnancy. Parackal et al [20] noted the low acceptability of the campaign targeting alcohol consumption during pregnancy, with typical comments including words such as “stupid.”

    Impact

    All but 2 of the included studies presented social media metrics such as impressions, engagement, views, users, followers, interactions, likes, comments, shares, tweets, and reach as the measure of impact.

    Chan et al [34] reported both the actual reach of the Facebook campaign (620,536 women) and the 85% percentage of the target audience on Facebook that it represented.

    Not all studies provided the same metrics, making any direct comparison between campaigns impossible. However, the highest number of impressions (115,450 on Facebook) was provided by Miller et al [36], the highest number of views (201,754 on Facebook videos) was provided by Parackal et al [20], and the highest number of likes (19,600 likes on the Facebook page) was provided by Verduci et al [37].

    Unintended Consequences

    Although not the main focus, one study presented data on the unintended consequences of health messaging. Martin et al [31] identified negative conversations and discouraging tweets toward maternal vaccination, claiming links to autism and fetal deaths, as well as a mistrust toward public health authorities.

    Quality Assessment

    Within the 3 studies [21,32,35] that were double assessed, 2 ratings were disagreed on. However, after discussion, the reviewers agreed on the final rating. Moreover, it was also agreed that no studies should be excluded.

    It was noted that the biggest concern was that many of the included studies did not report how the data had been extracted, as well as whether the intended audience had been reached by the message or campaign. Figure 2 presents the methodological quality of the included studies.

    ‎
    Figure 2. Methodological quality ratings for each criterion across all included studies.

    Principal Results

    The purpose of this systematic review was to retrieve and analyze previous research on evaluating the impact, effectiveness, acceptability, and unintended consequences of messaging on prenatal health behaviors using social media data. The secondary aim was to review the methods that had been used to carry out these evaluations, both in terms of methodology and the quality of the studies. Despite the number of articles identified in the first screening process (n=1139 after duplicates were removed), this review suggests that not many studies have used data from social media to evaluate prenatal health messaging and campaigns. This was unexpected since social media plays a key role in providing health information to the public [8].

    This review highlighted a gap in the literature in terms of evaluating the effectiveness, acceptability, and unintended consequences of prenatal health messaging. The majority of studies reported impact in terms of social media metrics (eg, impressions, reach, shares, and comments), while a study [34] presented the actual impact (reach) of the campaign in terms of the size of the audience that they reached on Facebook. None of the included studies evaluated the effectiveness of the message or campaign by using social media data. For future research, it is important to create a reliable methodology to measure the effectiveness of prenatal health messaging or campaign. It has been noted that there is no consensus on how effectiveness is defined in this context, together with the issue of the feasibility of assessing effectiveness due to the lack of follow-up in many studies that use social media data. Other studies have, for example, evaluated effectiveness by using sales numbers for condoms after a social media campaign promoting condom use in Turkey [39]. Thus, many studies rely on the availability of other data sources than social media for effectiveness evaluation. This is also true for measuring the acceptability of the messaging or campaign. A total of 2 included studies [20,36] showed comments expressing a negative stance or skepticism, for example, the message was not believed or accused of not being evidence-based. A way of evaluating the acceptability of prenatal health messaging or campaigns could be to do a pilot launch of the intended information and only release it to a limited audience, and thereafter see how well the messaging is received. A total of 3 included studies had external people reviewing the content before the campaigns were launched [29,33,36]. However, none of them used social media data to evaluate this. Therefore, this information was not included in this review. Particularly neglected is the area of unintended consequences, with reports from only one study. Monitoring unintended consequences both in terms of broader prenatal health messaging as well as during the rollout of a communication campaign is a crucial course correction step as it can allow improvisation of health messages (if done prospectively and in a timely fashion) or at least to accumulate “lessons learnt” for a more realistic and rounded evaluation of the campaign’s success (when done retrospectively).

    An important factor when communicating and evaluating prenatal health messaging and campaigns is to recognize that the type of information sought can vary depending on whether it is on the internet or on social media. This includes search engines being used to search for health-related information and social media being used to find out more about the impact of health conditions [40]. Moreover, in a study conducted by Daly et al [41], 99.5% of the respondents found health websites and apps an acceptable place for health information, while 88.2% agreed that social media was acceptable.

    A total of 2 articles in this systematic review used influencers to spread their campaign [32,33]. It has been shown that there is a risk of misinformation on social media, especially health information spread by influencers [42]. This creates an online environment that needs accurate prenatal health information that reaches the intended audience, again showing the importance of assessing that the information is reaching the intended audience and is also accepted by them. Moreover, if using influencers to spread prenatal health messaging or campaigns, it is of priority to ensure that the influencers are disclosing if they are being paid to create trustworthiness for the information.

    The studies in this review varied in how they used social media data, with most of them using it for descriptive purposes. Other than that, there was no consistent methodology among the included studies. This aligns with a previous systematic review looking at health promotion interventions using social media networks, showing that a wide range of measures are used to evaluate social media health messaging relating to a wide range of topics [19]. Moreover, the methodological quality of the studies was assessed using a quality assessment tool adapted from Golder et al [26]. The biggest concern for the methodological quality was the choice of social media platform, especially the lack of a priori justification behind the selection of the type of social media data. This is crucial as audience characteristics may vary between social media platforms; as a consequence, the selection of a specific social media platform may have implications for health messaging. Scholars have argued that we live in a society of digital inequality where those who engage in the digital world have a greater advantage than those who do not [43]. Moreover, it has been shown that higher socioeconomic status has a relationship with a broader use of social media platforms and other digital media platforms [44]. The use of the internet and social media also differs between countries. In some low- and middle-income countries, less than 5% of the population is online [45]. Again, this highlights the importance of having the correct methodology to measure whether the intended audience is reached or if they can even be reached by the information.

    In addition to the uptake of health messaging, there are notable variations between the measurements of impact or impact metrics across the social media platforms (eg, retweets on Twitter or reach on Facebook). For future research, we suggest strengthening evaluations of messaging and campaigns related to prenatal health behaviors to create a robust methodology and trustworthy results, for example, by using causal inference methods applied to social media data and providing clear definitions of outcomes of interest, for example, effectiveness, acceptability, and were unintended consequences.

    Strengths and Limitations of This Review

    The principal strength of this review is its comprehensive and systematic approach. This extends to the inclusion of diverse social media platforms and targeting studies using any kind of social media data in a broad sense, even if the campaign or message was not communicated on social media channels. Moreover, the search included studies that did not have a main aim analyzing social media data but were still considered eligible if social media data were used. Second, the definitions of effectiveness, acceptability, and unintended consequences used in this review were not based on how the included articles had defined them since these definitions were often not provided. Instead, the definitions were decided upfront and applied to all studies to ensure consistency. Nevertheless, there are limitations of this review, such as language limitations. Given the spoken languages of the reviewers, only English and Swedish articles were eligible. Finally, to minimize publication bias, gray literature was included in the search strategy. Regardless of the measures to reduce publication bias, it must be noted that studies reporting significant positive results are more likely to be published than those reporting negative or no impact [46].

    Comparison With Previous Work

    This is the first review of the evaluation of prenatal health messaging using social media data. Previous systematic reviews have evaluated interventions delivered on social media, targeting any health behavior [19] as well as evaluating digital interventions [47]. Lim et al [19], in accordance with the evidence in this review, showed a lack of consistent and robust methodology among the included studies when it came to evaluating the effectiveness of health promotion interventions delivered on social media. Unlike this review, Lim et al [19] included studies not using social media as a data source for analysis, which allowed the inclusion of more studies (n=47). Another systematic review looking at health behaviors and social media is the one conducted by Chang et al [48] which looked at social media and weight management. Similar to this review, they reported a need for future research to measure social media’s role and effectiveness in influencing health behaviors.

    Conclusions

    Social media could be a valuable resource, both for communicating and evaluating prenatal health messaging and campaigns. This is true for those who are interested in capturing real-time data about any health care messaging through social media around a variety of health care domains. Previous research on evaluating prenatal health messaging using social media data showed that there is a need to develop a universal understanding of what measurements to use to carry out these evaluations. This is especially true with regard to measuring the effectiveness of the messaging, making sure that the message is clearly understood, and shaping behaviors. By developing a comprehensive set of recommendations covering all the evaluation steps, there is an exciting possibility for future research to be able to contribute to a better understanding of prenatal health messaging by using social media data as the rich data source it is.

    Acknowledgments

    NFF created the study protocol, with CM, JB, and LZ all contributing to its development. NFF completed the screening as well as the data extraction, with SAK contributing to the second data extraction as well as the quality assessment. NFF wrote the draft for the paper, with CM, JB, and LZ all providing critical feedback and contributed to reviewing and editing.

    This work was supported in part by a grant (MR/N0137941/1) for the GW4 BIOMED Medical Research Council Doctoral Training Partnership awarded to the Universities of Bath, Bristol, Cardiff, and Exeter from the Medical Research Council and UK Research and Innovation for NFF and SAK. During this work, CM was supported by the National Institute for Health and Care Research School for Public Health Research (PD-SPH-2015). The views expressed are those of the authors and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Search strategy.

    DOCX File , 14 KB
    Multimedia Appendix 2

    Full list of data extracted.

    DOCX File , 15 KB
    Multimedia Appendix 3

    PRISMA checklist.

    PDF File (Adobe PDF File), 74 KB
    1. Burd L. FASD and ADHD: are they related and how? BMC Psychiatry. 2016;16(1):325. [FREE Full text] [CrossRef] [Medline]
    2. Easey KE, Dyer ML, Timpson NJ, Munafò MR. Prenatal alcohol exposure and offspring mental health: a systematic review. Drug Alcohol Depend. 2019;197:344-353. [FREE Full text] [CrossRef] [Medline]
    3. da Silva Pereira PP, Da Mata FAF, Figueiredo ACG, de Andrade KRC, Pereira MG. Maternal active smoking during pregnancy and low birth weight in the Americas: a systematic review and meta-analysis. Nicotine Tob Res. 2017;19(5):497-505. [FREE Full text] [CrossRef] [Medline]
    4. Kemp S. Digital 2023 april global statshot report. DataReportal’s. 2023. URL: https://datareportal.com/reports/digital-2023-april-global-statshot [accessed 2023-12-01]
    5. Ortiz-Ospina E, Roser M. The rise of social media. Our World in Data. 2023. URL: https://ourworldindata.org/rise-of-social-media?ref=tms [accessed 2023-12-01]
    6. Auxier B, Anderson M. Social media use in 2021. Pew Research Center. 2021;1:1-4. [FREE Full text] [CrossRef]
    7. Krasnova H, Veltri NF, Eling N, Buxmann P. Why men and women continue to use social networking sites: the role of gender differences. J Strateg Inf Syst. 2017;26(4):261-284. [FREE Full text] [CrossRef]
    8. Wright EM, Matthai MT, Meyer E. The influence of social media on intrapartum decision making: a scoping review. J Perinat Neonatal Nurs. 2019;33(4):291-300. [FREE Full text] [CrossRef] [Medline]
    9. Wang J, Madnick S, Li X, Alstott J, Velu C. Effect of media usage selection on social mobilization speed: Facebook vs E-Mail. PLoS One. 2015;10(9):e0134811. [FREE Full text] [CrossRef] [Medline]
    10. Social media strategy development–a guide to using social media for public health communication. European Centre for Disease Prevention and Control. 2016. URL: https:/​/www.​ecdc.europa.eu/​sites/​default/​files/​media/​en/​publications/​Publications/​social-media-strategy-guide-for-public-health-communication.​pdf [accessed 2023-12-01]
    11. Zhu C, Zeng R, Zhang W, Evans R, He R. Pregnancy-related information seeking and sharing in the social media era among expectant mothers: qualitative study. J Med Internet Res. 2019;21(12):e13694. [FREE Full text] [CrossRef] [Medline]
    12. Baker B, Yang I. Social media as social support in pregnancy and the postpartum. Sex Reprod Healthc. 2018;17:31-34. [FREE Full text] [CrossRef] [Medline]
    13. Lagan BM, Sinclair M, Kernohan WG. Internet use in pregnancy informs women's decision making: a web-based survey. Birth. 2010;37(2):106-115. [CrossRef] [Medline]
    14. Sayakhot P, Carolan-Olah M. Internet use by pregnant women seeking pregnancy-related information: a systematic review. BMC Pregnancy Childbirth. 2016;16(1):65. [FREE Full text] [CrossRef] [Medline]
    15. Ford AJ, Alwan NA. Use of social networking sites and women's decision to receive vaccinations during pregnancy: a cross-sectional study in the UK. Vaccine. 2018;36(35):5294-5303. [FREE Full text] [CrossRef] [Medline]
    16. Infodemic. World Health Organization. URL: https://www.who.int/health-topics/infodemic#tab=tab_1
    17. Suarez-Lledo V, Alvarez-Galvez J. Prevalence of health misinformation on social media: systematic review. J Med Internet Res. 2021;23(1):e17187. [FREE Full text] [CrossRef] [Medline]
    18. Poon LC, Yang H, Kapur A, Melamed N, Dao B, Divakar H, et al. Global interim guidance on coronavirus disease 2019 (COVID-19) during pregnancy and puerperium from FIGO and allied partners: information for healthcare professionals. Int J Gynaecol Obstet. 2020;149(3):273-286. [FREE Full text] [CrossRef] [Medline]
    19. Lim MSC, Wright CJC, Carrotte ER, Pedrana AE. Reach, engagement, and effectiveness: a systematic review of evaluation methodologies used in health promotion via social networking sites. Health Promot J Austr. 2016;27(3):187-197. [FREE Full text] [CrossRef] [Medline]
    20. Parackal M, Parackal S, Eusebius S, Mather D. The use of Facebook advertising for communicating public health messages: a campaign against drinking during pregnancy in New Zealand. JMIR Public Health Surveill. 2017;3(3):e49. [FREE Full text] [CrossRef] [Medline]
    21. Parackal M, Parackal S, Mather D, Eusebius S. Dynamic transactional model: a framework for communicating public health messages via social media. Perspect Public Health. 2021;141(5):279-286. [FREE Full text] [CrossRef] [Medline]
    22. Ross C, Shaw S, Marshall S, Stephen S, Bailey K, Cole R, et al. Impact of a social media campaign targeting men who have sex with men during an outbreak of syphilis in Winnipeg, Canada. Can Commun Dis Rep. 2016;42(2):45-49. [FREE Full text] [CrossRef] [Medline]
    23. Hargittai E. Potential biases in big data: omitted voices on social media. Soc Sci Comput Rev. 2018;38(1):10-24. [FREE Full text] [CrossRef]
    24. Hargittai E. Is bigger always better? Potential biases of big data derived from social network sites. Annals Am Acad Pol Soc Sci. 2015;659(1):63-76. [FREE Full text] [CrossRef]
    25. Iacus SM, Porro G, Salini S, Siletti E. Controlling for selection bias in social media indicators through official statistics: a proposal. J Off Stat. 2020;36(2):315-338. [FREE Full text] [CrossRef]
    26. Golder S, Norman G, Loke YK. Systematic review on the prevalence, frequency and comparative value of adverse events data in social media. Br J Clin Pharmacol. 2015;80(4):878-888. [FREE Full text] [CrossRef] [Medline]
    27. Tully LA, Piotrowska PJ, Collins DAJ, Frick PJ, Anderson V, Moul C, et al. Evaluation of 'the father effect' media campaign to increase awareness of, and participation in, an online father-inclusive parenting program. Health Commun. 2019;34(12):1423-1432. [FREE Full text] [CrossRef] [Medline]
    28. Wyst KBV, Vercelli ME, O'Brien KO, Cooper EM, Pressman EK, Whisner CM. A social media intervention to improve nutrition knowledge and behaviors of low income, pregnant adolescents and adult women. PLoS One. 2019;14(10):e0223120. [FREE Full text] [CrossRef] [Medline]
    29. Bazzo S, Black D, Mitchell K, Marini F, Moino G, Riscica P, et al. 'Too young to drink'. An international communication campaign to raise public awareness of fetal alcohol spectrum disorders. Public Health. 2017;142:111-115. [FREE Full text] [CrossRef] [Medline]
    30. Carlson S, Dey A, Beard F. An evaluation of the 2016 influenza vaccination in pregnancy campaign in NSW, Australia. Public Health Res Pract. 2020;30(1):29121908. [FREE Full text] [CrossRef] [Medline]
    31. Martin S, Kilich E, Dada S, Kummervold PE, Denny C, Paterson P, et al. "Vaccines for pregnant women…?! absurd" - mapping maternal vaccination discourse and stance on social media over six months. Vaccine. 2020;38(42):6627-6637. [FREE Full text] [CrossRef] [Medline]
    32. Bonnevie E, Rosenberg SD, Goldbarg J, Ashley-West A, Smyser J. Building strong futures: the feasibility of using a targeted digital media campaign to improve knowledge about pregnancy and low birthweight among black women. Matern Child Health J. 2021;25(1):127-135. [FREE Full text] [CrossRef] [Medline]
    33. Bonnevie E, Barth C, May J, Carey T, Knell SB, Wartella E, et al. Growing and glowing: a digital media campaign to increase access to pregnancy-related health information for black women during the COVID-19 pandemic. Health Promot Pract. 2023;24(3):444-454. [FREE Full text] [CrossRef] [Medline]
    34. Chan L, Gordon A, Warrilow K, Wojcieszek A, Firth T, Loxton F, et al. Evaluation of movements matter: a social media and hospital-based campaign aimed at raising awareness of decreased fetal movements. Aust N Z J Obstet Gynaecol. 2021;61(6):846-854. [FREE Full text] [CrossRef] [Medline]
    35. Graham JE, Moore JL, Bell RC, Miller T. Digital marketing to promote healthy weight gain among pregnant women in Alberta: an implementation study. J Med Internet Res. 2019;21(2):e11534. [FREE Full text] [CrossRef]
    36. Miller CA, Kim SJ, Schwartz-Bloom RD, Bloom PN, Murphy SK, Fuemmeler BF. Informing women about the risks of exposing babies to tobacco smoke: outreach and education efforts using Facebook "boost posts". Transl Behav Med. 2022;12(5):714-720. [FREE Full text] [CrossRef] [Medline]
    37. Verduci E, Vizzuso S, Frassinetti A, Mariotti L, Del Torto A, Fiore G, et al. Nutripedia: the fight against the fake news in nutrition during pregnancy and early life. Nutrients. 2021;13(9):2998. [FREE Full text] [CrossRef] [Medline]
    38. Impressions. Meta. 2022. URL: https://www.facebook.com/business/help/675615482516035 [accessed 2023-12-01]
    39. Purdy CH. Using the internet and social media to promote condom use in Turkey. Reprod Health Matters. 2011;19(37):157-165. [FREE Full text] [CrossRef] [Medline]
    40. De Choudhury CM, Morris MR, White RW. Seeking and sharing health information online: comparing search engines and social media. Presented at: CHI '14: CHI Conference on Human Factors in Computing Systems; 26 April-1 May, 2014, 2014; Toronto Ontario Canada. [CrossRef]
    41. Daly MP, White J, Sanders J, Kipping RR. Women's knowledge, attitudes and views of preconception health and intervention delivery methods: a cross-sectional survey. BMC Pregnancy Childbirth. 2022;22(1):729. [FREE Full text] [CrossRef] [Medline]
    42. Chee RM, Capper TS, Muurlink OT. The impact of social media influencers on pregnancy, birth, and early parenting experiences: a systematic review. Midwifery. 2023;120:103623. [FREE Full text] [CrossRef] [Medline]
    43. Robinson L, Cotten SR, Ono H, Quan-Haase A, Mesch G, Chen W, et al. Digital inequalities and why they matter. Inf Commun Soc. 2015;18(5):569-582. [FREE Full text] [CrossRef]
    44. Yates S, Lockley E. Social media and social class. Am Behav Sci. 2018;62(9):1291-1316. [CrossRef]
    45. Max R. The Internet’s history has just begun. Our World in Data. 2018. URL: https://ourworldindata.org/internet-history-just-begun [accessed 2023-12-01]
    46. Ioannidis JPA, Munafò MR, Fusar-Poli P, Nosek BA, David SP. Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends Cogn Sci. 2014;18(5):235-241. [FREE Full text] [CrossRef] [Medline]
    47. Oh SS, Moon JY, Chon D, Mita C, Lawrence JA, Park E, et al. Effectiveness of digital interventions for preventing alcohol consumption in pregnancy: systematic review and meta-analysis. J Med Internet Res. 2022;24(4):e35554. [FREE Full text] [CrossRef] [Medline]
    48. Chang T, Chopra V, Zhang C, Woolford SJ. The role of social media in online weight management: systematic review. J Med Internet Res. 2013;15(11):e262. [FREE Full text] [CrossRef] [Medline]

    ‎
    FASD: fetal alcohol spectrum disorders
    HPHC: Healthy Parents, Healthy Children
    NSW: New South Wales
    PROSPERO: International Prospective Register of Systematic Reviews
    WHO: World Health Organization

    Edited by A Mavragani; submitted 08.12.22; peer-reviewed by L Schölin, A Price, D Carvalho; comments to author 11.10.23; revised version received 27.10.23; accepted 29.11.23; published 20.12.23.

    Copyright

    ©Nessie Felicia Frennesson, Cheryl McQuire, Saher Aijaz Khan, Julie Barnett, Luisa Zuccolo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 20.12.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.