Appraising Unmet Needs and Misinformation Spread About Polycystic Ovary Syndrome in 85,872 YouTube Comments Over 12 Years: Big Data Infodemiology Study

Introduction

Polycystic ovary syndrome (PCOS) is the most common endocrinopathy in women, with a prevalence of 8% to 10% worldwide [1]. It results in a substantial economic burden due to metabolic, reproductive, emotional, and psychological complications [2]. A systematic review of the lived experiences of people with PCOS highlighted the need for better public awareness to limit stigma and build positive social support [3]. Several researchers have explored the opinions of the public at local and regional levels [4-6]. However, we did not find similar research at a worldwide level. A recent study highlighted that the lived experiences of people with PCOS are influenced by ethnicity and birthplace [7]. As the logistics and financial burden of conducting large-scale studies to confirm this worldwide would be heavy, we must consider surrogate methods.

Social media can act as a surrogate for public views and sentiments by providing a forum for people to voice their ideas and participate in debates on various subjects. This provides an opportunity for academics and researchers to learn more about the views and opinions of the public. Infodemiology is the science of distribution and determinants of information in an electronic medium, specifically the internet or in a population, with the ultimate aim of informing public health and public policy [8]. Big data analytics have reformed changing health paradigms with rigorous analytical reviews to provide better clinical solutions [9]. This involves formulating quantitative and qualitative assessments previously too large for conventional software [10]. Infodemiology also helps to critically evaluate resource allocation and misinformation spread online, providing evidence-based recommendations for multisectoral actions to develop legal policies and increase health literacy [11]. Our group explored the recurring themes associated with PCOS on Twitter [12,13]. However, Twitter’s ability to reflect the wider world is constrained by linguistic and cultural obstacles and the local concentration of the platform’s user base resulting in a digital divide. Further, Twitter’s algorithms may amplify prejudices and stifle other viewpoints.

YouTube is an online video-sharing platform and is the second most visited website after Google [14]. As the internet continues to develop as the primary source of health-related information, YouTube has a significant potential to drive health-related conversations, with over 2 billion users monthly [15,16]. Analysis of YouTube’s comments provides an excellent opportunity to understand the public’s perceptions on a global scale. While some studies have analyzed the content and reliability of YouTube videos related to reproductive health and PCOS, the findings were limited by small sample sizes and a lack of focus on a comprehensive analysis of the public’s comments [17,18]. Therefore, we delved into the vast realm of YouTube comments on PCOS videos, meticulously analyzing the keywords, sentiments, prevailing trends, and recurring themes expressed by the public, ultimately unravelling the rich tapestry of opinions surrounding PCOS. As recent studies have shown that people with PCOS are not satisfied with the care received, by examining these trends, we contribute to a better understanding of the dynamics of public interactions and shed light on areas for potential improvement in PCOS care delivery [3]. Our specific objectives were (1) to identify trends over time, study the context of YouTube comments about PCOS and find associated themes, (2) to identify gender-based differences in these YouTube comments, and (3) to identify the underlying sentiments of these YouTube comments.

Methods

Data Extraction

This study was conducted in May 2023. All videos published on YouTube with “PCOS” OR “polycystic ovary syndrome” in the title or description from May 2011 to April 2023 were included in the study. The earliest possible data available in the public domain were from May 2011. The YouTube application programming interface was used to extract comments on videos. The extracted comments were analyzed using Mozdeh (University of Wolverhampton). Only the initial comment for each individual was included to prevent any individual’s opinions from overshadowing the results. We included up to 350 recent comments to ensure popular videos with more comments did not sway the overall analysis. A similar method has been used elsewhere [19,20].

Word Association Detection, Contextualization, and Content Analysis

We analyzed the overall keyword frequency to identify recurring themes in the comments. This was performed after removing keywords related to prepositions or other connecting words. Time series graphing was done to understand overall trends in the YouTube comments. Association-mining comparison analysis was done to study differences and formulate a list of keywords for which there was statistical evidence of gender differences in their use by male versus female commentators. Words were considered gendered (ie, had evidence of gender differences) if they met the following criteria: (1) they appeared in at least two profiles (to rule out typos), (2) they appeared in a higher proportion of comments from that gender than from other genders, and (3) the findings were statistically significant.

A 2 × 2 chi-squared test was used to determine whether the evidence supported that a word was used disproportionately by male or female individuals for each word in the overall gendered word frequency table. The Benjamini-Hochberg procedure [21] was used to control the risk of false positives from running multiple tests. It adjusts the chi-squared threshold to keep the likelihood of drawing at least one false positive conclusion below 5%. After identifying gendered keywords, network analysis was done using the Fruchterman-Reingold algorithm [22,23] to identify the connections with other similar keywords and study associated themes. The disconnected nodes were placed in corners, and the node size was set proportional to the number of comments for the label. Finally, after identifying the associated keywords for each gender, the YouTube comments with each of those keywords were manually read to identify underlying themes and were categorized. The reported analysis highlights major themes, subthemes, and representative quotes.

Gender Identification

Although YouTube does not track commenters’ gender, certain commenters’ usernames can be used to guess their gender. These data served as a proxy for population gender data. Usernames were divided into numerous parts (using spaces or intercapped compounds) where possible. The gender of the commenter was then determined by matching the first part of the name to one used at least 90% by men or women in the 1990 US census [24]. US census data were chosen because the United States is a cosmopolitan country with people from various regions and ethnicities. The appropriate gender was also ascribed for the terms Mr, Mrs, Ms, and Miss [19].

Sentiment Analysis

The strength of the positive and negative sentiment of each comment was identified with SentiStrength, which is incorporated within the Mozdeh software and uses a lexicon of sentiment terms with linguistic rules [25]. Sentiment analysis was done for each comment and was not specifically targeted at only those comments including PCOS, PCOD, polycystic ovary syndrome, or any other similar term. A score of 1 (not positive) to 5 (extremely positive) and a second independent score of 1 (not negative) to 5 (extremely negative) was given to each text. SentiStrength was chosen due to its ability to analyze negative and positive sentiment independently using a dual method, which is crucial for the objectives of the research; its accuracy on YouTube comments is close to human levels [25-27]. For social science research purposes, lexical software that uses a predefined set of sentiment terms with linguistic rules, such as SentiStrength, is preferred to machine learning because the latter can identify contentious themes as a proxy for sentiment.

The average positive and negative sentiment strength of each group’s comments were determined independently. The conventional normal distribution formula was used for each group to construct a 95% CI. Due to the skewed and discrete nature of the data—as opposed to continuous data—this is an approximation. Due to the possibility of interdependence among comments made on the same YouTube video, the data also violates the statistical independence assumption. Therefore, the CIs should be considered indicative estimations rather than robust numbers. To compensate, the changes in average sentiment were considered significant only when CIs did not overlap. As a slight overlap between 2 associated CIs is consistent with statistically significant differences, our methodology is somewhat conservative [28,29]. Sentiment trends were also studied from 2011 to 2023.

Ethics Statement

No human or animal participants were involved, and no patient data were collected. Only publicly accessible data were analyzed; no identifiable data are reported in this study.

Results

Overview

From 940 videos, 85,872 total comments were fetched and analyzed. We identified a specific gender for 13,106 comments. Of these comments, 1506 were matched to male users (11.5%), and 11,601 to female users (88.5%).

Trends and Keyword Analysis

We noticed an increasing trend in the total number of comments and the length of YouTube comments by users in the past 5 years (Figure 1). The most frequently used keywords (after excluding words referring to PCOS or the video itself, prepositions, and other connecting words) were period (n=9352), thank (n=8911), month (n=4566), doctor (n=4481), weight (n=4318), problem (n=4204), help (n=4086), hair (n=3833), time (n=3718), and diet (n=3262).

‎

Female Versus Male Word Association, Contextualization, and Content Analysis

The various gender-specific keywords are listed in Table 1. Keywords associated with diagnosing PCOS, symptoms of PCOS, pills for PCOS (medication), and pregnancy were significantly associated with female users. Keywords associated with herbal treatment, natural treatment, curing PCOS, and online searches were significantly associated with male users. Network analysis showed 72 nodes with 4912 connections (indicated by arrows) for female users (Figure 2), and 45 nodes with 996 connections for male users (Figure 3). The key themes associated with female users were symptoms of PCOS (such as irregular periods and acne), positive personal experiences (such as helpful and love), negative personal experiences (such as fatigue and pain), motherhood (such as infertility and trying to conceive), self-diagnosis, and use of professional terminology detailing their journey (Table 2). The key themes associated with male users were misinformation regarding the “cure” for PCOS; using natural and herbal remedies to cure PCOS; fake or spam testimonies in which spammers disguised themselves as patients to sell their courses and consultations; finding treatment for PCOS; sharing perspectives of female family members, such as their partners; or relationship experiences (Table 3).

‎

‎

Sentiment Analysis

The average positive and negative sentiment scores associated with the extracted comments overall were 1.6651 (95% CI 1.6593-1.6709) and 1.4742 (95% CI 1.4683-1.4802), respectively, with a net positive difference of 0.1909. The average positive and negative sentiment scores for female users were 1.8266 (95% CI 1.8101-1.8430) and 1.6944 (95% CI 1.6750-1.7139), respectively, with a net positive difference of 0.1321. Male users’ average positive and negative sentiment scores were 1.7337 (95% CI 1.6892-1.7783) and 1.5279 (95% CI 1.4808-1.5750), respectively, with a net positive difference of 0.2058. The sentiments were majority positive when the estimated sentiment strength was weak or moderate (sentiment strength 2 or 3). However, negative sentiments dominated when the estimated sentiment strength was strong or very strong (sentiment strength 4 or 5; Table 4). Female users with very strong sentiment strength had higher negative sentiments when compared to similar male users. From 2011 to 2016, numerous variations in sentiments were seen. From 2017 onwards, the sentiment trends appear to have plateaued with slightly more positive sentiment (Figure 4).

‎

Discussion

Principal Findings

To the best of our knowledge, this is the first attempt to comprehensively study all YouTube comments related to PCOS for evidence synthesis. While comments from female users revolved around lived experiences, those from male users focused on selling cures and spam or fake news. The sentiment trend has flattened and is slightly positive in the last few years, especially following May 2017, which may suggest an improved lived experience with PCOS in recent times, as seen graphically (Figure 4). Further studies are needed to study temporal trends in sentiment analysis, as our study yields weak evidence about slight positive sentiment. Positive stories highlight the importance of supporting networks and practical strategies in managing worries and discomfort within the context of PCOS conversations. Viewers express gratitude toward YouTubers who share their personal journeys and raise awareness about PCOS. Conversely, negative experiences shed light on the psychosocial distress accompanying this condition, demanding attention and understanding.

Brandolini’s law asserts that refuting incorrect information takes much more work than generating and disseminating it [30,31]. To address the imbalance between spreading falsehoods and dispelling them, this law stresses the significance of fact-checking and promoting trustworthy sources of information. Therefore, more concrete efforts are needed from YouTube, medical professionals, the public, and concerned local and international organizations to tackle misinformation and ensure that people are not misguided.

Online misinformation and fake news can significantly harm women with PCOS. False claims about PCOS’s causes, symptoms, and treatments spread rapidly through social media and websites, leading to confusion and delayed medical care. Unproven remedies promoted as cures can divert women from evidence-based management, worsening their condition and mental well-being. Misinformation can influence lifestyle choices, encouraging extreme diets and exercise routines that impact health negatively. Distrust in medical professionals may arise, preventing informed decisions and proper care. Stigmatization and isolation can result from perpetuated stereotypes. Additionally, misinformation can misguide family planning decisions due to misconceptions about fertility and pregnancy. Addressing this issue requires promoting digital literacy, reliable health resources, and supportive online communities to counterbalance the harmful effects of misinformation and ensure that women with PCOS receive accurate information and care.

Analyzing the sentiment of social media data from women with PCOS provides insights into their emotional well-being, highlighting the challenges, anxieties, and support-seeking behaviors they share online. By monitoring sentiment changes over time, health care providers can gauge the effectiveness of interventions and tailor support accordingly. It also improves patient-provider communication, allowing health care professionals to address specific concerns during appointments. Sentiment analysis informs targeted awareness campaigns that resonate with women’s emotional experiences, reducing stigma and promoting accurate information. Moreover, it sheds light on the psychological impact of PCOS, guiding research and interventions to address not only physical but also emotional aspects of the condition.

Several studies have highlighted gender disparities in diagnosis, management, and funding. And gender differences may exacerbate health disparities for health conditions that affect a single sex. Gender bias in health care has manifested through the underrepresentation of women in studies, trivialization of their complaints, and biased research grant awards [32]. Women’s exclusion from clinical trials like the aspirin study prompted action in the 1980s to include them, but issues persisted [33]. Women’s physical complaints historically were wrongly considered to be psychological, as seen with “hysteria.” Myalgic encephalomyelitis/chronic fatigue syndrome was initially dismissed as psychogenic [34]. Gender bias extends to research grants, with studies suggesting women face bias during grant renewals [35]. Canadian research grant analysis indicated biases in assessing female principal investigators based on their gender, not their research quality [36].

International awareness initiatives such as PCOS Awareness Month (September) may be used to develop targeted action-driven campaigning and involve underrepresented communities from low-, middle-, and high-income countries in fostering pragmatic communities. Dedicated themes may be used each year for a specific topic concerning PCOS as has been done and suggested for other awareness campaigns [37-39].

Strengths and Limitations

The strengths of our study include the analysis of the comments on all PCOS videos on YouTube over 12 years. We used a multidimensional approach that included association mining, network analysis, subgroup analysis based on gender, sentiment analysis, and overall trends to analyze all the extracted YouTube comments. The methodology has been previously tested in various peer-reviewed studies. However, as commenting on YouTube videos is optional, an overrepresentation of participation by one gender depending on the topic may be seen (as in our case, since PCOS is a female-specific condition), limiting the generalizability of our findings. Gender estimation and sentiment assessment also have their limitations. Several usernames did not fit our classification criteria and were left unassigned. Although our methodology can distinguish between predominately male and female users, it is ineffective for nonbinary genders, possibly because people are named by their parents before they establish their gender identity; there are thus no nonbinary-specific first names [40]. The reliability of our findings may be affected by videos with clickbait titles or descriptions. Furthermore, the inherent methodological limitations of sentiment analysis, association mining, and network analysis cannot be disregarded.

It is important to note that social media should be viewed as one source among others when attempting to understand public views and sentiments. Its limitations include the potential for echo chambers, manipulation of information, varied dynamic trends corresponding to public events [41,42], and the exclusion of individuals who are not active on these platforms. Therefore, combining social media analysis with other research methods, such as surveys, focus groups, and traditional media analysis, can provide a more comprehensive understanding of public views and sentiments. Nonetheless, our study shares important coverage on the public dissemination of PCOS-related information.

Conclusions

There has been an increasing trend for sharing information about PCOS on YouTube in recent years, with a similar rise in viewers sharing their perspectives. There is a disparity in views on PCOS between women and men, with the latter associated with non–evidence-based approaches and misinformation. The improving sentiment noticed with YouTube comments may reflect better health care services recently. Prioritizing and promoting evidence-based care and disseminating pragmatic online coverage is warranted to improve public sentiment and limit misinformation spread.