Exploring Perceptions About Paracetamol, Tramadol, and Codeine on Twitter Using Machine Learning: Quantitative and Qualitative Observational Study

Introduction

More than 25 million adults are affected by chronic pain in the United States [1,2]. The prevalence rate of chronic pain varies between 11% and 40% [3]. The treatment for chronic pain is complex, and the majority of guidelines recommend a psychological approach as well as pharmacological treatment [4-6]. The pharmacological treatment includes nonopioid as well as opioid medication [3].

Over-the-counter (OTC) drugs are generally effective and safe; thus, they are used for the control of mild pain when no medical consultation is needed [7]. Paracetamol is widely used for the treatment of mild pain, with a good safety profile when used as recommended; however, there are increasing reports of it being used inadequately [7,8]. In a study analyzing OTC drugs, which included paracetamol, 24% of adults confirmed that they took a higher dose than the recommended [9]. Another study found that many people do not pay attention to the information in the pharmacological leaflet, and more than 50% are unaware of the active ingredient they are taking [10]. Codeine and tramadol form part of the weak opioid group; they are not exempt from secondary effects, and on many occasions, they can be obtained as OTC drugs [11].

Moreover, despite being illegal, the sale of prescription medication on the internet has been detected [11,12]. Twitter represents a platform where criminal agents can commercialize and sell these drugs. Sales through Twitter are subject to lower standards of regulation and supervision than other web-based platforms [13].

The investigation of beliefs and attitudes of patients has traditionally been investigated with surveys, interviews, and clinical trials [14,15]. Recently, investigators have been making more use of social media platforms for the monitoring of public health, the study of attitudes toward treatments, and to increase the knowledge of the medical experience of patients [16-18]. Twitter is the most used social media platform for health research for many reasons, namely due to the public feed users have.

Twitter has been effectively used in analyzing social stigma toward specific circumstances such as obesity or mental health [19,20], as well as in the field of health education [21], including dental health, tobacco and vaping [22,23], and vaccinations [24,25]. There is also evidence of a correlation between Twitter posts and real-life clinical events, such as suicides or substance and alcohol abuse [26-28]. In this context, the exploration of tweets discussing the perception of drugs for better comprehension, complementation, and therapeutic decision-making has been thoroughly investigated in many areas of medicine, including opioid medication [29-31].

The main objectives of this study are (1) to conduct a quantitative analysis of Twitter posts from the years 2019 and 2020 regarding weak opioids and paracetamol; (2) to characterize the user profile that engages the most in these conversations; (3) to identify the most frequently discussed medical and nonmedical topics; and (4) to pinpoint the topics and user types that generate the most interest. Our hypothesis is that paracetamol will be the most tweeted drug due to its widespread use among the general population, that health care professionals will have the highest engagement in these discussions, and that the most discussed medical aspect will be the drug’s efficacy, while the most discussed nonmedical aspect will be its price. Last, we hypothesize that posts published by health care professionals will generate the most interest among Twitter users.

Methods

Data Collection

In this quantitative and qualitative observational study, we have focused on tweets that referenced the weak opioids tramadol and codeine. Paracetamol was chosen as the control drug due to its efficacy in different types of pain, its safety profile, its extensive clinical use in a wide range of patients, and its availability and accessibility. For that, we have collected all the tweets making reference either to the active ingredient or commercial names approved in Spain or the United States of paracetamol, codeine, or tramadol: paracetamol, acetaminophen, tramadol, adolonta, capdol, captor ceparidin, diliban, dolodol, enanplus, paxiflas, pazital, tioner, tracimol, tradonal, conzip, rybix, odt, ryzolt, ultram, codeina, codeine, codeisan, histaverin, notusin, and perduretas y tuzistra xr. The tweets had to meet the following requirements in order to be included: (1) posted from an open account, (2) contain a keyword from the list mentioned above, (3) have been published between January 2019 and December 2020, and (4) be written in either Spanish or English. We also collected data complementary to the tweets: the number of retweets and likes generated by each tweet, as well as the description of the user’s profile. The tool used for the collection of tweets was Tweet Binder, which allows access to 100% of public tweets.

Process of Content Analysis

A total of 152,056 tweets were collected; of those, 49,462 were discarded due to the tweet being written in a language different from English or Spanish (Figure 1). The investigators created a codebook to analyze the tweets. Modifications in the codebook were carried out after an analysis of 300 tweets by 3 investigators to elaborate the final codebook. The interrater reliability between raters was assessed, obtaining κ values ranging from 0.71 to 0.89 for the different categories. In the codebook, the type of user is the first domain classified, and then it distinguishes between medical and nonmedical content. We determined that a tweet should be classified as “medical content” when it made a clear allusion to the efficacy, adverse effect, or dissemination of the drug. Also, we have classified the tweets that included personal opinions, distinguishing those that had a positive message from those that had a negative one. Finally, we identified those tweets that pose questions. In terms of the type of user, we distinguished between patients, family members and friends, health care professionals, or institutions. To determine the type of user, we examined the Twitter profile of the publisher (useful when identifying health care professionals or institutions), the pronouns used (useful to distinguish patients and family members or friends), or the content of the tweet itself (for example, it is common for patients and health care professionals to reveal themselves as such). In terms of the content, if it was of a medical nature, we classified it depending on whether it made reference to the efficacy of a drug or its adverse effects. We also analyzed if the information stated was scientifically correct, identifying those tweets that included links to scientific papers. Moreover, in the nonmedical content, we distinguished between four themes: (1) commercial issues, (2) economic aspects, (3) solidarity, and (4) trivialization. Finally, looking at the active ingredient, we classified the tweets into 3 categories: paracetamol, tramadol, and codeine. We selected a total of 1000 tweets for each drug, which were classified manually.

‎

Multilingual Machine Learning Classifier

The goal of this initial tagging of 1000 tweets was to provide the data to train, test, and validate machine learning classifiers so that all the extracted tweet classifications could be inferred. To train the classifiers, a transformer multilingual model, xlm-roberta neural net, was used, and the library to deploy it was ktrain [32,33]. The following additional features were generated to improve the understanding of the selected set: the number of tokens that the sentence contains, the total length of the tweet in characters, the language of the tweet, and the extracted hashtags from the tweet. Also, to improve the machine learning classifier performance, we generated a clean text that takes the mentions (@) and hyperlinks out of the tweet so that it is more readable and less noisy.

Out of the 1000 manually labeled tweets, we reserved 10% (100/1000) to use as a blind set for model validation, and then the setup for training the classifier was 80% (800/1000) training and 20% (200/1000) validation. First, the training was done on the classifiable feature, and then, out of the tweets that were labeled classifiable, the rest of the classifications were trained. The weighted average F₁-score of the training validation and against the blind data set was above 0.80 in all the cases except in the user and interest areas, which were slightly lower. These analyses were performed with Python 3.7 (Python Software Foundation) and using the libraries pandas, NumPy, JSON, and ktrain.

Statistical Analysis

The frequency distribution and percentages of tweets according to different categories based on the characteristics of the tweet were reported across different tables. The comparison of the proportion of tweets between categories was carried out using the Pearson chi-square test, from which the P value of statistical significance is reported. Likewise, the accuracy of the different tweet distributions obtained by these multilingual machine learning models is reflected by the weighted average F₁-score (a combination of precision and recall; the closer to 1, the less possibility of classification error). Retweet-to-tweet and like-to-tweet ratios according to pain drug, type of user, content, and other characteristics of the tweet were also calculated.

We used linear regression models to evaluate the associations between tweet content, type of user, and other characteristics of the tweets and the number of likes and retweets. Individual beta coefficients were adjusted for the rest of the characteristics of the tweet. These analyses were conducted with the software packages Stata (version 16; StataCorp) and Excel (Microsoft Corp).

Ethical Considerations

This project received approval from the ethics committee of the Hospital Principe de Asturias (OE 14_2020) and is compliant with the research ethics principles of the Declaration of Helsinki (seventh revision, 2013). This study did not directly involve human participants, nor did it include any intervention; instead, it used only publicly available tweets (subject to universal access through the internet according to the terms of service that all users on Twitter accept). Nevertheless, we have taken care to not directly reveal in this report any username, and we have avoided citing tweets that could be offensive or compromised to someone.

Results

Patients and Health Care Professionals

Patients are the most active when it comes to talking about analgesia on Twitter; however, health care professionals are the most desired.

Of the 102,594 tweets included in the analysis, 81.03% (83,129/102,594) of the tweets were considered classifiable. The other 19,465 tweets did not include sufficient information to be classifiable (Figure 1). As shown in Table 1, the distribution of tweets was similar in both groups, with 42,840 tweets about paracetamol (42,840/83,129, 51.5% of the total) and 42,131 tweets mentioning the weak opioid group (42,131/83,129, 50.7% of the total). There were 900 tweets that mentioned on the same tweet “tramadol” and “codeine”; therefore, we included them in both groups. Another 1842 tweets mentioned paracetamol and weak opioids in the same tweet, and they are also included in both categories.

Regarding Twitter users, it is notable that nearly 75% (60,771/83,129) of tweets were published by users classified as patients, while health care professionals and health institutions only published 8% (6692/83,129) and 4% (4031/83,129), respectively (Table 1). However, despite having a low percentage of tweets, the tweets posted by these groups obtained the highest ratio of like-tweet and tweet-retweet. Health care professionals obtained a mean of 8.6 likes and 2.4 retweets per tweet, while tweets written by patients only obtained a mean of 3.7 likes and 0.7 retweets per published tweet (Table 1).

About the content of the tweet, nearly 2/3 (61.1%) of tweets made reference to medical content. In terms of the opinion expressed by the users, in 63% (52,528/68,305) of cases it was considered positive, 19% (15,777/68,305) negative, and the rest was undetermined (Table 1). Twitter users were asked a question in 6% (5244/83,129) of tweets.

Efficacy of Paracetamol and Opioids

Efficacy regarding paracetamol and opioids is the most recurrent topic on Twitter; however, secondary effects generate the most interest.

Of those tweets that included medical content, 80.8% (41,080/50,822) mention the efficacy of the drug, and of those, 6.9% (3501/50,822) describe a good or sufficient efficacy. Conversely, we found that 40.7% (20,700/50,822) of tweets with medical content made mention of the adverse effects. Also, 7.2% (3661/50,822) of tweets with medical content included a reference to a scientific article. In terms of likes and retweets, the tweets that described a positive efficacy of the drug obtained a smaller number of likes and retweets per tweet than those that described a null or low efficacy (Table 2). Moreover, tweets that mentioned adverse effects generated more retweets and likes than those discussing the efficacy of the drug. Finally, the medical tweets that generated the greatest number of likes and retweets were those that made reference to scientific articles (Table 2).

Considering the nonmedical tweets, 2 out of 3 were considered to trivialize the drug. Only 1.6% (533/32,307) expressed a sentiment of solidarity or support for the users of paracetamol or weak opioids. The rest made reference to commercial or economic aspects of the drugs. Solidarity was the content that generated the highest number of retweets by far (Table 2).

Patient Participation

Patients participate mostly in conversations regarding opioids but not in those talking about paracetamol.

Users classified as patients published 73.10% (60,771/83,129) of the tweets analyzed. The proportion of tweets posted by each type of user for each type of drug was significantly different. The patient group was the only one where tweets referring to opioids were more prevalent (34,609/60,771, 56.9%) than tweets referring to paracetamol (27,420/60,771, 45.1%; Table 3). For the rest of the users, the majority of tweets made reference to paracetamol, with this trend being stronger in health care professionals and health institutions, where approximately 3 of every 4 tweets made reference to paracetamol.

The distribution of medical content for each drug was statistically different (P<.001; Table 4). Nonmedical content was predominant (23,871/32,307, 73.9%) in tweets regarding opioids, while in paracetamol tweets, medical content was predominant (33,943/50,822, 66.8%). Statistically significant differences were also found in the distribution of medical and nonmedical content among Twitter users (Table 5). Medical content was the most predominant of all user categories. However, patients were the users with fewer medical content publications (34,144/60,771, 56.2%) and health care professionals were the subgroup with the most publications of medical content (5652/6692, 84.5%).

Within the medical content, we found statistically significant differences in the percentage of tweets that made a positive or negative reference to the efficacy as well as adverse effects between tweets concerning paracetamol and tweets concerning opioids (P<.001; Table 4). In the case of paracetamol, 76.1% (25,814/33,943) of tweets regarding medical content expressed a null or insufficient efficacy, while in the case of tramadol and codeine, this percentage decreased to 70.5% (7636/10,829) and 70.2% (5546/7902), respectively (Table 4). On the other hand, only 6.1% (2083/33,943) of medical tweets regarding paracetamol made reference to an adequate efficacy of the drug. Tramadol reached a higher percentage of tweets in terms of efficacy, whereas codeine reached a lower percentage (Table 4). In terms of side effects, the percentage was higher for the opioid group. The side effects of tramadol and codeine were mentioned in 44.5% (4819/10,829) and 46% (3632/7902) of medical tweets, respectively. Among the nonmedical content, we also found statistically significant differences in the distribution between the different drugs (Table 4). Commercial aspects of paracetamol and tramadol were predominant, in contrast to codeine, where it was a minority. Tweets expressing support and solidarity were the minority in the 3 drugs, while trivialization was abundant, especially in the case of opioids (Table 4). Up to 87.3% (16,493/18,902) of nonmedical tweets referring to codeine trivialized the drug.

Discussion

Principal Findings

In this study, we found that analgesia is a common topic of debate among Twitter users, and patients make up the majority of the participants in these discussions. Health care professionals and health care institutions, despite not being greatly represented in these discussions, tend to generate high levels of interest when they participate. Despite the severity of the matter in discussion, two-thirds of tweets with nonmedical content trivialized drugs, especially tramadol and codeine. Interestingly, in tweets related to tramadol and codeine, nonmedical content is more common, whereas in tweets related to paracetamol, medical content is more common.

Our results show that these drugs are more talked about than chemotherapy, antidiabetics, statins, antidepressants, or antipsychotics [34-36]. This finding can be explained due to the increased presence of pain. Pain can be a symptom that is present in a multitude of diseases, but it can also be a syndrome [3]. Its treatment is a topic of worry for many organizations, given that, in many cases, it is a chronic treatment. On a pharmacological level, the primary tools are opioids because paracetamol and similar drugs usually fall short in their treatment of chronic diseases [37,38]. However, opioids, including the weakest ones, are associated with gastrointestinal side effects in the short term and addiction effects in the long term [39]. The latter is a topic of concern, primarily because of the opioid epidemic that the United States is currently going through and because of the increasing use of these drugs in Western countries [40,41]. In this context, over the past few years, social media have been used as a tool for pharmaco-surveillance, and also to monitor analgesic drugs. A study that analyzed social media posts published in Pennsylvania found a correlation between the number of posts on social media suggesting opioid abuse and the data collected by the National Survey on Drug Use and Health [42]. Another study done in North Carolina found a relationship between the number of tweets and the number of deaths due to synthetic opioid and heroin overdoses [43].

Social media can be a way of accessing social groups that are less represented in traditional studies, such as young adults, Hispanic or Latin, African Americans, and women [44]. Furthermore, given that posts on social media are spontaneous, they can be a better reflection of what patients and the general public think in comparison to studies that use traditional methods [45,46]. In multiple studies, it has been shown that patients tend to show a favorable image of themselves, as well as being complacent with doctors. This results in them omitting their negative opinions and prejudices toward certain drugs to the doctor and expressing these in informal web-based forums instead, such as social media [47]. Furthermore, users on social media post what they think or experience in real time, which avoids the recall bias that is present in medical consultation [34].

Interestingly, 80% (41,080/50,822) of tweets with medical content discuss the efficacy of the drug. This is a higher percentage than what is found in other studies. For example, in a study that analyzed what Twitter users discussed regarding the efficacy of approved drugs to treat obesity, a lower percentage was found [48]. Moreover, a study with similar characteristics that analyzed what Twitter users posted regarding approved drugs for the treatment of attention-deficit/hyperactivity disorder (ADHD) also found a lower percentage of tweets discussing the efficacy of the drug in comparison to this study [49]. In another study that analyzed papers regarding psychotherapy, the percentage of tweets discussing drug efficacy was even lower [50]. These results suggest that when users on social media discuss analgesics, the main worry is the efficacy of the drug, which is not the case in other pathologies. Despite this, what is most interesting is that in this study, the majority of the tweets that reference the efficacy of the drug express that it is null or insufficient, whereas, in the case of the drugs approved for the treatment of ADHD or obesity, the majority of tweets regarding drug efficacy were positive [48,49]. On the other hand, in the study that analyzed posts regarding drugs used for the treatment of ADHD, the percentage of tweets that mentioned adverse effects was greater than the one found in this study [49]. Despite the findings in this study, the tweets that mentioned adverse effects did not receive a greater number of retweets and likes than the other posts, which is different from the findings in other areas of health. For example, a paper that analyzed posts referencing pharmacological interventions used to regulate fertility found that posts referencing adverse effects achieved the greatest retweet-tweet ratio [51]. These findings suggest that, in reference to analgesia, the public is more concerned about the efficacy of the drug. This is probably because pain is one of the most undesirable and incapacitating sensations that can be experienced [3]. However, the impression is that Twitter users underestimate the adverse effects associated with the use of opioids.

Another interesting finding in this study is the limited perception of risk perceived by users regarding the use of weak opioids. The population’s risk perception is one of the key elements when it comes to an adequate use of a drug or substance. For example, it has been estimated that the low perception of risk by adolescents in relation to cannabis use is one of the key factors explaining the high consumption rates [52,53]. This is why it is worrying that most of the tweets posted regarding tramadol and codeine had nonmedical content, which contrasts with other pharmacological groups. For example, in a study that analyzed publications on Twitter referring to antipsychotics, less than 40% of tweets mentioned nonmedical aspects of the treatment [35]. This percentage was even lower (16.32%) in publications about antidepressants [36]. The perceived risk also influences the prescription of opioids by health care professionals. In this sense, a study showed that health care professionals prescribe fewer opioids after being notified of a patient’s death due to an overdose [54].

Furthermore, we found differences in the medical content between paracetamol and weak opioids. Tramadol and codeine were considered effective in a small percentage of tweets, as well as paracetamol, which shows how complex the treatment of pain is. Drugs used to treat other diseases with a worse prognosis, such as schizophrenia, receive a better outcome in that aspect [35]. Multiple motives can explain why a high percentage of users consider these analgesics to have null or low effectiveness. In the first place, pain is a very ego-dystonic symptom. Second, it tends to appear in healthy patients, who until the start of pain, were asymptomatic and tolerated the symptom with difficulty. Third, it is very incapacitating, and as it appears in a functional patient, the impact is greater. Fourth, as there is a stronger existing treatment that is in a way more effective, the population is more demanding as it knows there is a faster alternative to relieve the pain. Finally, there are studies that have evaluated the patient´s perception of the treatment, finding that the positive outcomes of the treatment tend to be transmitted during a medical consultation, whereas the bad experiences are shared on social media or other media platforms [47].

Even though pain is an incapacitating symptom, there have been a low number of tweets demonstrating support and solidarity. This could be due to the invisibility of pain and its unknown origin in many cases. Diseases that, due to their nature or their treatment, are more visual, and we know the cause, generate greater understanding, support, and solidarity than those that are invisible or of a complex nature with social and behavioral implications [55,56]. Frequently, people tend to empathize more when a disease is more notable or manifest than when it is not. This partly explains the stigmatization of mental illness over many years. However, the low number of tweets showing compassion have been retweeted greatly. Twitter is a social media platform to launch campaigns and create awareness [57].

Finally, the theme of trivialization is a matter of concern. On multiple occasions, the trivialization of both physical and mental diseases has been described, for example, for HIV, depression, and psychosis [58-60]. Similarly, a high percentage of trivialization has been described in papers referencing antipsychotics [35]. It is necessary to create sensitizing campaigns to change this engrained mentality within the population. Perhaps this trivialization may be linked to the low percentage of professionals and institutions that have intervened in this debate. In a previous study, a low participation rate of professionals and institutions was also found in social media discussions regarding naloxone and opioids [29]. However, it is important that health care professionals and health care institutions are present in medical debates on social media, given that it is a place where many patients seek information. Particularly, in a survey carried out in the United Kingdom and Ireland, it was found that codeine users are more likely to search for help on the internet than from their doctor [61]. In fact, more than one-third of patients with chronic pain prefer to use the internet to seek information related to pain, and 60% have more confidence in the information found on the internet than the information that their doctor provides [62]. Moreover, when health care professionals intervene on Twitter, their publications are retweeted more often than other users [63]. Additionally, social media users frequently associate paracetamol with depression and suicide attempts which correlates with epidemiologic data given that its wide availability makes it a frequently used drug for self-harm [64,65].

Limitations

This study has certain limitations. First, despite having used a search tool that has access to 100% of tweets, it is possible that tweets that referenced paracetamol, codeine, or tramadol with different keywords than those used in this study may have been left out. Second, Twitter users may include abbreviations, grammatical errors, or slang language, which makes tweet searching more difficult. Third, the succinctness of some tweets, combined with their lack of context, makes them difficult to interpret. Fourth, tweets may not be representative of the views of the rest of society. Finally, the content of tweets regarding analgesics may vary with time. Therefore, despite the time period of this study lasting 2 years, the results may not coincide with what is published today.

Conclusions

The users most actively engaged in these conversations are the patients themselves, suggesting they resort to Twitter to express their distress and seek solutions. Patients who have pain show great interest in finding relief for their symptoms. As a result, their tweets tend to focus on the effectiveness of the drug rather than its possible side effects. This fact can pose risks, among other factors, as a minority appears content with the analgesic efficacy. From a public health point of view, it is useful that health care professionals and institutions are more involved in these discussions, being able to inform people about the effects of the analgesia. Future studies should characterize the type of pain that the users argue about in order to design more targeted interventions. Additionally, it will also be necessary to study what is being published regarding other strategies to alleviate pain, such as surgical interventions or psychotherapeutic interventions. Finally, it is important that the authorities monitor the conversations related to analgesics on social media, given that it has been demonstrated that the purchase without a prescription and the illegal sale of these drugs are common on the internet and in social media [66,67].