Emergency Physician Twitter Use in the COVID-19 Pandemic as a Potential Predictor of Impending Surge: Retrospective Observational Study

Introduction

The contagiousness, fatality rate, and long-term sequelae thus far attributed to COVID-19, the disease caused by SARS-CoV-2, have led to significant strains on the health care system. Since the World Health Organization (WHO) first reported “a cluster of pneumonia cases” in Wuhan, China, on January 4, 2020 [1], the social media platform Twitter has become a source of both official health information and unofficial medical discourse regarding the ongoing pandemic. Boasting 180 million daily active users [2], not only does the service allow account holders to share links, media, and brief strings of text, but it has evolved into a public forum for unvetted information that can augment, if not supersede, more traditional dissemination methods.

On December 11, 2020, the US Food and Drug Administration (FDA) used Twitter to announce its authorization for immediate emergency use of the COVID-19 vaccine developed by Pfizer and BioNTech [3]. In its Twitter message, or tweet, about the decision, the FDA (@US_FDA) reiterated its aim to “assure the public and medical community that it has conducted a thorough evaluation of the available safety, effectiveness, and manufacturing quality information” [4]. Directly addressing Twitter’s medical community in this way was intentional: throughout the COVID-19 pandemic, many physicians turned to social media rather than traditional medical information channels to discuss the merits and demerits of possible treatments, prior to the availability of formal clinical guidance. Myriad treatment modalities and prevention strategies have been proposed at all levels, and Twitter has served as a means of disseminating everything from guidelines and data to anecdotes and opinions [5].

Utilizing social media to aid in the mapping of an ongoing crisis is not new, and Twitter use has previously been linked to, among others, the H1N1 and Zika virus epidemics [6,7]. Yet even with unparalleled international effort, formal forecasting models of COVID-19 have largely failed [8], and many geopolitical comparisons in popular media now in hindsight appear to have been premature [9-12]. As the front and, for many Americans, only door into the US health care system, emergency departments continue to be looked to for public health surveillance and treatment strategies, as a kind of finger on the epidemiological pulse of their communities [13,14].

Emergency physicians, in particular, have long been at the forefront of physician engagement with social media, relying on a budding network of fellow clinicians collaborating on what has become known as free open-access medical education [15]. The COVID-19 pandemic only further accentuates the unique role of the emergency physician community online, as frontline providers who not only take on substantial risk but who may also be able to provide substantial insight. Facing changed admission criteria, expanded alternate care sites, and recycled equipment, emergency physicians have been forced into the unenviable position of making difficult triaging and resource allocation decisions. This study, therefore, seeks to characterize the sentiment and topic trends in emergency physician discourse on Twitter throughout the prevaccination pandemic, as a potential harbinger of the surge needs that followed.

Methods

Sampling and Data Collection

This work was approved as exempt human subjects research through the Beth Israel Deaconess Medical Center Institutional Review Board in Boston, Massachusetts. In order to access Twitter’s application programming interface (API), a developer account was applied for and obtained. Python 3.8.5 and the Tweepy library (Python Software Foundation) [16] then made it possible to acquire all unique followers of the three major physician professional societies in emergency medicine: the American College of Emergency Physicians (ACEP; @ACEPNow), the Society for Academic Emergency Medicine (SAEM; @SAEMonline), and the American Academy of Emergency Medicine (AAEM; @aaeminfo) [17-21]. Because sex and gender are not directly recorded by Twitter but have previously been shown to influence social media engagement and even clinical diagnosis and management [22-24], gendered nouns and pronouns stated in user bios were considered in their place. Those users with privacy settings that would render tweets protected from analysis were removed. Each user bio was then initially screened by textual search for including any of the 157 text strings decided by the research team as connoting a public acknowledgement of one’s role as an emergency physician, such as “emergency medicine physician,” “emergency D.O.,” or “ER doc.”

Exclusion criteria included aspiring emergency physicians and students, organizations, physicians from other specialties, as well as users belonging to other professions, living outside the United States, or without a clear location at the state level. However, emergency physicians still in training, whether described as interns or residents, were not excluded. Exclusion for any of these reasons was determined by two practicing emergency physicians each reviewing and sorting all users manually and independently, with any discrepancies decided by consensus.

A chain-referral sampling method was then employed in order to expand the study group to include those US-based emergency physicians on Twitter not following one of the major professional organization accounts [25]. Followers of already-included participants were then aggregated to create a composite list of potential additional participants. After applying the same exclusions, this new group of users was then appended to the original as a more comprehensive sampling of US emergency physicians on Twitter.

All available tweets up to Twitter’s own limit of 3200 per user were acquired for each study participant. Tweets were removed if reposted as a retweet from a different post, if non-English, or if falling outside the study period from and including January 4, 2020, based on the date of the initial WHO announcement, to and including December 14, 2020, based on the date of the first FDA-approved vaccination in the United States [26].

Sentiment and Topic Generation

Several different methods have previously been employed to conduct sentiment analysis of tweets specific to the health care field, with 46% of such tweets demonstrating sentiment of some kind [27]. Here, the open-source Valence Aware Diction and Sentiment Reasoner (VADER) analysis tool was used to determine both the direction and extent of tweet sentiment polarity, based on a lexicon of sentiment-related words. VADER has been shown to outperform human raters and, in handling emoji and slang, is particularly suited for social media text [28,29]. Sentiment polarity ratings were summed and standardized as a compound score between –1 and 1. By convention, tweets with a compound score between –0.05 and 0.05 were classified as neutral [29,30].

Using the gensim Python package, all tweets were tokenized and preprocessed, including removal of punctuation, special characters, mentions of other users, stop words of little topic value, and links to external websites. Hashtags, which users sometimes use to denote a contextual theme [31,32], were converted to text. Because frequently co-occurring words can exist with unique meaning, two- or three-word phrases were also considered as independent tokens, as in “healthcare_workers.” These preprocessed tweets then underwent unsupervised topic modeling in order to discern meaningful content themes. Latent Dirichlet allocation (LDA) is a common method for topic modeling that has previously been utilized to analyze health care–related tweets [33,34]. Topic coherence, as proposed by Röder et al due to its higher correlation with human topic ranking [35], was then maximized by iteratively modeling over a range of topic numbers as well as α parameters. The resulting topic model was then used to assign a dominant topic to each tweet included in the sample.

In order to contextualize these sentiment and topic trends within the ongoing pandemic, daily COVID-19 case counts and COVID-19 inpatient bed utilization (CIBU) rates were acquired from the US Centers for Disease Control and Prevention and the US Department of Health and Human Services [36,37]. These data were converted to 7-day simple moving averages to account for lower weekend reporting and other daily fluctuations [38,39]. Tweet volume, sentiment, and dominant topic trends were then correlated with new COVID-19 cases and CIBU through Prism, version 9.0.2 (GraphPad Software).

Further comparison between public health and Twitter data was made possible by plotting the 7- and 28-day simple moving averages and observing their intersection as a potential indicator for momentum using Excel, version 16.47.1 (Microsoft). Similarly, a moving average convergence/divergence oscillator (MACD) was generated by subtracting the 28-day exponential moving average from the 7-day exponential moving average. This MACD was then monitored for both (1) turning positive and (2) crossing above its own 7-day exponential moving average. These cross signals based on simple moving averages and on the MACD are both loosely derived from lagging indicators of historical price patterns that are commonly used in finance to guide investment decisions and have previously been applied directly to SARS-CoV-2 infection data [40,41].

Results

The three key US emergency physician organizations had a collective 42,918 followers of their primary Twitter accounts as of December 11, 2020. When those following more than one professional organization were only counted once, there were 27,022 unique followers, with 10,905 (40.4%) belonging to at least two of the three groups (Figure 1). As an approximation for cohesion, the overlap coefficient of the three handles was 0.43, calculated as the ratio of the intersection over the maximum possible intersection ([A∩B∩C]/min[|A|,|B|,|C|]) [42]. After exclusions, 2073 US-based emergency physicians were identified, with high interrater reliability (κ=0.96).

‎View this figure

There were 1,510,802 followers of the initial cohort acquired on December 12 and 13, 2020, with 734,644 found to be internally unique as well as distinct from the original user list assessed. Applying the same inclusion and exclusion criteria resulted in 3110 emergency physicians, 1433 of whom could clearly be identified as located in specific US states, territories, or districts through their public Twitter location and description (κ=0.94). Combining the two groups, there were 3463 US-based emergency physicians included in the study.

Study participants had been using Twitter for an average of 6.6 (SD 3.5) years, with an average of 183.8 (SD 491.0) total tweets (Table 1). Only 910 out of 3463 (26.3%) participants explicitly described themselves as a resident or intern currently in training. The most common US states represented were New York (433/3463, 12.5%) and California (395/3463, 11.4%), and the most contributory US region was the Northeast (1057/3463, 30.5%). Self-identified gender was infrequent (902/3463, 26.0%), with 466 of those 902 participants (51.7%) identifying as a man.

Tweets collected for the study group totaled 1,941,894 as of December 24, 2020, with 630,915 (32.5%) of those obtained falling between January 4 and December 14, inclusive (Figure 2). Because of a cap on the number of tweets able to be pulled through the official Twitter API, 44 out of 3463 (1.3%) users appeared to have exceeded the limit, such that not all tweets would have been captured. Despite truncation, these avid users still contributed 140,938 of all 1,941,894 (7.3%) collected tweets. Overall, 256,636 (40.7%) retweets and 39,532 (6.3%) non-English tweets were removed, leaving 334,747 (53.1%) unique English-language tweets for analysis. Daily volume went from a pre-March mean of 481.9 (SD 72.7) tweets to a mean of 1065.5 (SD 257.3) tweets thereafter.

‎View this figure

After preprocessing, 1,958,230 semantic units, or tokens, were found within the corpus of tweets, with a total vocabulary of 12,401. LDA modeling over a range of topic numbers and parameters settled on a total of 20 content topics for this study. Two physicians then worked together to manually and jointly label these 20 topics based on discussion of key terms and representative tweets. For example, the topic with the top five terms of “resident,” “student,” “residency,” “learn,” and “year” was labeled as medical training. In this way, the most prevalent topics were found to relate to the health care system (45,570/334,747, 13.6%), collaboration (20,112/334,747, 6.0%), and politics (18,186/334,747, 5.4%) (Table 2). Notably, the health care system was the dominant topic throughout the study period, with two exceptions: it was supplanted from June 2 to 9 by race relations and from November 6 to 10 by politics.

Daily change in 7-day moving averages for specific tweet topics and sentiment polarity demonstrated small Pearson correlation coefficients for the topics of pandemic response (r=0.26, 95% CI 0.15-0.36), epidemiology (r=0.25, 95% CI 0.14-0.35), and clinical care (r=0.23, 95% CI 0.13-0.33) with reported COVID-19 cases (all P<.001) (Table 2). There was greater correlation for these topics with hospital bed utilization (all P<.001). While the three topics considered jointly were even more correlated (r=0.41, 95% CI 0.31-0.50), they still fell short of the correlation seen with the 9.4% of included tweets containing “covid,” “coronavirus,” “corona virus,” “cov-2,” “cov2,” or “pandemic” within the tweet text (r=0.47, 95% CI 0.38-0.56) (all P<.001). A proportional stacked area chart reveals an early overall increase in Twitter use as daily case counts rose, particularly among COVID-19–related topics (Figure 3). Aggregated sentiment scores reached a nadir on June 6, when race relations was the dominant topic in the sample, and again on October 6, the day after then–US President Donald Trump was discharged from his COVID-19 hospital admission [43].

‎View this figure

Over the full study period, three peaks emerged in both COVID-19–related discussion and CIBU, with the rise in tweets appearing to precede the corresponding rise in CIBU. This may be better appreciated with attention directed to where the 7-day moving average crosses above the 28-day moving average as a signal of topic momentum (Figure 4). After the first recorded domestic COVID-19 case on January 22, 2020, February 25 was the first such cross and preceded a period of sustained increase in CIBU from February 28 to an April 9 peak. The next occurrence was on June 22, occurring alongside the second period of sustained increase in CIBU from June 15 to a peak on July 21. A brief cross on July 31 was short-lived, but the subsequent cross on September 13 was maintained and corresponded to a rise in COVID-19 hospital burden that started September 24 and continued through the remainder of the study period, with several episodic crosses seen thereafter. When the MACD is also considered, February 22 (Figure 4, point A) marks a cross above both the zero centerline as well as its 7-day moving average, 46 days before the April 9 peak in CIBU. The next such cross occurred on June 24 (Figure 4, point B), 27 days before the second peak, while the third surge in CIBU appeared to coincide with a first crossing on September 30 (Figure 4, point C).

‎View this figure

Because the breadth and diversity of the United States may obfuscate local trends, the four most contributory states of California, New York, Pennsylvania, and Texas were similarly plotted (Figures 5-8). All four experienced a spring signal and subsequent surge, although New York has been recognized among them as an early epicenter [44]. Only Texas appears to have had a sustained cross of the 7-day moving average above the 28-day moving average from June 18 to July 15. This notably preceded the only significant summer peak among these states, reaching a maximum CIBU of 20.5% on July 20; in comparison, California reached a second peak of 14.3% on July 25 while neither New York nor Pennsylvania exceeded 10% again before November. The mean time from the preceding cross of moving averages in COVID-19–related emergency physician tweets to peak CIBU across the four states and the nation was 45.0 (SD 12.7) days.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

Discussion

Principal Findings

Emergency physician engagement on Twitter has grown considerably since the start of the COVID-19 pandemic in both the topics raised and the sentiments conveyed. Furthermore, the analysis described here demonstrates conversations with increasing focus on pandemic response, clinical care, and epidemiology. That these topics correlate better with CIBU than simple case counts supports the idea that they may well serve as a kind of barometer of health care system strain. Momentum in these conversations, in fact, as shown by crossings of tweet count moving averages, were shown to occur before key rises in CIBU, which may with future research lend itself to the larger effort of predicting surge based on multiple data streams.

The COVID-19 pandemic has required emergency physicians to adapt continually to an extraordinary inadequacy of resources. While strains on the ventilator supply received much attention [45], early limitations in testing and bed availability also created significant clinical challenges [46], let alone the mental health effects that are likely to be far-reaching [47]. In the case of personal protective equipment (PPE) shortages, many frontline providers resorted to individual means to acquire makeshift supplies, and some even turned to social media, as in the case of the #GetMePPE Twitter hashtag, in order to spur necessary action [48-50]. Despite that potential good, such public distress and debate from the medical frontline has, at times, spurred controversy and even real-world, professional repercussions [51].

With overwhelming caseloads and without clinical consensus, frontline physicians have been forced to decide between various treatment modalities based on unclear and, at times, contradictory information, with significant moral distress [52]. The effort to maintain appropriate patient care despite these unknowns, when faced with a need for resource rationing [53], is a de facto implementation of crisis standards of care. While contingency planning is situational and incorporates some aspects of triage practiced routinely in overcrowded emergency departments across the country, the formal triggering of crisis standards, and implicit divergence from conventional standards, enacts systematic change in protocols and care plans during a sustained period of large-scale strain [54-56].

Taken together, this retrospective look at emergency physician Twitter use suggests a new way of considering the pandemic surge, as emergency physician utilization of Twitter reached unprecedented highs. There are likely several reasons for this. The online community has been shown to provide psychological benefit, potentially exacerbated by the isolation faced in providing crisis care, and by a perceived collapse in trust in the existing infrastructure and policy guidance [57,58]. That collaboration was the only topic among 20 to have a compound sentiment range that did not cross zero may relate to this yearning for support. Still, positive mean sentiment scores among the vast majority of topics were unexpected, given recent work pertaining to general public perceptions of the pandemic [59]. There may well be some sway to a self-perceived personal and professional connection to the dominant issue of the day. While the root cause is undoubtedly complex and multifaceted, the increase in emergency physician engagement with social media is likely here to stay.

Whether emergency physicians online can truly act as an early indicator for policy makers remains to be seen, but the community is undoubtedly a subset of the broader pandemic response and is worth looking at more closely. Moving averages have been used to indicate movement in financial markets but are not true predictors of future trends. Given the potential for false signaling and the challenge in determining what constitutes a sustained or meaningful cross prospectively, derived crosses of the kind shown here will likely need corroboration from a variety of other metrics as well as comparison to other samples and controls. Even so, the idea that an indicator of a physician behavioral trend online may also signal momentum in real-world hospital bed utilization has clear implications for the future. This is particularly relevant when it is considered that the study group itself was sourced directly from followers of major professional organizations for whom early recognition of surge would empower a more coordinated and efficacious policy response.

To make such a tool operationally relevant, collaboration between government and private sector partners will likely be necessary to build adequate data pathways allowing for public health surveillance in real time. While emergent topic generation from a retrospective corpus of tweets is not a feasible option for rapid and predictive modeling, this work suggests that even a simple collection of tweets containing disease-specific text strings can nonetheless yield important, potentially meaningful information to inform resource allocation and other policy decisions. Ultimately, all disasters are media events, affecting both how and what information is conveyed. There is, therefore, no great leap of faith in acknowledging that social media, too, may have an important part to play. Future research must delve further into how such tools can one day be used in the early recognition of, and response to, health care strain of such magnitude.

Limitations

In holding to strict inclusion criteria, this study overlooked Twitter users who were not explicit in their self-identification as practicing emergency physicians. Emergency physicians were made the narrow focus of this work based on their key roles as clinical decision makers overseeing department throughput, but inclusion of nurses, technicians, and nonphysician midlevel providers may add breadth. Additionally, follower referral from within the sample may have introduced bias that could have been avoided by subsampling with some individuals selected for study participation and others only for referral [25]. Even so, a comprehensive list of emergency physicians on Twitter compiled in 2016 concluded that there were only 2234 such users around the globe [60]. Social media use has undoubtedly risen since, particularly with the influx of a growing number of emergency medicine residents [61], but the sample provided here does appear to be appropriately sized for this purpose.

Even so, this sample size was insufficient in both participant number and geographic spread to allow for more granular geographic analysis by city or county, although public health surveillance often occurs at this level [62,63]. Both demographic characteristics and spaciotemporal effects at the level of the individual participant have previously been shown to bias tweet sentiment and content, but these were not controlled for in this study [64,65].

Reliance on Twitter may itself limit generalizability, given its comparatively higher representation of young, urban, and minority users when compared with the general US population [66]. Only one social network platform was analyzed, and, insofar as it serves as a public forum, what medical professionals say online does not necessarily correlate with what they think or feel [67]. The study group, however, was not aware of its participation in this research, thereby avoiding that influence on behavior [68]. Excluding reposted tweets may have overlooked certain sentiments and reactions. Additionally, LDA topic modeling depends not only on the size of the overall corpus but on the length of the individual documents themselves. Although methods such as aggregation into larger documents have been proposed in order to overcome tweet brevity [69], doing so would not have allowed for the temporal and user-specific analysis intended. Finally, care must be taken in interpreting relationships between variables, such as physician tweets and CIBU, when both variables have undergone averaging or smoothing, which can sometimes suggest correlation where none exists.

Conclusions

This work reveals both the opportunity and the pressing need to explore social media use by the emergency physician community as a means of anticipating surge needs. By acting as gatekeepers to the hospital, emergency physicians are uniquely positioned to act as early indicators of hospital surge, and finding methods such as Twitter usage, which can track and analyze these indicators, could be vital to future pandemic planning and response.