Published on in Vol 22, No 11 (2020): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/22600, first published .
Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media

Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media

Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media

Original Paper

1School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, United States

2Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States

3Department of Psychiatry, University of Rochester, Rochester, NY, United States

Corresponding Author:

Koustuv Saha, BTech

School of Interactive Computing

Georgia Institute of Technology

North Ave NW

Atlanta, GA, 30332

United States

Phone: 1 4046929496

Email: koustuv.saha@gatech.edu


Background: The COVID-19 pandemic has caused several disruptions in personal and collective lives worldwide. The uncertainties surrounding the pandemic have also led to multifaceted mental health concerns, which can be exacerbated with precautionary measures such as social distancing and self-quarantining, as well as societal impacts such as economic downturn and job loss. Despite noting this as a “mental health tsunami”, the psychological effects of the COVID-19 crisis remain unexplored at scale. Consequently, public health stakeholders are currently limited in identifying ways to provide timely and tailored support during these circumstances.

Objective: Our study aims to provide insights regarding people’s psychosocial concerns during the COVID-19 pandemic by leveraging social media data. We aim to study the temporal and linguistic changes in symptomatic mental health and support expressions in the pandemic context.

Methods: We obtained about 60 million Twitter streaming posts originating from the United States from March 24 to May 24, 2020, and compared these with about 40 million posts from a comparable period in 2019 to attribute the effect of COVID-19 on people’s social media self-disclosure. Using these data sets, we studied people’s self-disclosure on social media in terms of symptomatic mental health concerns and expressions of support. We employed transfer learning classifiers that identified the social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support). We then examined the changes in psychosocial expressions over time and language, comparing the 2020 and 2019 data sets.

Results: We found that all of the examined psychosocial expressions have significantly increased during the COVID-19 crisis—mental health symptomatic expressions have increased by about 14%, and support expressions have increased by about 5%, both thematically related to COVID-19. We also observed a steady decline and eventual plateauing in these expressions during the COVID-19 pandemic, which may have been due to habituation or due to supportive policy measures enacted during this period. Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis.

Conclusions: We studied the psychosocial effects of the COVID-19 crisis by using social media data from 2020, finding that people’s mental health symptomatic and support expressions significantly increased during the COVID-19 period as compared to similar data from 2019. However, this effect gradually lessened over time, suggesting that people adapted to the circumstances and their “new normal.” Our linguistic analyses revealed that people expressed mental health concerns regarding personal and professional challenges, health care and precautionary measures, and pandemic-related awareness. This study shows the potential to provide insights to mental health care and stakeholders and policy makers in planning and implementing measures to mitigate mental health risks amid the health crisis.

J Med Internet Res 2020;22(11):e22600

doi:10.2196/22600

Keywords



The impacts of global public health emergencies extend beyond medical repercussions; they affect individuals and societies on many levels, causing disruptions [1,2]. In an article written by the American Psychological Association following the Ebola outbreak [3], the epidemic was described as an “epidemic of fear.” In the United States, it was labeled by the media as “fearbola” to describe a paranoia that infected the country. Reports of similar feelings of anxiety, stress, and uncertainty have been repeatedly reported during other global outbreaks and pandemics [4]. The ongoing outbreak of SARS-COV-2 has led to a pandemic of illness (COVID-19) that has caused 16 million cases and 700,000 deaths worldwide, reported as of the end of July 2020 [5]. According to recent surveys from the Census Bureau, the Centers for Disease Control and Prevention, and other studies, the COVID-19 crisis has been reported to be associated with rapid rises in psychological distress across many nations [6], with women, the young, the less educated, and some ethnic minority groups reporting greater mental health strain [7]. On the one hand, people are worried about the direct effects of potential infection, including fears of death, lasting disabilities, or exacerbating chronic illnesses. On the other hand, actions to mitigate the spread of COVID-19, including social distancing, quarantines, and business closures with resulting job losses, are a source of life disruptions and emotional distress.

Fear and anxiety about a disease as infectious as COVID-19 can trigger new-onset or exacerbate existing mental illness [8]. Therefore, the practical impact of the crisis is far greater than the actual number of infection cases or fatalities [8,9]. Although expressions of distress may stem from concern and worry relating to the direct impacts of the disease, they may relate as much to disruption of regular routines, sleep and eating patterns, having children out of school and at home full-time, economic hardships and unusual volatility in financial markets, and forced geographical displacement or confinement. Indeed, some people are at risk of developing posttraumatic distress due to exposure to the multifaceted uncertainties or from confronting dying people or lost loved ones. Although disease mitigating efforts such as “social distancing” and “self-quarantining” are recommended [10-12], individuals in medical isolation may experience increased symptoms of anxiety and depression, as well as feelings of fear, abandonment, loneliness, and stigmatization [13,14].

Despite concerns about the myriad of social and behavioral issues associated with the COVID-19 pandemic [15,16], there has been scant research to examine its psychosocial impacts or how to predict and mitigate them. Although it is anticipated that COVID-19 will have broadly ramifying effects [17,18], public health workers and crisis interventionists are limited in their ability to extend services and support in a timely, preemptive fashion. Although surveys are a step forward to support such efforts [7], due to their retrospective recall bias, limited scalability, and inability to provide real-time insights, public health workers are not only unable to prioritize services for the most vulnerable populations but also, more specifically, less equipped to direct prevention efforts toward individuals with greater propensities for adverse psychological impacts.

This paper seeks to address the aforementioned gap by drawing insights into people’s expressed mental health concerns by leveraging social media data. The rise in online and social media activity has provided an unprecedented opportunity to enhance the identification and monitoring strategies of various mental and psychosocial disorders [7,19]. Over 80% of US adults use social media daily [20], placing it ahead of texting, email, and instant messaging, and disclose considerably more about themselves online than offline [21,22]. Social media provides a real-time platform where people often candidly self-disclose their concerns, opinions, and struggles during this pandemic [23]. In particular, our study targets the following research aims:

  • Aim 1: To quantitatively assess the psychosocial effects of the COVID-19 pandemic using social media data
  • Aim 2: To examine how the psychosocial effects of the COVID-19 pandemic have varied over time
  • Aim 3: To examine if social media language reflects the major psychosocial concerns during the COVID-19 pandemic

For these research questions, we measured psychosocial effects in terms of symptomatic mental health expressions of anxiety, depression, stress, and suicidal ideation, and expressions seeking emotional and informational support. Our study is founded on a large body of work on studying mental health and psychosocial dynamics with social media data [24-30]. Several studies have leveraged Twitter (which is also the data we used) to study health attributes and public health [30], including symptoms related to diseases [31], disease contagion [32], obesity and physical health [33], mood and depressive disorders [28,34], mental health self-disclosures [27], posttraumatic stress disorder [35], addictive behaviors and substance use [36,37], etc. Because social media data (and Twitter posts in particular) are recorded in the moment, they provide rich information about both the individual as well as the larger world [30]. In particular, we draw on two kinds of prior work: symptomatic mental health expressions and support expressions. Related to the former, Saha et al [25], in their study on the effects of psychiatric medications on Twitter, developed classifiers of mental health symptomatic expressions using social media language, which we replicated in this study. Related to the latter, we draw upon Sharma and De Choudhury’s [38], and Saha and Sharma’s [39] developed classifiers of social support expressions, specifically emotional and informational support.


Data

Using Twitter Data

To study people’s psychosocial expressions on social media, we obtained Twitter data. Twitter is one of the most popular social media platforms, and its public-facing, microblogging–based design enables candid self-disclosure and self-expressions for individuals [27].

Twitter Streaming Application Programming Interface

We collected data in our study using the Twitter Streaming application programming interface (API). The Twitter Streaming API is an official data collection API that Twitter shares with researchers providing free access to a 1% sample of its data on parameters set by researchers. That is, for a given set of parameters, Twitter queries the volume of available data at a particular moment [40]. If the volume of the query exceeds 1% of all Twitter posts at that moment, then the response is sampled to be less than 1%. However, the Twitter Streaming API is like a black-box with a lack of transparency in the sampling methodology [41,42], yet this is one of the few forms of unfettered and large-scale social media data access to researchers outside social media companies [43] and has been used in prior research, including in health-related studies [35,44,45]. The Discussion section revisits the limitations of our study due to the challenges of the Twitter Streaming API.

For the purposes of our study, we used two kinds of parameters: (1) language of a Twitter post as “english” and (2) geolocation bounds set to be within the geographic coordinates of the United States. Therefore, our following analyses concern Twitter data that at least fulfill both these criteria. We note that the location filter additionally prevents any retweets in the data set, as retweets are not geolocation labeled by design on Twitter [40], allowing us to study only originally created Twitter posts.

Treatment Data

In particular, we focused our study on the US population and leveraged the Twitter Streaming API. Using geo-bounded coordinates, we collected 1% of real-time Twitter data originating from the United States. We collected 59,096,694 Twitter posts between March 24, 2020, and May 24, 2020. Because this data set comes from the same period when the COVID-19 outbreak occurred, we labeled this data set as the treatment data set. We note that this period saw an exponential growth in reported COVID-19 infection cases (about 50,000-1 million) and fatalities (about 1000-56,000) in the United States [46]. During these 2 months, federal and state policies and laws were enacted to control or mitigate the spread of the outbreak, including school and work closures; stay-at-home orders; and the Coronavirus Aid, Relief, and Economic Security Act [47].

Control Data

To understand the social media expressions particularly attributed to the COVID-19 crisis, we obtained a control data set that originated from the same geographical location (the United States) and a similar time period but from the previous year (2019). Prior work [47,48] motivated this approach of obtaining control data that acts as a baseline and likely minimizes confounding effects due to geo-temporal seasonality in lifestyle, activities, experiences, and unrelated events that may have some psychosocial bearing. We obtained a similarly sized data set of 40,875,185 Twitter posts shared between March 24, 2019, and May 24, 2019.

Both the treatment and control data sets were collected in real time, and therefore, they were the entire 1% sample of Twitter posts returned in real time; we did not conduct any additional sampling on this data. We note that the size of the control data was smaller than that of the treatment [42] despite each consisting of the same 2-month duration. This could be because the volume of posts [40,42] on Twitter increased significantly in 2020 [49], leading to an increase in the 1% sample as well. However, we cannot make any such conclusion, especially because of the lack of transparency in how Twitter conducts the 1% sampling [42].

Psychosocial Effects of COVID-19

Study Design

Toward our first research aim of understanding the psychosocial impacts of the COVID-19 outbreak, we conducted two types of analysis on our Twitter data set, which we describe in the following sections. Our study builds upon the vast, rapidly growing literature studying mental health concerns and psychosocial expressions within social media data [19,21,24-28,34,48,50-52]. We adopted a quasi-experimental study design, which minimizes for geo-temporal confounds by using Twitter data sets from similar regions and similar times of the year in a treatment (2020) and a control (2019) year.

Symptomatic Mental Health Expressions

Drawing on the work previously referenced, we hypothesize that people’s self-disclosure expressions on social media can reveal symptomatic mental health expressions attributed to the COVID-19 crisis. We examined symptomatic expressions of anxiety, depression, stress, and suicidal ideation. These are not only some of the most critical mental health concerns but also have been attributed to be consequences of the pandemic outbreak [15,53,54].

To identify mental health symptomatic expressions in social media language, Saha et al [25] built machine learning classifiers using transfer learning methodologies—the main idea here is to infer mental health attributes in an unlabeled data by transferring a classifier trained on a different labeled data set. These classifiers are n-gram–based (n=1,2,3) binary support vector machine (SVM) models, where the positive class of the training data sets stems from appropriate Reddit communities (r/depression for depression, r/anxiety for anxiety, r/stress for stress, and r/SuicideWatch for suicidal ideation), and the negative class of training data sets comes from nonmental health–related content on Reddit—a collated sample of 20 million posts gathered from 20 subreddits from the landing page of Reddit during the same period as the mental health subreddit posts, such as r/AskReddit, r/aww, and r/movies. These classifiers perform at a high accuracy of approximately 0.90 on average on held-out test data [25].

Clinical Validity

Saha et al’s [25] classifiers used here have also been shown to transfer well on Twitter with an 87% agreement between machine-predicted labels and expert appraisal [48], where experts annotated posts in the classification test data using Diagnostic and Statistical Manual of Mental Disorders, 5th Edition [55] criteria of mental health symptoms. Bagroy et al [56] reported additional validation of such derived insights with feedback from clinical experts [55]. In this study, the outcomes of the mental health expression classifiers were compared with those given by human coders on the same (random) sample of social media posts; the latter coded the posts based on a codebook developed using prior qualitative and quantitative studies of mental health disclosures on social media and literature in psychology on markers of mental health expressions. Coders not only agreed with the outcomes of the classifiers (κ=0.83) but also noted that the classifiers could identify explicit expressions of firsthand experience of psychological distress or mental health concerns (“i get overwhelmingly depressed”) as well as expressions of support, help, or advice around difficult life challenges and experiences (“are there any resources I can use to talk to someone about depression?”). Further details about these classifiers, including their detailed performance, predictive features demonstrating model interpretability, and efficacy of transfer to Twitter data, can be found in [25,48,56]. We used these classifiers to machine label both our treatment and control data sets. Textbox 1 shows example Twitter posts in our data set that exhibit mental health symptomatic expressions (because many of these labels were comorbid, we show example posts that exhibit one or more of these mental health symptomatic expressions).

Example paraphrased posts in the treatment data that exhibited high symptomatic mental health expressions.
  • I am so sick and tired of the #coronavirus (anxiety, stress)
  • The kind of person I am, I don’t deserve to meet these people (depression)
  • 2020 is the saddest year. There is a lack of money, necessities needed for daily life are gone from stores! I am at work as a healthcare professional hurting for my patients because they can’t see their family. (anxiety, stress)
  • I am too overwhelmed by school and having a crippling anxiety to keep up with everything online I’m seriously NOT OK! I CAN’T TAKE THIS! (anxiety, suicidal ideation)
  • During the online lecture, the prof. asked each of us how we were doing and feeling, I said, I am too anxious to know what’s next, and i keep thinking about what’s gonna happen, and she was like okay chill! (depression, anxiety, stress)
Textbox 1. Example paraphrased posts in the treatment data that exhibited high symptomatic mental health expressions.
Support Expressions

Social support is considered an essential component in helping people cope with psychological distress [57]. Research reports that supportive interactions can even have a “buffering effect” [57,58]; that is, they can be protective against the negative consequences of mental health. With the wide adoption of web and social media technologies, support seeking (and providing) is increasingly happening online and has been shown to be efficacious [21,59]. In fact, a meta-analysis indicated that online support is effective in decreasing depression and increasing self-efficacy and quality of life [60]. In the context of suicide, certain types of social support in Reddit communities may reduce the chances of future suicidal ideation among those seeking mental health help [61]. Oh et al [62] further showed that surveyed Facebook users demonstrated a positive relationship between having health concerns and seeking health-related social support. Indeed, during global crises such as COVID-19, when many of the physical sites for health care (including mental health) have been closed or have restricted access, it is likely that online support has proliferated [63]. Fear of potential infection may further have alienated individuals in need to pursue formal treatment, therapy, and support, perhaps channelizing their support-seeking efforts online and on social media.

According to the “Social Support Behavioral Code” [64], two forms of support that have received theoretical and empirical attention are emotional and informational support. Emotional support (ES) corresponds to empathy, encouragement, and kindness, while informational support (IS) corresponds to information, guidance, and suggestions [38,65]. These two forms of support have been found to be most prevalent and effective in several studies of online support and social media [38,62,66,67]. Social media enables individuals to self-disclose and express their emotional and informational needs [67]. Andalibi et al [66] found that these two kinds of support can co-occur with other forms of support, such as posts seeking ES often seeking esteem and network support, and Attai et al [68] noted that Twitter is effective in seeking and providing health-related informational needs, contextually related with our problem of interest.

To identify support expressions on social media, we used an expert-appraised data set and classifier built in prior studies [38,39]. These are binary SVM classifiers identifying the degree (high and low) of ES and IS in social media posts. When the predictions of these classifiers were cross-validated with expert annotations from Sharma and De Choudhury’s [38] data, the classifiers were found to have k-fold cross-validation accuracies of 0.71 and 0.77 in ES and IS classifications, respectively [39]. Similar to the symptomatic expressions classifiers, the classifiers of support expressions are transferred from Reddit and typically performed well in our data set due to the high linguistic equivalence between Reddit and Twitter data sets [34]. We further manually inspected a random set of 125 Twitter posts in our data set using the methods outlined in prior studies [25,56] to rate each Twitter post with binary high or low ES and IS. We found that the manual ratings and classifier ratings showed a high agreement of 88% and 93%, respectively, indicating statistically significant transfer classification on Twitter. We used these classifiers to label the presence of ES and IS in our treatment and control data sets. Textbox 2 shows a few example paraphrased posts of support expressions in our treatment data set.

Example paraphrased posts in the treatment data on support expressions.
  • To our residents, the town is here for you but we need your help if we are going to keep our hospitals from being overwhelmed. (emotional and informational support)
  • I hope you are safe and healthy! Keep the faith God never fails us & always has perfect plan. (emotional support)
  • my wife is laid off. She has been trying for days to get ahold of unemployment. Please help. We need income. Running low on basic things. (emotional and informational support)
  • Dear God, we are going through some struggles these days. Could you please send us some sunshine? Thanks so much. (emotional support)
  • According to my mom, kindness is needed more than ever now. So, send love to you! How are you being kind to others today? (emotional support)
Textbox 2. Example paraphrased posts in the treatment data on support expressions.

Examining Psychosocial Expressions Over Time and Language

Approach Overview

Next, we describe the methods to examine how the COVID-19 pandemic may have caused changes in psychological expressions by comparing our treatment (outbreak year) and control (no outbreak year) data sets. For both our data sets, we aggregated the number of posts that expressed symptomatic and support expressions by day and by type. We compared the pervasiveness of each kind of measure in the data sets along with conducting statistical significance in their differences using two-sample t tests and effect sizes (Cohen d).

Temporal Variation

For our second research aim, we compared the daily variation of measures between the treatment and control data sets, we transformed our data into standardized z scores. Our data sets relied on the Twitter Streaming API and were subject to daily inconsistencies of available data each day [41]. Transformed z scores are not sensitive to such absolute values and inconsistencies, and essentially quantify the number of SDs by which the value of the raw score is above or below the mean. Similar standardization techniques have been adopted in prior social media time series studies [48,69]. Z scores were calculated as (x − μ) / σ, where x is the raw value, μ is the mean, and σ is the SD of the population. Here, to obtain population μ and σ, in addition to our treatment and control data, we also included a year-long Twitter data of over 240 million Twitter posts (September 2018 to August 2019). For each of the measures in symptomatic and support expressions, we first calculated μ and σ on the per-day occurrence of that particular measure in the data set of over 300 million Twitter posts (combining 240 million posts between September 2018 and August 2019, and 60 million posts in the treatment data between March and May 2020). For each measure, we then calculated the z score per day and interpreted the positive z scores as values above the mean and negative z scores as those below the mean.

Linguistic Differences

For our third research aim, we examined COVID-19–related linguistic differences in the psychosocial expressions on social media, we employed an unsupervised language modeling technique, the Sparse Additive Generative Model (SAGE) [70]. Given any two data sets, SAGE selects salient keywords by comparing the parameters of two logistically parameterized-multinomial models using a self-tuned regularization parameter to control the trade-off between frequent and rare keywords. We conducted SAGE to identify distinguishing n-grams (n=1,2,3) between the treatment and control data sets, where each n-gram was returned with a SAGE score. The magnitude of an n-gram’s SAGE score signals the degree of its “uniqueness” or saliency, and in our case, a positive SAGE score (above 0) indicated that the n-gram was more salient in the treatment data, whereas a negative SAGE score (below 0) denoted greater saliency in the control data.

SAGE allowed us to obtain how the expressions differ during the COVID-19 outbreak as compared to the control period. We conducted two SAGE analyses, one each for symptomatic and support expressions. For the symptomatic expressions, we first obtained posts that were indicative of either anxiety, depression, stress, or suicidal ideation in the treatment and control data sets, and obtained SAGE for both. We used a similar method for support expressions by obtaining posts that were indicative of either emotional or informational support.

Finally, we cross-examined the salient keywords across symptomatic and support expressions to study how concerns were prevalent in either or both of the expression types. We measured log-likelihood ratios (LLRs) along with add-one smoothing, where LLRs close to 0 indicated comparable frequencies, LLRs<1 indicated the greater frequency in symptomatic expressions, and LLRs>1 indicated the greater frequency in support expressions. Together, these linguistic analyses enabled us to obtain psychological concerns and understand how COVID-19 has psychosocially affected individuals, and to contextualize these concerns in the literature on consequences of global crises.


Summary of Results

We summarize our first set of results in Table 1. For all our measures, we found statistical significance (as per t tests) as well as significant effect sizes (Cohen d>0.4 for all measures [71]) in social media expressions in the treatment data as compared to that in the control data. Assuming that most other confounders were minimized due to the geo-temporal similarity of the data sets, our findings indicated that the COVID-19 outbreak led to an increase in people’s symptomatic and support expressions of mental health. We elaborate on the results in the following sections.

Table 1. Comparing social media expressions in the treatment (2020) and control (2019) data sets.
ExpressionTreatment (2020), mean (SD)Control (2019), mean (SD)Δ (%)Cohen dt test (df)P value
Symptomatic mental health expressions

Anxiety1.65 (0.20)1.35 (0.08)21.321.9612.60 (151)<.001

Depression9.00 (0.60)8.17 (0.35)10.181.7110.72 (151)<.001

Stress19.31 (0.77)18.61 (0.43)3.760.813.65 (151).009

Suicidal ideation3.14 (0.31)2.62 (0.13)19.732.1413.54 (151)<.001
Support expressions

Emotional support8.56 (0.84)8.17 (0.50)4.770.462.87 (151).004

Informational support1.75 (0.18)1.67 (0.08)4.780.563.58 (151) <.001

Temporal Variation

Figure 1 shows the changes in symptomatic mental health expressions for the same period in the treatment (2020) and control (2019) years. We found that the treatment and control data sets showed significant differences in people’s symptomatic expressions (Table 1), among which anxiety showed the most significant increase (21.32%), followed by suicidal ideation (19.73%), depression (10.18%), and stress (3.76%). Figure 2 shows the evolution of support expressions change in the treatment and control data sets. The differences were significant (Table 1), and we found that ES increased by 4.77% and IS increased by 4.78%.

Figure 1. Comparison of symptomatic mental health expressions on social media posts in the same period (March 24 to May 24) in 2019 and 2020 (COVID-19 outbreak year). Idn.: ideation.
View this figure
Figure 2. Comparison of support expressions on social media posts in the same period (March 24 to May 24) in 2019 and 2020 (COVID-19 outbreak year).
View this figure

In both the plots of Figures 1 and 2, we found a general trend of negative slope (average slope=–0.03) within the treatment year, which was closer to zero slope (average slope=3.19*10−4) in the control data set. This may suggest that within the treatment year, people’s mental health expressions gradually leveled out over time, despite the growing rate of COVID-19 active cases. The plots indicated that psychological expressions almost converge at the tails. This could likely be due to people’s habituation with the situation and surroundings with the passage of time [70,72], as has been observed for other crisis events [48,73]; however, this needs to be explored further. Within the control data set, we observed a sudden peak on April 28, 2019, which could be attributed to a shooting incident at a synagogue in San Diego [74]. The observations reflected that the COVID-19 pandemic has increased people’s mental health expressions on social media, aligning with other contemporary literature and media reports [8,53].

Linguistic Expressions

Symptomatic Mental Health Expressions

Table 2 summarizes the language differences as per SAGE for posts expressing high mental health expressions in the treatment and control periods—keywords with positive SAGE saliently occurred in the treatment data, whereas those with negative SAGE saliently occurred in the control data. A majority of the keywords that occurred in the treatment period were contextually related to the COVID-19 pandemic, such as covid19, coronavirus, social distancing, and stayathome isolation. These keywords were used in posts expressing mental health concerns either explicitly (eg, “Social distancing is both sad and anxiety-inducing at the same moment”) or implicitly (eg, “In order to get my family treated, I will do more than beg, and I will donate 25K for research to develop COVID19 vaccine”). We also found that the treatment period used keywords referring to key personnel such as dr fauci (referring to Anthony Fauci, one of the leads in the incumbent White House Coronavirus Task Force in the United States and Director of the National Institute of Allergy and Infectious Diseases since 1984 [75]) and political figures like Nury Martinez and Donald Trump. Further, we found keywords such as essential workers, doctor jobs, and risking lives, which describe high-risk worker situations (eg, “I am not complaining about going to work, rather, I am concerned about risking my health for work”), and certain treatment suggestions that evolved during this period [76], such as garlic, malaria, and hydroxychloroquine (eg, “I do worry tho! He is focused on job numbers, approval? ratings and repeating mistruths. His spouting of 2 drugs, one for malaria & the other a Z-pack. A Senior couple tried these untested drugs; wife is in ICU & husband also hospitalized! This is irresponsible & dangerous!”).

Table 2. Top salient n-grams (n=1,2,3) for symptomatic mental health concerns in the treatment and control data sets (SAGE analysis [70]).
Salient in treatment (2020)Salient in control (2019)
KeywordSAGEa,bKeywordSAGEc
covid1911.17hospitality–2.81
lord marvelous10.87trainee–2.78
coronavirus10.58crimes–2.74
social distancing9.92delay–2.55
nury martinez9.66traffic–2.55
working councilwoman9.66accident–2.39
bored daily8.69finance accounting–2.26
stayathome isolation8.69half finance–2.22
quarantinelife8.62auburn–2.21
quarantine got7.87half technology–2.19
securityguard7.63pete–2.19
essential workers7.62parttime–2.18
dr fauci7.56robert half–2.12
went tired7.48tickets–2.08
coronaviruspandemic7.44marvel–2.07
flattenthecurve7.44tournament–1.92
doctorjobs7.32muslim–1.90
garlic7.29florida–1.90
hydroxychloroquine7.28boston–1.88
n95 masks7.26cashier–1.87
masks gloves7.26playoffs–1.86
practice social distancing7.21sales representative–1.85
physicianjobs doctorjobs7.13springfield–1.84
quarantine life6.98border–1.84
plz help small6.96barista–1.77
small donation6.96israel–1.77
stay home orders6.92nc click–1.76
self quarantine6.88playoff–1.75
positive covid196.79bracket–1.75
risking lives6.79terrorist–1.65

aSAGE: Sparse Additive Generative Model.

bPositive SAGE scores indicated greater saliency in the treatment (2020) data.

cNegative SAGE scores indicated greater saliency in the control (2019) data.

Support Expressions

Table 3 lists the top keywords as per SAGE for support posts in the treatment and control periods. Keywords with positive SAGE saliently occurred in the treatment data, whereas those with negative SAGE saliently occurred in the control data. We found keywords that explicitly relate to COVID-19 occurred in the treatment period. We also found that the treatment period consisted of posts that seek support related to job and pay, such as losing jobs, need pay, and furloughed (eg, “Many in our community have lost their jobs, are underinsured and are struggling to make ends meet. Providing pantries, hot meals, hotspots and distance learning opportunities is now more critical than ever, please donate”). Our data also revealed the prevalence of contextually related keywords such as masks, ppe, hoarding, stockpile, and sanitizer that are medically recommended prevention and containment measures of COVID-19 infection (eg, “Please contact me if you have any N95 mask or know to obtain some. My sister and a few friends work in the OR and they do not have the supplies to stay safe, they have patients who have #COVID19. TY! #HealthcareHeroes”).

Table 3. Top salient n-grams (n=1,2,3) for support expressions in treatment and control data sets (SAGE analysis [70]).
Salient in treatment (2020)Salient in control (2019)
KeywordSAGEa,bKeywordSAGEc
lord7.93hospitality–2.86
fauci6.70duke–2.51
ventilators6.59shift supervisor–2.24
quarantine6.47tampa–2.21
securityofficer6.11advisor–1.95
n955.53customerservice–1.92
hope staying safe5.36investigation–1.89
ppe5.25manager retail–1.87
wearing masks5.20traffic–1.87
uncertain times5.16muslim–1.86
healthcare workers5.01store manager–1.85
furloughed5.00tickets–1.85
asymptomatic4.95playoffs–1.83
people quarantine4.90cubs–1.82
fighting stigma4.82border–1.81
staysafe4.67cashier–1.79
food bills4.66springfield–1.79
disinfectant4.64delay–1.76
hand sanitizer3.08barista store–1.76
clorox3.03boston–1.76
medical supplies2.97counter–1.75
trying times2.89barista–1.74
risking lives2.87columbia–1.73
stockpile2.86meeting retail–1.73
father passed2.36informational meeting–1.73
hoarding2.31stlouis–1.72
mask2.31marvel–1.70
medical professionals2.27marketing–1.68
losing jobs2.27server–1.67
toilet paper2.05accident–1.64

aSAGE: Sparse Additive Generative Model.

bPositive SAGE scores indicated greater saliency in the treatment (2020) data.

cNegative SAGE scores indicated greater saliency in the control (2019) data.

Linguistic Comparability

Finally, Table 4 shows the results of the lexical comparability analysis, where LLRs demarcate the top keywords used for symptomatic mental health expressions and support expressions within the treatment data set. We found that keywords, such as safety precautions (wear masks), health care and treatment (health care workers, hospitalized, beds, and icu), and life and death (passed away, kill people, human lives, and deaths), comparably overlapped in both kinds of psychological expressions (LLRs~0). These keywords were also used to raise awareness and express solidarity with health care and high-risk workers (eg, “Taking all safety precautions and adhering to the guidelines established by our health care professionals will keep us safe”). Our lexico-psychological analyses revealed that more clinically relevant keywords and symptoms occurred frequently in symptomatic expressions (LLRs>0; eg, sleep schedule and tested positive), whereas socially relevant and stressful circumstances were more prevalent in support expressions (LLRs<0; eg, im single parent, starve, and lost jobs).

Table 4. Distribution of social media keywords across high symptomatic mental health and support expressions within the treatment period using LLRs.
LLRsa>0bLLRs~0cLLRs<0d
KeywordLLRKeywordLLRKeywordLLR
sleep schedule0.75infected–0.01im single parent–1
lonely0.64wear masks–0.01starve–1
anxiety0.62need help–0.01meditate–1
isolation0.56kill people–0.01sorry loss–0.73
stay safe0.56need pay0care people–0.7
bored0.56health care workers0hard times–0.45
tested positive0.52passed away0people sick–0.4
quarantine life0.52seriousness0helping people–0.4
homeschooling0.51human lives0sorry hear–0.39
tired0.5deaths0urged–0.33
doctor0.48domestic violence0new yorkers–0.29
fighting stigma0.46comforting0lost jobs–0.21
depression0.45hospitalized0hope family–0.21
stuck inside0.42beds0selfish–0.21
sane0.41icu0.01desperate–0.21

aLLR: log likelihood ratio.

bKeywords with LLRs>0 distinctly occurred in high symptomatic expressions.

cKeywords with LLRs~0 occurred comparably in both symptomatic and support expressions.

dKeywords with LLRs<0 distinctly occurred in support expressions.


Principal Results

Our study suggests that social media posts during the COVID-19 pandemic contain a significantly higher frequency of symptomatic mental health and support expressions than a comparable data set from the same period in the previous year. The effect sizes and statistical differences observed in our analyses provide evidence that COVID-19 may have led to mental health concerns compared to other normative times. We also found that they topically relate to the ongoing crisis situation and include concerns such as treatment, precautionary measures, loss of jobs, school closings, stockpiling of basic livelihood necessities, feeling lonely, boredom, and tired of the restrictions and constraints put on by the ongoing pandemic. Our findings suggest that although the COVID-19 pandemic has amplified mental health risks and concerns, it may have heightened a sense of belonging and solidarity among individuals—bringing them together, raising collective awareness, and encouraging them to provide support to one another. We also found expressions of solidarity with health care and high-risk workers, suggesting that people have been considerate about these workers and have expressed desire to set up opportunities for donating to those who have lost jobs during the crisis; this also aligns with recent media reports and World Health Organization guidelines of tackling the pandemic [77,78]. Media reports have also indicated how benevolent neighbors have been, tending to their older adult neighbors by delivering their groceries and other basic necessities [79].

However, mental health experts say that, although the crisis is amplifying risk factors for suicide, the COVID-19 outbreak’s effect on individuals’ mental and emotional well-being is complex [80]. Suicide is multifaceted, and although economic loss is a risk factor, so are depression, isolation, and fear of the future. At the same time, the crisis is possibly creating a sense of belonging for individuals at risk for suicide, as stress and anxiety are normalized, and people come together to better support one another during a crisis [81,82]. Our data showed a significant impact of COVID-19 on suicidal ideation, which calls for enhanced importance of population-scale mental health care, such as using approaches like universal screening (ie, Zero Suicide Model) [83]. As Florida [84] noted in a recent article, “The long-term toll on mental health of social isolation, remote work, and economic insecurity could have impacts akin to post-traumatic stress disorder; yet, the new focus on mental health may reduce stigma and increase the availability of support services.” Indeed, the world beyond the crisis may be one in which mental health is more honestly recognized and supported.

Interestingly, we noted that our findings indicate a gradual leveling out of these expressions, both symptomatic and supportive, which may reflect a developing new normal. In February 2020, it seemed unthinkable that the white-collar workforce of many countries would soon be working solely from home; it seemed unthinkable that air travel would plummet by 96% and that all major sporting events would be called off. Indeed, epidemiologists surmise that many if not most of the changes surrounding the rhythms of our daily life are likely to fade over time, just as they did after the 1918 influenza epidemic [84]. In other words, the pandemic could make us revisit and possibly reform many of our lifestyle choices and civic roles, and the persistent discussion of the new normal may help bring order to our current turbulence. Others have argued that perhaps the crisis is a prelude to a new paradigm, as recently noted by the World Economic Forum [84,85]: “Feeling unsettled, destabilized and alone can help us empathize with individuals who have faced systematic exclusions long-ignored by society even before the rise of COVID-19 – thus stimulating urgent action to improve their condition.” We should, therefore, “revel in the discomfort of the current moment to generate a ‘new paradigm,’ not a ‘new normal.’” The leveling out trend in our data gives empirical ground to these conjectures.

Nevertheless, if robust antiviral treatments are developed and rolled out relatively quickly or if a vaccine becomes available soon enough, presumably, the changes will be short-lived, and the new normal may be temporary. However, if the pandemic comes back in larger waves over the next few seasons, as was the case with historical epidemics, the economic, political, and social crises that have arisen as a consequence will lead to deeper ramifications in turn, leading to longer lasting or permanent changes. Future research will need to explore the persistence of the new normal and the emergence of a possible new paradigm as the pandemic evolves, and therein the mental health impacts further along in the crisis. A study like ours on the ongoing pandemic is a step toward leveraging large-scale online data to understand people’s response to the crisis and thinking about means to address the major concerns. Our study bears implications in digital technology driven mental health interventions to provide tailored support to people’s concerns during the crisis; a recent work by Rudd and Beidas [86] pointed out four point guidelines to build innovative and expansive solutions toward improving public mental health. The variety of concerns and help-seeking factors reflected in our study can also help several stakeholders, ranging across mental health facilitators and policy makers, toward early preparedness and interventions for mental health support. Similarly, our methods can offer the potential to build public health surveillance technologies that surface early warning signs of the effects of the various events related to the pandemic and other crises. The potential of social media to assist in the response to the pandemic is clear but also dependent on the accuracy of underlying methods. The reach of social media allows for broad access that transcends national borders or cultural differences. Using this access to meet the increasing need for help seeking, online and social media data is in a prime position to offer people personalized guidance toward accurate information, health care resources, and even basic lifestyle interventions. Underlying this potential is, thus, the ability of social media data to classify the state and needs of each individual and use that information to tailor a customized response. Precedents for such a system are abound as seen in several prior studies [27-31,37,87-89].

Comparison With Prior Work

COVID-19 is not the first pandemic—catastrophic pandemics have been occurring at regular intervals throughout human history, with the 1918 influenza epidemic being the last one before the current pandemic [84,85,90]. The backdrop of the 1918 pandemic was that it happened just before the advent of modern psychiatry as a science and a clinical specialty—a time when psychoanalysis was gaining recognition as an established treatment within the medical community [91,92]. Consequently, psychiatry has had little opportunity to consider such historically important phenomena through its clinical, scientific lens until now. Although outbreaks of the Zika and Ebola viruses, Middle East respiratory syndrome, and severe acute respiratory syndrome managed to draw global attention, stirring up anxiety and uncertainty in societies, scholars have noted that participation of mental health experts in pandemic preparedness has remained negligible [93]. Consequently, our ability to understand mental health responses as well as the mental health burden in pandemic outbreaks have been limited [94]. For instance, a routinely practiced method of infection control, quarantine and social distancing, have received surprisingly little attention in psychiatric literature so far. Baumeister and Leary [95] contended that humans need frequent contacts, and crisis events further stimulate a need for affiliation and intimacy. Therefore, prolonged isolation and separation from families and their community can have profound effects on individuals even if they are not directly affected by the disease [4]. In the current pandemic, the additional layer of extensive social media use and exposure to often sensationalized online news while in physical isolation may add new complexities to implementing emotional epidemiology in managing concerns, fears, and misconceptions [96,97], as these tools have been argued to bear negative effects on psychological well-being [97,98].

By adopting social media as a lens to unpack these previously less understood dimensions of a pandemic’s mental health effects, our study is one step toward closing some of the previously noted gaps. The published literature posits that the distress and anxiety among individuals in this COVID-19 pandemic may increase the incidence of mental disorders [53,54,99]; data thus far from the United States point to a population increase in psychological distress of 10% compared to 2018 data [8], a trend that is in line with our results. These rates may be higher in those regions heavily exposed to COVID-19 or among individuals working during the pandemic, with a recent review reporting over 20% prevalence of anxiety, which is also consistent with our findings [8].

Prior work found that mental health discourse on Twitter ranges across stigmatizing, inspirational, resource, medical, and social dimensions of expressions [100], and our study revealed similar topical diversity in our data set. Further, we detected through social media many of the stresses associated with the pandemic (eg, prolonged isolation, exposure to pandemic-related death, loss of income and career, increased workload, and lack of pertinent and accurate information). These results align with epidemiological findings that COVID-19 has led to elevated mental health symptoms for individuals. Nelson et al [101] surveyed 2000 individuals from the United States, Canada, and Europe, and found elevated symptoms of anxiety and depression compared to historical norms and observed factors similar to the concerns we detected regarding symptomatic expressions and those related to seeking support. They also reinforced the summary data released by the Crisis Text Line (a major crisis helpline in the United States) listing major concerns of crisis support sought during this period [102], with 80% of conversations mentioning “virus,” 34% mentioning “anxiety,” 34% feeling solidarity with friends and family, etc. Along similar lines, there have been numerous reports about the increasing number of mental health crisis helpline calls during this period [103,104], providing further support and external validation that our social media findings reflect many of the same elements of distress expressed offline during this crisis.

Next, our temporal analyses pointed to a steady decline in people’s expressed psychosocial concerns during the 2-month study period (Figures 1 and 2), which conforms with similar findings in Google search queries as stay-at-home orders and other COVID-19–related policy changes were implemented in the United States [105]. We note contemporary social computing research studying various aspects of the social media discourse related to COVID-19 [63,106-108]. By providing complementary evidence to observations by Mackey et al [106] and Stokes et al [107] on expressed (mental health) concerns during the crisis, our study further underscores their findings using a comparable (control) data set, reinforcing and providing empirical credibility to the impression that the COVID-19 pandemic has indeed caused or contributed directly to the mental health concerns that we describe.

Limitations and Future Work

We note some limitations in our study, many of which present excellent directions for future research. We recognize the lack of transparency related to the Twitter Streaming API. Recent research has also questioned the credibility of the “1% Twitter stream” aspect, noting that actual sampling data was smaller than what it ideally should have been [41]. Given these data limitations, we decided against conducting several descriptive and fine-grained analyses (such as comparing regions), and refrained from making claims based on comparing absolute numbers of those impacted by various mental health concerns. For example, we cannot define based on our data whether there were increased or decreased Twitter postings during our COVID-19 study period compared to the same months in 2019. Again, we chose to filter English-only Twitter posts given both algorithmic limitations of our methodologies and lower prevalence of non-English data (particularly in the US context). However, future work can extend our methodologies to conduct analyses in other languages to draw richer insights.

Despite the strengths of Twitter as a data source that provides us unobtrusive access to large-scale, unstructured, and naturalistic data of people’s candid self-disclosure and that it has been a valuable source to study disaster and crisis response [109], we acknowledge that this data inherently has many biases such as self-selection and representation [110]. We can only study those who self-select to express on Twitter. Pew Internet Center surveys reported that social media platforms are underrepresentative of minorities, although Twitter is an exception, which overrepresents minorities such as Blacks, Hispanics, and women [20]. There is already a digital divide in terms of social media use where the population is skewed toward young adults and white-collar workers. Further, technology and social media could be a luxury to marginalized and underprivileged populations, and any sort of technology-driven support and assistance will disproportionately affect different individuals based on technology use [88]. Similarly, a single platform cannot provide a complete picture; different platforms (eg, Facebook, Reddit, Twitter, and instant messaging services) have unique design strengths and weaknesses both in terms of their affordances as well as who uses them. Therefore, as highlighted in a recent article by Chunara and Cook [23], public health surveillance (including that for COVID-19) can account for several factors such as the “population at risk” in epidemiology and demographic disparities in the use and behavioral expressions on social media.

We understand that our study is observational and, as any other observational study, does not measure “true causality.” Watts [111] noted the impossibility to test all explanations and confounders simultaneously. However, by including and comparing against control data, we minimized geo-temporal and seasonal confounds, thereby enabling us to provide stronger evidence and insights than purely correlational analyses regarding the effects of the COVID-19 pandemic on people’s mental health. We also note that support expressions in our study can not only include support-seeking but also support-providing expressions. This has also enabled us to observe how solidarity and sense of belonging proliferated during the COVID-19 crisis. Future work can build separate high precision classifiers for each kind of expression to disentangle the prevalence of seeking and providing expressions during the crisis.

Further, although we did have data beyond May 24, 2020, we decided to exclude those to keep our focus on the effects on social media expressions due to COVID-19 and minimize those that followed the death of George Floyd on May 25, 2020, in the light of the Black Lives Matter protests throughout the United States [112]. We also are aware that, with the continuing nature of the pandemic, our conclusions are restricted to the mental health and support-seeking concerns expressed during a finite study period. Events since the end of the study period underscore the dynamic nature of these events, as different parts of the United States are heavily affected, while others are recovering, and some remain relatively spared. It will be important to extend this work temporally; increase the size of future samples; and, whenever possible, add geospatial specificity to future analyses. The latter will be especially important for potential supportive interventions locally if one has the resources and the ability to assemble recurring, near-real-time local “snapshots” as a basis for community-focused preventive interventions. Further, our analyses can be extended to retrospectively examine the aftereffects of particular global and local events, such as policy changes, related to the pandemic.

Conclusion

Our study, like those of others studying other major events, further reinforces the potential utility of accessing and analyzing social media data in near-real-time to take the temperature of communities. This will require a more focused and robust collection of locally targeted information to build samples that are sufficiently large to produce reliably representative data sets to be useful for public health interventions. Further work is now needed to track mental health–related expressions and those reflecting needs for support throughout the pandemic, which has seen dynamic changes associated with disease spread to areas that had been less affected during the early months of the outbreak. This geo-specific research may further enhance our understanding of the causal connections between COVID-19 spread and waves of expressed distress. Having the ability to present locally pertinent, contemporaneous analyses offers the opportunity for local public health and mental health providers as well as political leaders to develop and deploy targeted support services in a timely fashion.

Acknowledgments

MDC was partly supported by a COVID-19–related Rapid Response Research grant #2027689 from the National Science Foundation.

Conflicts of Interest

JT receives unrelated research support from Otsuka.

  1. Hays JN. Epidemics and Pandemics: Their Impacts on Human History. Santa Barbara, CA: ABC-CLIO; 2005.
  2. Boscarino JA. Community disasters, psychological trauma, and crisis intervention. Int J Emerg Ment Health 2015;17(1):369-371 [FREE Full text] [Medline]
  3. Lu S. An epidemic of fear. American Psychological Assocation. 2015.   URL: https://www.apa.org/monitor/2015/03/fear [accessed 2020-07-25]
  4. Morganstein JC, Fullerton CS, Ursano RJ, Donato D, Holloway HC. 18 - pandemics: health care emergencies. In: Ursano RJ, Fullerton CS, Weisaeth L, Raphael B, editors. Textbook of Disaster Psychiatry. Cambridge, England: Cambridge University Press; 2017:270-284.
  5. COVID-19 coronavirus pandemic. Worldometers.   URL: https://www.worldometers.info/coronavirus/ [accessed 2020-07-25]
  6. Basu T. The coronavirus pandemic is a game changer for mental health care. MIT Technology Review. 2020.   URL: https:/​/www.​technologyreview.com/​2020/​03/​20/​905184/​coronavirus-online-therapy-mental-health-app-teletherapy/​ [accessed 2020-07-25]
  7. Resnick B. A third of Americans report anxiety or depression symptoms during the pandemic. Vox. 2020.   URL: https://www.vox.com/science-and-health/2020/5/29/21274495/pandemic-cdc-mental-health [accessed 2020-07-25]
  8. McGinty EE, Presskreischer R, Han H, Barry CL. Psychological distress and loneliness reported by US adults in 2018 and April 2020. JAMA 2020 Jul 07;324(1):93-94 [FREE Full text] [CrossRef] [Medline]
  9. Holmes EA, O'Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry 2020 Jun;7(6):547-560 [FREE Full text] [CrossRef] [Medline]
  10. World Health Organization. Operational considerations for case management of COVID-19 in health facility and community. Interim guidance. Pediatr Med Rodz 2020 Jul 14;16(1):27-32. [CrossRef]
  11. Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, Centre for the Mathematical Modelling of Infectious Diseases COVID-19 Working Group, et al. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Glob Health 2020 Apr;8(4):e488-e496 [FREE Full text] [CrossRef] [Medline]
  12. Saxena SK, editor. Coronavirus Disease 2019 (COVID-19) Epidemiology, Pathogenesis, Diagnosis, and Therapeutics. Singapore: Springer; 2020.
  13. Miller G. Social distancing prevents infections, but it can have unintended consequences. Science 2020 Mar 17:1. [CrossRef]
  14. Varatharaj A, Thomas N, Ellul MA, Davies NWS, Pollak TA, Tenorio EL, CoroNerve Study Group. Neurological and neuropsychiatric complications of COVID-19 in 153 patients: a UK-wide surveillance study. Lancet Psychiatry 2020 Oct;7(10):875-882 [FREE Full text] [CrossRef] [Medline]
  15. Pfefferbaum B, North CS. Mental health and the Covid-19 pandemic. N Engl J Med 2020 Aug 06;383(6):510-512. [CrossRef] [Medline]
  16. Fiorillo A, Gorwood P. The consequences of the COVID-19 pandemic on mental health and implications for clinical practice. Eur Psychiatry 2020 Apr 01;63(1):e32 [FREE Full text] [CrossRef] [Medline]
  17. Heymann DL, Shindo N, WHO Scientific and Technical Advisory Group for Infectious Hazards. COVID-19: what is next for public health? Lancet 2020 Feb 22;395(10224):542-545 [FREE Full text] [CrossRef] [Medline]
  18. Weinstock CP. Ripple effects of COVID-19 strain mental health systems. US News. 2020.   URL: https:/​/www.​usnews.com/​news/​healthiest-communities/​articles/​2020-06-04/​coronavirus-ripple-effects-strain-mental-health-systems [accessed 2020-07-25]
  19. Chancellor S, De Choudhury M. Methods in predictive techniques for mental health status on social media: a critical review. NPJ Digit Med 2020;3:43. [CrossRef] [Medline]
  20. Social media fact sheet. Pew Internet Center. 2019.   URL: https://www.pewresearch.org/internet/fact-sheet/social-media/ [accessed 2020-07-25]
  21. Andalibi N, Haimson OL, De Choudhury M, Forte A. Understanding social media disclosures of sexual abuse through the lenses of support seeking and anonymity. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2016 Presented at: CHI '16; May 2016; San Jose, CA. [CrossRef]
  22. Qiu L, Lin H, Leung AK, Tov W. Putting their best foot forward: emotional disclosure on Facebook. Cyberpsychol Behav Soc Netw 2012 Oct;15(10):569-572. [CrossRef] [Medline]
  23. Chunara R, Cook SH. Using digital data to protect and promote the most vulnerable in the fight against COVID-19. Front Public Health 2020;8:296. [CrossRef] [Medline]
  24. Guntuku SC, Schwartz HA, Kashyap A, Gaulton JS, Stokes DC, Asch DA, et al. Variability in language used on social media prior to hospital visits. Sci Rep 2020 Mar 12;10(1):4346. [CrossRef] [Medline]
  25. Saha K, Sugar B, Torous J, Abrahao B, Kıcıman E, De Choudhury M. A social media study on the effects of psychiatric medication use. Proc Int AAAI Conf Weblogs Soc Media 2019 Jun 07;13:440-451 [FREE Full text] [Medline]
  26. Saha K, Kim SC, Reddy MD, Carter AJ, Sharma E, Haimson OL, et al. The language of LGBTQ+ minority stress experiences on social media. Proc ACM Hum Comput Interact 2019 Nov;3(CSCW):1-22. [CrossRef] [Medline]
  27. Ernala SK, Labetoulle T, Bane F, Birnbaum ML, Rizvi AF, Kane JM, et al. Characterizing audience engagement and assessing its impact on social media disclosures of mental illnesses. 2018 Presented at: Twelfth International AAAI Conference on Web and Social Media; June 2018; Palo Alto, CA.
  28. De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting depression via social media. 2013 Presented at: The 7th International AAAI Conference on Weblogs and Social Media; July 2013; Cambridge, MA.
  29. Saha K, Weber I, Birnbaum ML, De Choudhury M. Characterizing awareness of schizophrenia among Facebook users by leveraging Facebook advertisement estimates. J Med Internet Res 2017 May 08;19(5):e156 [FREE Full text] [CrossRef] [Medline]
  30. Dredze M. How social media will change public health. IEEE Intell Syst 2012 Jul;27(4):81-84 [FREE Full text] [CrossRef]
  31. Paul MJ, Dredze M. You are what you tweet: analyzing Twitter for public health. 2011 Presented at: Fifth International AAAI Conference on Weblogs and Social Media; July 2011; Barcelona, Spain.
  32. Sadilek A, Kautz H, Silenzio V. Predicting disease transmission from geo-tagged micro-blog data. 2012 Presented at: Twenty-Sixth AAAI Conference on Artificial Intelligence; July 2012; Toronto, ON.
  33. Gore RJ, Diallo S, Padilla J. You are what you tweet: connecting the geographic variation in America's obesity rate to Twitter content. PLoS One 2015;10(9):e0133505 [FREE Full text] [CrossRef] [Medline]
  34. Saha K, Chan L, De Barbaro K, Abowd GD, De Choudhury M. Inferring mood instability on social media by leveraging ecological momentary assessments. Proc ACM Interact Mob Wearable Ubiquitous Technol 2017 Sep 11;1(3):1-27. [CrossRef]
  35. Coppersmith GA, Harman CT, Dredze MH. Measuring post traumatic stress disorder in Twitter. 2014 Presented at: Eighth International AAAI Conference on Weblogs and Social Media; 2014; Ann Arbor, MI.
  36. Kiciman E, Counts S, Gasser M. Using longitudinal social media analysis to understand the effects of early college alcohol use. 2018 Presented at: 12th International AAAI Conference on Web and Social Media; June 2018; Palo Alto, CA.
  37. Liu J, Weitzman E, Chunara R. Assessing behavioral stages from social media data. CSCW Conf Comput Support Coop Work 2017;2017:1320-1333 [FREE Full text] [CrossRef] [Medline]
  38. Sharma E, De Choudhury M. Mental health support and its relationship to linguistic accommodation in online communities. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 2018 Presented at: CHI '18; April 2018; Montreal, QC. [CrossRef]
  39. Saha K, Sharma A. Causal factors of effective psychosocial outcomes in online mental health communities. Proc Int AAAI Conference Web Soc Media 2020;14(1):590-601.
  40. Filter realtime tweets. Twitter Developer.   URL: https:/​/developer.​twitter.com/​en/​docs/​twitter-api/​v1/​tweets/​filter-realtime/​guides/​basic-stream-parameters [accessed 2020-08-17]
  41. Pfeffer J, Mayer K, Morstatter F. Tampering with Twitter’s sample API. EPJ Data Sci 2018 Dec 19;7(1):1. [CrossRef]
  42. Morstatter F, Pfeffer J, Liu H. When is it biased?: assessing the representativeness of twitter's streaming API. In: Proceedings of the 23rd International Conference on World Wide Web. 2014 Presented at: WWW '14 Companion; April 2014; Seoul, Korea. [CrossRef]
  43. Boyd D, Crawford K. Critical questions for big data. Inf Commun Soc 2012 Jun;15(5):662-679. [CrossRef]
  44. Burton SH, Tanner KW, Giraud-Carrier CG, West JH, Barnes MD. "Right time, right place" health communication on Twitter: value and accuracy of location information. J Med Internet Res 2012 Nov 15;14(6):e156 [FREE Full text] [CrossRef] [Medline]
  45. Tsuya A, Sugawara Y, Tanaka A, Narimatsu H. Do cancer patients tweet? Examining the twitter use of cancer patients in Japan. J Med Internet Res 2014 May 27;16(5):e137 [FREE Full text] [CrossRef] [Medline]
  46. Centers for Disease Control and Prevention.   URL: https://www.cdc.gov/ [accessed 2020-05-09]
  47. Timeline of the COVID-19 pandemic in the United States. Wikipedia.   URL: https://en.wikipedia.org/wiki/Timeline_of_the_COVID-19_pandemic_in_the_United_States [accessed 2020-07-25]
  48. Saha K, De Choudhury M. Modeling stress with social media around incidents of gun violence on college campuses. Proc ACM Hum-Comput Interact 2017 Dec 06;1(CSCW):1-27. [CrossRef]
  49. Iqbal M. Twitter revenue and usage statistics (2020). Business of Apps. 2020.   URL: https://www.businessofapps.com/data/twitter-statistics/#2 [accessed 2020-08-17]
  50. De Choudhury M, Kiciman E, Dredze M, Coppersmith G, Kumar M. Discovering shifts to suicidal ideation from mental health content in social media. Proc SIGCHI Conf Hum Factor Comput Syst 2016 May;2016:2098-2110 [FREE Full text] [CrossRef] [Medline]
  51. Guntuku SC, Ramsay JR, Merchant RM, Ungar LH. Language of ADHD in adults on social media. J Atten Disord 2019 Oct;23(12):1475-1485. [CrossRef] [Medline]
  52. Saha K, Weber I, De Choudhury M. A social media based examination of the effects of counseling recommendations after student deaths on college campuses. Proc Int AAAI Conf Weblogs Soc Media 2018 Jun;2018:320-329 [FREE Full text] [Medline]
  53. Bedford J, Enria D, Giesecke J, Heymann DL, Ihekweazu C, Kobinger G, WHO Strategic and Technical Advisory Group for Infectious Hazards. COVID-19: towards controlling of a pandemic. Lancet 2020 Mar 28;395(10229):1015-1018 [FREE Full text] [CrossRef] [Medline]
  54. Taha S, Matheson K, Cronin T, Anisman H. Intolerance of uncertainty, appraisals, coping, and anxiety: the case of the 2009 H1N1 pandemic. Br J Health Psychol 2014 Sep;19(3):592-605. [CrossRef] [Medline]
  55. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. Washington, DC: American Psychiatric Association; 2013.
  56. Bagroy S, Kumaraguru P, De Choudhury M. A social media based index of mental well-being in college campuses. Proc SIGCHI Conf Hum Factor Comput Syst 2017 May;2017:1634-1646 [FREE Full text] [CrossRef] [Medline]
  57. Kummervold PE, Gammon D, Bergvik S, Johnsen JK, Hasvold T, Rosenvinge JH. Social support in a wired world: use of online mental health forums in Norway. Nord J Psychiatry 2002;56(1):59-65. [CrossRef] [Medline]
  58. Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychol Bull 1985 Sep;98(2):310-357. [Medline]
  59. De Choudhury M, De S. Mental health discourse on Reddit: self-disclosure, social support, and anonymity. 2014 Presented at: Eighth International AAAI Conference on Weblogs and Social Media; May 2014; Ann Arbor, MI.
  60. Rains SA, Young V. A meta-analysis of research on formal computer-mediated support groups: examining group characteristics and health outcomes. Hum Commun Res 2009;35(3):309-336. [CrossRef]
  61. De Choudhury M, Kıcıman E. The language of social support in social media and its effect on suicidal ideation risk. Proc Int AAAI Conf Weblogs Soc Media 2017 May;2017:32-41 [FREE Full text] [Medline]
  62. Oh HJ, Lauckner C, Boehmer J, Fewins-Bliss R, Li K. Facebooking for health: an examination into the solicitation and effects of health-related social support on social networking sites. Comput Hum Behav 2013 Sep;29(5):2072-2080. [CrossRef]
  63. Luo C, Li Y, Chen A, Tang Y. What triggers online help-seeking retransmission during the COVID-19 period? Empirical evidence from Chinese social media. PLoS One 2020;15(11):e0241465 [FREE Full text] [CrossRef] [Medline]
  64. Cutrona CE, Suhr JA. Controllability of stressful events and satisfaction with spouse support behaviors. Commun Res 1992;19(2):154-174. [CrossRef]
  65. Nambisan P. Information seeking and social support in online health communities: impact on patients' perceived empathy. J Am Med Inform Assoc 2011 May 01;18(3):298-304. [CrossRef] [Medline]
  66. Andalibi N, Haimson OL, Choudhury MD, Forte A. Social support, reciprocity, and anonymity in responses to sexual abuse disclosures on social media. ACM Trans Comput-Hum Interact 2018 Oct 17;25(5):1-35. [CrossRef]
  67. Wang YC, Kraut R, Levine JM. To stay or leave?: the relationship of emotional and informational support to commitment in online health support groups. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. 2012 Presented at: CSCW '12; February 2012; Seattle, WA. [CrossRef]
  68. Attai DJ, Cowher MS, Al-Hamadani M, Schoger JM, Staley AC, Landercasper J. Twitter social media is an effective tool for breast cancer patient education and support: patient-reported outcomes by survey. J Med Internet Res 2015 Jul 30;17(7):e188 [FREE Full text] [CrossRef] [Medline]
  69. Golder SA, Macy MW. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science 2011 Sep 30;333(6051):1878-1881 [FREE Full text] [CrossRef] [Medline]
  70. Eisenstein J, Ahmed A, Xing EP. Sparse additive generative models of text. 2011 Presented at: 28th International Conference on International Conference on Machine Learning; June 2011; Bellevue, WA.
  71. Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York, NY: Taylor & Francis Group; 2013.
  72. Rankin CH, Abrams T, Barry RJ, Bhatnagar S, Clayton DF, Colombo J, et al. Habituation revisited: an updated and revised description of the behavioral characteristics of habituation. Neurobiol Learn Mem 2009 Sep;92(2):135-138 [FREE Full text] [CrossRef] [Medline]
  73. De Choudhury M, Monroy-Hernández A, Mark G. "Narco" emotions: affect and desensitization in social media during the mexican drug war. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2014 Presented at: CHI '14; April 2014; Toronto, ON. [CrossRef]
  74. Medina J, Mele C, Murphy H. One dead in synagogue shooting near San Diego; officials call it hate crime. The New York Times. 2019 Apr.   URL: https://www.nytimes.com/2019/04/27/us/poway-synagogue-shooting.html [accessed 2020-07-25]
  75. Anthony S. Fauci, M.D. National Institute of Allergy and Infectious Diseases.   URL: https://www.niaid.nih.gov/about/anthony-s-fauci-md-bio [accessed 2020-07-25]
  76. Colson P, Rolain J, Lagier J, Brouqui P, Raoult D. Chloroquine and hydroxychloroquine as available weapons to fight COVID-19. Int J Antimicrob Agents 2020 Apr;55(4):105932 [FREE Full text] [CrossRef] [Medline]
  77. Mental health and psychosocial considerations during the COVID-19 outbreak. World Health Organization. 2020.   URL: https://apps.who.int/iris/bitstream/handle/10665/331490/WHO-2019-nCoV-MentalHealth-2020.1-eng.pdf [accessed 2020-07-20]
  78. Broom D. A pandemic of solidarity? This is how people are supporting one another as coronavirus spreads. World Economic Forum. 2020.   URL: https://www.weforum.org/agenda/2020/03/covid-19-coronavirus-solidarity-help-pandemic/ [accessed 2020-08-18]
  79. Free C. People across the country are delivering groceries free. It’s ‘solidarity, not charity.’. The Washington Post. 2020.   URL: https:/​/www.​washingtonpost.com/​lifestyle/​2020/​04/​27/​people-across-country-are-delivering-groceries-free-its-solidarity-not-charity/​ [accessed 2020-07-25]
  80. Reger MA, Stanley IH, Joiner TE. Suicide mortality and coronavirus disease 2019-a perfect storm? JAMA Psychiatry 2020 Apr 10:1. [CrossRef] [Medline]
  81. Cohn MA, Mehl MR, Pennebaker JW. Linguistic markers of psychological change surrounding September 11, 2001. Psychol Sci 2004 Oct;15(10):687-693. [CrossRef] [Medline]
  82. González-Sanguino C, Ausín B, Castellanos MÁ, Saiz J, López-Gómez A, Ugidos C, et al. Mental health consequences during the initial stage of the 2020 coronavirus pandemic (COVID-19) in Spain. Brain Behav Immun 2020 Jul;87:172-176 [FREE Full text] [CrossRef] [Medline]
  83. Labouliere CD, Vasan P, Kramer A, Brown G, Green K, Rahman M, et al. "Zero Suicide" - a model for reducing suicide in United States behavioral healthcare. Suicidologi 2018;23(1):22-30 [FREE Full text] [Medline]
  84. Florida R. The lasting normal for the post-pandemic city. Bloomberg. 2020.   URL: https://www.bloomberg.com/news/features/2020-06-25/the-new-normal-after-the-coronavirus-pandemic [accessed 2020-07-25]
  85. Asonye C. There's nothing new about the 'new normal'. Here's why. World Economic Forum. 2020.   URL: https://www.weforum.org/agenda/2020/06/theres-nothing-new-about-this-new-normal-heres-why/ [accessed 2020-07-25]
  86. Rudd BN, Beidas RS. Digital mental health: the answer to the global mental health crisis? JMIR Ment Health 2020 Jun 02;7(6):e18472 [FREE Full text] [CrossRef] [Medline]
  87. Yoo DW, Birnbaum ML, Van Meter AR, Ali AF, Arenare E, Abowd GD, et al. Designing a clinician-facing tool for using insights from patients' social media activity: iterative co-design approach. JMIR Ment Health 2020 Aug 12;7(8):e16969 [FREE Full text] [CrossRef] [Medline]
  88. Pendse SR, Lalani FM, De Choudhury M, Sharma A, Kumar N. "Like Shock Absorbers": understanding the human infrastructures of technology-mediated mental health support. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020 Presented at: CHI '20; April 2020; Honolulu, HI. [CrossRef]
  89. Saha K, Bayraktaroglu AE, Campbell AT, Chawla NV, De Choudhury M, D'Mello SK, et al. Social media as a passive sensor in longitudinal studies of human behavior and wellbeing. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 2019 Presented at: CHI '19; May 2019; Glasgow, Scotland. [CrossRef]
  90. Cunha BA. Influenza: historical aspects of epidemics and pandemics. Infect Dis Clin North Am 2004 Mar;18(1):141-155. [CrossRef] [Medline]
  91. Aassve A, Alfani G, Gandolfi F, Le Moglie M. Epidemics and trust: the case of the spanish flu. Working Papers 661, Innocenzo Gasparini Institute for Economic Research 2020:1.
  92. Phillips H. The recent wave of 'Spanish' flu historiography. Social Hist Med 2014 Sep 01;27(4):789-808. [CrossRef]
  93. Huremović D. Psychiatry of Pandemics: a Mental Health Response to Infection Outbreak. Cham, Switzerland: Springer; 2019.
  94. Ornell F, Schuch JB, Sordi AO, Kessler FHP. "Pandemic fear" and COVID-19: mental health burden and strategies. Braz J Psychiatry 2020;42(3):232-235 [FREE Full text] [CrossRef] [Medline]
  95. Baumeister RF, Leary MR. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychol Bull 1995 May;117(3):497-529. [Medline]
  96. Gao J, Zheng P, Jia Y, Chen H, Mao Y, Chen S, et al. Mental health problems and social media exposure during COVID-19 outbreak. PLoS One 2020;15(4):e0231924 [FREE Full text] [CrossRef] [Medline]
  97. Best P, Manktelow R, Taylor B. Online communication, social media and adolescent wellbeing: a systematic narrative review. Child Youth Services Rev 2014 Jun;41:27-36. [CrossRef]
  98. Berryman C, Ferguson CJ, Negy C. Social media use and mental health among young adults. Psychiatr Q 2018 Jun;89(2):307-314. [CrossRef] [Medline]
  99. Pappa S, Ntella V, Giannakas T, Giannakoulis VG, Papoutsi E, Katsaounou P. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: a systematic review and meta-analysis. Brain Behav Immun 2020 Aug;88:901-907 [FREE Full text] [CrossRef] [Medline]
  100. Saha K, Torous J, Ernala SK, Rizuto C, Stafford A, De Choudhury M. A computational study of mental health awareness campaigns on social media. Transl Behav Med 2019 Nov 25;9(6):1197-1207 [FREE Full text] [CrossRef] [Medline]
  101. Nelson BW, Pettitt A, Flannery JE, Allen NB. Rapid assessment of psychological and epidemiological correlates of COVID-19 concern, financial strain, and health-related behavior change in a large online sample. PLoS One 2020;15(11):e0241990 [FREE Full text] [CrossRef] [Medline]
  102. Lublin N. Notes on coronavirus: how is America feeling? Part 7. Crisis Text Line. 2020.   URL: https://www.crisistextline.org/mental-health/notes-on-coronavirus-how-is-america-feeling-part-7/ [accessed 2020-07-25]
  103. Miltimore J. ‘A Year's Worth of Suicide Attempts in Four Weeks’: the unintended consequences of COVID-19 lockdowns. Foundation for Economic Education. 2020.   URL: https:/​/fee.​org/​articles/​a-years-worth-of-suicide-attempts-in-four-weeks-the-unintended-consequences-of-covid-19-lockdowns/​ [accessed 2020-07-25]
  104. Seaman J. Suicides in Colorado dropped 40% during first 2 months of coronavirus pandemic but calls to crisis line spiked. The Denver Post. 2020.   URL: https:/​/www.​denverpost.com/​2020/​05/​23/​colorado-suicides-dropped-coronavirus-pandemic-calls-to-crisis-line-spiked/​ [accessed 2020-07-25]
  105. Jacobson NC, Lekkas D, Price G, Heinz MV, Song M, O'Malley AJ, et al. Flattening the mental health curve: COVID-19 stay-at-home orders are associated with alterations in mental health search behavior in the United States. JMIR Preprints. Preprint posted online on May 26, 2020. [CrossRef]
  106. Mackey T, Purushothaman V, Li J, Shah N, Nali M, Bardier C, et al. Machine learning to detect self-reporting of symptoms, testing access, and recovery associated with COVID-19 on Twitter: retrospective big data infoveillance study. JMIR Public Health Surveill 2020 Jun 08;6(2):e19509 [FREE Full text] [CrossRef] [Medline]
  107. Stokes DC, Andy A, Guntuku SC, Ungar LH, Merchant RM. Public priorities and concerns regarding COVID-19 in an online discussion forum: longitudinal topic modeling. J Gen Intern Med 2020 Jul;35(7):2244-2247 [FREE Full text] [CrossRef] [Medline]
  108. Guntuku SC, Sherman G, Stokes DC, Agarwal AK, Seltzer E, Merchant RM, et al. Tracking mental health and symptom mentions on Twitter during COVID-19. J Gen Intern Med 2020 Sep;35(9):2798-2800 [FREE Full text] [CrossRef] [Medline]
  109. Imran M, Castillo C, Diaz F, Vieweg S. Processing social media messages in mass emergency: survey summary. In: Companion Proceedings of the Web Conference 2018. 2018 Presented at: WWW '18; April 2018; Lyon, France. [CrossRef]
  110. Olteanu A, Castillo C, Diaz F, Kıcıman E. Social data: biases, methodological pitfalls, and ethical boundaries. Front Big Data 2019 Jul 11;2:13. [CrossRef]
  111. Watts DJ. Common sense and sociological explanations. AJS 2014 Sep;120(2):313-351. [CrossRef] [Medline]
  112. George Floyd: what happened in the final moments of his life. BBC News. 2020.   URL: https://www.bbc.com/news/world-us-canada-52861726 [accessed 2020-07-25]


API: application programming interface
ES: emotional support
IS: informational support
LLR: log-likelihood ratio
SAGE: Sparse Additive Generative Model
SVM: support vector machine


Edited by G Eysenbach; submitted 26.07.20; peer-reviewed by B Rudd, B Hoffman; comments to author 11.08.20; revised version received 19.08.20; accepted 26.10.20; published 24.11.20

Copyright

©Koustuv Saha, John Torous, Eric D Caine, Munmun De Choudhury. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.11.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.