This paper is in the following e-collection/theme issue:

JMIR Theme Issue: COVID-19 Special Issue (2452) Natural Language Processing (692) Medicine 2.0: Social Media, Open, Participatory, Collaborative Medicine (1641) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1517) Outbreak and Pandemic Preparedness and Management (1727) Machine Learning (1465) Artificial Intelligence (1371) Infodemiology and Infoveillance (1349) JMH Theme Issue: COVID-19 and Mental Health: Impact and Interventions (383) Infoveillance, Infodemiology, Digital Disease Surveillance, Infodemic Management (1041) Infectious Diseases (non-STD/STI) (1575) Hygiene and Infection Prevention (317) Anxiety and Stress Disorders (1053)

Published on in Vol 23, No 4 (2021): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/27341, first published .
Emotions of COVID-19: Content Analysis of Self-Reported Information Using Artificial Intelligence

Emotions of COVID-19: Content Analysis of Self-Reported Information Using Artificial Intelligence

Emotions of COVID-19: Content Analysis of Self-Reported Information Using Artificial Intelligence

Authors of this article:

Achini Adikari1 Author Orcid Image ;   Rashmika Nawaratne1 Author Orcid Image ;   Daswin De Silva1 Author Orcid Image ;   Sajani Ranasinghe1 Author Orcid Image ;   Oshadi Alahakoon2 Author Orcid Image ;   Damminda Alahakoon1 Author Orcid Image
Article Authors Cited by (32) Tweetations (10) Metrics

    Original Paper

    1Research Centre for Data Analytics and Cognition, La Trobe University, Melbourne, Australia

    2College of Engineering and Science, Victoria University, Melbourne, Australia

    *all authors contributed equally

    Corresponding Author:

    Achini Adikari, BSc

    Research Centre for Data Analytics and Cognition

    La Trobe University

    Kingsbury Drive

    Melbourne

    Australia

    Phone: 61 394793109

    Email: A.Adikari@latrobe.edu.au


    Background: The COVID-19 pandemic has disrupted human societies around the world. This public health emergency was followed by a significant loss of human life; the ensuing social restrictions led to loss of employment, lack of interactions, and burgeoning psychological distress. As physical distancing regulations were introduced to manage outbreaks, individuals, groups, and communities engaged extensively on social media to express their thoughts and emotions. This internet-mediated communication of self-reported information encapsulates the emotional health and mental well-being of all individuals impacted by the pandemic.

    Objective: This research aims to investigate the human emotions related to the COVID-19 pandemic expressed on social media over time, using an artificial intelligence (AI) framework.

    Methods: Our study explores emotion classifications, intensities, transitions, and profiles, as well as alignment to key themes and topics, across the four stages of the pandemic: declaration of a global health crisis (ie, prepandemic), the first lockdown, easing of restrictions, and the second lockdown. This study employs an AI framework comprised of natural language processing, word embeddings, Markov models, and the growing self-organizing map algorithm, which are collectively used to investigate social media conversations. The investigation was carried out using 73,000 public Twitter conversations posted by users in Australia from January to September 2020.

    Results: The outcomes of this study enabled us to analyze and visualize different emotions and related concerns that were expressed and reflected on social media during the COVID-19 pandemic, which could be used to gain insights into citizens’ mental health. First, the topic analysis showed the diverse as well as common concerns people had expressed during the four stages of the pandemic. It was noted that personal-level concerns expressed on social media had escalated to broader concerns over time. Second, the emotion intensity and emotion state transitions showed that fear and sadness emotions were more prominently expressed at first; however, emotions transitioned into anger and disgust over time. Negative emotions, except for sadness, were significantly higher (P<.05) in the second lockdown, showing increased frustration. Temporal emotion analysis was conducted by modeling the emotion state changes across the four stages of the pandemic, which demonstrated how different emotions emerged and shifted over time. Third, the concerns expressed by social media users were categorized into profiles, where differences could be seen between the first and second lockdown profiles.

    Conclusions: This study showed that the diverse emotions and concerns that were expressed and recorded on social media during the COVID-19 pandemic reflected the mental health of the general public. While this study established the use of social media to discover informed insights during a time when physical communication was impossible, the outcomes could also contribute toward postpandemic recovery and understanding psychological impact via emotion changes, and they could potentially inform health care decision making. This study exploited AI and social media to enhance our understanding of human behaviors in global emergencies, which could lead to improved planning and policy making for future crises.

    J Med Internet Res 2021;23(4):e27341

    doi:10.2196/27341

    Keywords

    COVID-19; pandemic; lockdown; human emotions; affective computing; human-centric artificial intelligence; artificial intelligence; AI; machine learning; natural language processing; language modeling; infodemiology; infoveillance 


    Overview

    The COVID-19 pandemic continues to devastate the world, with more than 80 million infections and more than 1.7 million deaths [1]. It is an unprecedented public health emergency that has prompted most governments to enforce hard borders, strict social distancing, and rigorous quarantine restrictions. The impact and aftermath of these public health emergency measures have resulted in a severe psychological burden for all individuals. The economic and social fallout has affected individuals and communities alike, resulting in mental health disorders and emotional distress that adds to already overwhelmed health care systems and services worldwide [2-4]. Although researchers and authorities are readily investing in vaccinations, social distancing regulations, and health care facilities to eliminate the virus, long-term mental health impacts will also require undivided attention and action to minimize the detrimental effects. Given this context, developing an understanding of the psychological and emotional burden will aid and accelerate postpandemic recovery and enable policy making for future emergencies of this scale.

    Social restrictions and physical distancing measures during this pandemic have led to increased use of social media as a medium of communication by individuals and communities [5]. Expressions on social media during such crises are critical and representative of public opinion, as it has become the primary medium of communication for the exchange of information, experiences, and emotions with others who are facing similar challenges [6-8]. Identification of such opinions in the form of topics enables identification of people’s past and ongoing concerns throughout the pandemic [9]. Recent studies have shown a rapid growth of social media content focused on informational and emotional sharing as means of emotion regulation, avoiding mental health issues, and adjustments to the quality of life in lockdown [10,11]. This phenomenon has been described as the social sharing of emotion, which postulates that individuals who experience emotions are often eager to share and talk about their emotions [12]. Crises similar to the current pandemic may lead to the amplification of expression of emotions within a community or group of people [13-15]. The extent and frequency of sharing depend on the intensity of the emotional episode, and such social sharing occurs as a means of regulating one’s emotions. Thereby, the social sharing of emotions during a crisis creates a spreading and escalation of emotions within the group or community impacted by the event, which, in turn, reflect the mental health status of the community.

    Given the volume and variety of content being shared on social media, online behaviors have been investigated as a proxy for offline human behaviors, with several studies reporting conclusive results and outcomes [16-18]. A massive number of social media conversations represents a cross-section of society that encompasses people’s opinions across different demographic dimensions. These emotions and opinions create a pool of untapped self-reported information. Therefore, it is relevant and useful to study the emotions and concerns voiced over such internet-mediated communication to better understand the mental health and emotional well-being during a time of crisis. Several recent studies have investigated the use of social media during the COVID-19 pandemic to discover topics, emotions, and sentiments from social media conversations [19-21] as well as different information sharing behaviors [22]. While the use of social media to understand public opinion is well established, studies suggest that more focus should be given to showcase how social media can be used to improve health knowledge [23].

    On this premise, this study aims to investigate concerns and emotions expressed over social media to infer insights on people’s mental health as the pandemic progresses. It has been shown that emotions and concerns expressed on social media represent and relate to people’s underlying mental health [24-27]. This establishes social media as a lens through which to comprehend people’s mental health during a constrained situation such as the current pandemic, where conducting clinical trials is challenging.

    We selected Twitter as the test bed social media platform for our experiments, given that it provides fast-paced, frequent, current affairs–focused end-user engagement, in comparison to other social media platforms [28]. We extracted tweets related to the pandemic that were posted from January to September 2020 and specifically focused on an Australian context based on the following reasons. First, Australia was one of the countries where distinct phases of the lockdown and their impact were clearly visible. This enables the temporal analysis of emotions and concerns in each phase to gauge insights into how citizens react and change their emotion-related behaviors as the pandemic progresses. The four phases are (1) prepandemic—when the outbreaks were yet to be declared a global pandemic and no positive cases have been reported in Australia, (2) the first lockdown—social restrictions following the first wave of positive cases locally, (3) easing restrictions—relaxation of social restrictions when the case numbers were brought under control, and (4) the second lockdown—following the emergence of new cases. Next, as Australia is one of the few countries with a sizeable population to successfully suppress both first and second waves of the pandemic [29,30], it was pertinent to study the emotions related to COVID-19 as experienced by the Australian public. Despite the successful management, a recent study mentions that mental health problems were widespread among Australians during the lockdown periods, where one-quarter of the participants in the study showcased mild to moderate symptoms of anxiety or depression [31,32]. The use of social media among the Australian public is well established for being representative of a broad cross-section of society [33-35], especially during emergencies as a powerful communication tool [36,37]. This allows us to investigate the emotions expressed via virtual platforms in order to determine similar behaviors related to Australian citizens’ mental health.

    In this study, we focus on the application of artificial intelligence (AI) approaches that have been validated across several studies [18,27,38] to analyze human emotions. The algorithms are used to quantify emotion intensities, detect emotion state transitions, and identify profiles of impacted individuals based on the concerns they have expressed on social media to provide a holistic view of people’s mental health. This approach can be described as an ensemble of machine learning algorithms and natural language understanding (NLU) techniques that establish an end-to-end pipeline to extract, process, normalize, analyze, and aggregate self-reported information in its unstructured format into topics and emotions expressed during the key phases of the pandemic. We have also compared the two lockdown periods and the social media expressions in response to restrictions imposed during each lockdown to capture changes in temporal emotions and related concerns.

    The rest of the paper is organized as follows. The Methods section presents an overview of the methodology used in this study and the Results section presents the outcomes of the analysis. The paper concludes with the Discussion section, which outlines the implications of the study, limitations, and potential avenues for future research.

    Background

    The current literature reports many research studies that detect sentiment and emotions from social media using a variety of techniques. Sentiment analysis has been explored using lexical and statistical approaches [39-42] and recent research studies have extended this by using advanced deep learning models that can detect the bipolarity of social media conversations [43-46]. While sentiment analysis classifies posts into negative and positive categories, emotion detection extends this categorization by detecting granular variants of emotions [47]. Many emotion models, such as Ekman’s basic emotion model [48], the valence and arousal model [49], and Plutchik’s emotion model [50], are being used as the basis for emotion detection. Most of the approaches of emotion detection rely on prelabeled data via crowdsourced annotations and semantics to develop machine learning models [51-53]. Current research studies report the use of many deep learning models, such as word embedding models [46,54], convolutional neural networks [55], and recurrent neural networks [56-58], for emotion classification. Emoticons and emojis have also been used in studies to infer emotions from social media as they represent user-annotated labels for emotion classification [56,59]. In addition to the emotion classification, emotion intensity is also captured via the annotated data sets [52]. A more recent research study reports on the use of the stacked ensemble method for emotion intensity detection that enables the identification of expressions that denote the emotion intensity [60].

    However, these approaches are focused on detecting sentiment or emotion from a social media post but do not focus on emotion modeling, which makes it possible to explore underlying changes of intensity, shifts, and patterns of change in emotions and to generate insights by linking to associated human behaviors [61]. The lack of labeled data for emotion detection is also a major drawback in using supervised machine learning approaches. Therefore, it requires a robust method to explore emotions in unstructured, voluminous social media data that does not require prior knowledge about the data. In this study, we use an ensemble of word embedding and NLU techniques to derive emotions from unlabeled social media conversations. As the foundation of the emotion analysis, we use Plutchik’s model [50], which comprises eight basic emotions: anger, sadness, joy, trust, anticipation, fear, disgust, and surprise. The emotion classification is based on these eight emotions, and their intensities are mapped to show the strength of each emotion.


    Data

    Twitter conversations that were posted from January to September 2020 within Australia or an Australian context were extracted. Popular hashtags related to COVID-19 in Australia were used to query the data sources. Content by news channels and bots was identified by examining an unusual volume of data generated by each user and was eliminated from the analysis. This process yielded 73,000 public conversations by Twitter users in Australia. The extracted data were cleaned, preprocessed, and anonymized before being used in the analysis.

    In order to identify different phases of the pandemic, the stages from Table 1 were formed to align with the timeline of the pandemic experience in Australia [62].

    Figure 1 presents the high-level view and the workflow of the framework with the respective components.

    Table 1. The four main phases of the COVID-19 pandemic from January to September 2020.
    Time period (in 2020)Phase
    January to FebruaryPrepandemic
    March to MayFirst lockdown
    JuneEasing restrictions
    July to SeptemberSecond lockdown
    ‎
    Figure 1. The high-level view and workflow of the artificial intelligence framework. NLU: natural language understanding.
    View this figure

    Emotion Analysis

    The emotion extraction, intensity generation, and emotion transition modeling process is shown in the emotions component of the architecture diagram in Figure 1 (a). The emotion analysis was conducted to represent the emotions of each stage as well as the temporal changes in emotions over time. The emotion extraction was carried out on chronologically ordered social media conversations in order to represent emotions over a conversation as a sequence of individual, single-post emotions. First, we trained a word2vec [63] word embedding model, which is a deep learning technique to create vector representations of textual data. This allows the positioning of semantically similar terms together, thereby enabling the finding of closely associated terms for each basic emotion. This querying process yielded a rich vocabulary for representing each emotion, from the basic emotions, which was used as the basis for emotion extraction.

    Next, emotion negation was handled to fine-tune the extracted emotions. This process resulted in a sequence of emotions that denoted the chronologically ordered emotions and change of emotions over a conversation. The intensity of emotions in the four stages was used to model an intensity profile for each stage. We have used a modified intensity-capturing algorithm presented in Adikari et al [16] that accommodates the frequency and the presence of expressions that increase or inhibit the valence of the emotion. This allowed for the identification of prominent emotions and their differences in each stage, as seen in Figure 1 (b).

    The temporal emotion analysis was conducted by modeling emotion transition models, which models the probabilities of emotion state changes over a course of time. Affective computing research suggests that basic emotions at a particular instance—represented by one post of a tweet in this application—can be regarded as discrete states, and the interactions among these can be modeled to represent the likelihood of the change in the state of emotions [64]. For this purpose, a mathematical model can be used to model the interactions between the emotion states. We have used the sequence of emotions at each stage to generate respective emotion transition models using Markov models [65,66], as seen in Figure 1 (c). The use of emotion transitions to capture individual and group emotions using social media conversations has been successful in a smart city context [17] and for analysis of mental health via online data sources [16]. We have applied this to model the emotion state transitions of each stage, which reflect the flow of emotions as the pandemic progressed.

    Topic and Topic Association Analysis

    The topic analysis was conducted based on a series of NLU techniques to capture topics of discussions from social media data, as seen in Figure 1 (d). The topics represent people’s concerns and, therefore, encompass useful insights on their discourse patterns related to COVID-19 concerns. Outcomes of this analysis help in better interpreting the outcomes of the emotion analysis by providing contextual information for different emotion changes.

    To conduct the topic analysis, the data set was first cleaned and processed in order to remove weblinks, stop words, news links, and repeating content generated by bots. This cleaned data set was then split into each stage and taken for the topic analysis. Given the large amount of content, an unsupervised topic modeling technique was used, which groups similar terms based on latent Dirichlet allocation [67]. Once the prominent topic clusters were identified, a trained word2vec embedding model and an automated keyphrase extraction algorithm [68] were used to identify similar topics and subtopics. The outcomes generated the most frequently mentioned terms as well as semantically prominent terms, which allowed us to understand a discrete topic from the terms. The analysis was conducted separately for the four stages to understand which topics were prominent in each stage; thereafter, a topic association map was created to identify common topics across stages to demonstrate people’s continuing concerns.

    Behavior Profiling

    Behavior profiling was conducted to identify different groupings of citizens in the community based on their emotions and concerns expressed over social media. Identification of such profiles can form a visualization of different profiles that exist in society. This representation is useful to better understand different citizen needs and can inform decision making for relevant authorities.

    For profiling, we used an improved variant of the growing self-organizing map (GSOM) algorithm [69]. The GSOM presents a map topography that self-structures by adapting its size and shape based on the attributes and variations of input data. The GSOM algorithm utilizes competition and correlative learning based on the data provided. As input data are presented, nodes of the network compete among each other for ownership of the input, and the winners strengthen their relationships with this input. The competitive learning process is repeated for the complete data set for several cycles, and ultimately the map associates output nodes with patterns in the input data set. The GSOM acts as a knowledge map and can discard outdated information and overfitting of knowledge during the knowledge acquisition process [70,71] that allows the retention of relevant information only. The primary reason to incorporate such mature self-structuring AI was to discover hidden representations of behavior profiles based on their discourse and emotion patterns, which is otherwise challenging to interpret from a large amount of data. The ability of the GSOM algorithm to uncover patterns among data without prior training or supervision makes it possible to detect different groups of behaviors present among the citizens.

    Ethics Statement

    We have obtained ethics approval for this research from the La Trobe University Human Ethics Committee.


    Topics and Topic Associations

    Social media contains a high volume of expressions by individuals and groups alike. However, most of these expressions can be aggregated into a finite number of themes and topics [72]. We conducted a topic analysis and a topic association analysis to explore the distinct and common themes across the four stages of the pandemic. Table 2 presents the topics discussed during the four stages of the pandemic, from January to September 2020. The most prominent topics in each stage are presented in Table 2, where the quantitative measure indicates the weight of each topic based on the volume. The topics were also associated across stages as depicted in Figure 2. Key observations are that government decisions and travel restrictions were discussed during all four stages, and both lockdown stages expressed similar topics oriented toward lifestyle factors. These common topics included education, mental health, family welfare, and facilities. Panic buying was only mentioned during the first lockdown and was entirely absent during the second lockdown. During the easing restrictions stage, people mostly talked about social distancing and safety as well as hotel quarantine, which were common topics during the second lockdown as well. Apart from the common topics, the second lockdown mainly focused on health care, community transmission, economy, and vaccine. These denote that the conversations were more informed and regulated by the second lockdown. It should be noted that only conversations with prominent topics were included in the analysis and that general conversations have been omitted from the topic analysis. The outcomes of this analysis also aid in comprehending the emotions by providing contextual information related to each stage.

    Table 2. Topics of discussion across the four stages of the pandemic.
    Pandemic stage and main themesVolume of conversations, n (%)
    Prepandemic (n=15,302)

    		Global concern6021 (39.35)

    		Local cases3522 (23.02)

    		Travel restrictions2518 (16.46)

    		Government decisions2127 (13.90)

    		International students588 (3.84)

    		New virus526 (3.44)
    First lockdown (n=23,201)

    		Work benefits6116 (26.36)

    		Education5832 (25.14)

    		Government decisions3765 (16.23)

    		Travel restrictions3069 (13.23)

    		Mental health2568 (11.07)

    		Family welfare1176 (5.07)

    		Facilities413 (1.78)

    		Panic buying262 (1.13)
    Easing restrictions (n=11,801)

    		Government decisions2811 (23.82)

    		Travel restrictions2700 (22.88)

    		Hotel quarantine2294 (19.44)

    		Breaching rules1554 (13.17)

    		Work benefits1184 (10.03)

    		Social distancing and safety925 (7.84)

    		Environment concerns333 (2.82)
    Second lockdown (n=16,198)

    		Social distancing and safety2910 (17.96)

    		Health care2112 (13.04)

    		Work benefits2046 (12.63)

    		Government decisions1902 (11.74)

    		Community transmission1793 (11.07)

    		Facilities897 (5.54)

    		Economy886 (5.47)

    		Hotel quarantine867 (5.35)

    		Education841 (5.19)

    		Family welfare829 (5.12)

    		Mental health489 (3.02)

    		Vaccine381 (2.35)

    		Travel restrictions 245 (1.51)
    ‎
    Figure 2. Topic associations between the four stages of the pandemic.
    View this figure

    Emotion Classification and Intensities

    The AI approach used for the study of emotions expressed during the pandemic began with the classification of emotions, based on the widely cited psychological emotion model proposed by Plutchik [50]. Classification of emotions was followed by the quantification of the intensity of each of the eight emotions as they were expressed during the four stages.

    The outcomes of the emotion analysis suggested that both sadness and fear emotions were the most intense during the prepandemic stages. This is indicative of the central emotional response during a pandemic being fear [2]. In addition, it is said that given the instinctive defensive systems for combating ecological threats, humans experience negative emotions resulting from a threat that can be contagious. The high intensity of sadness and fear identified during the announcement stage of the pandemic exhibits this emotional pattern. This behavior was also confirmed by the emotion transition model in this stage, where higher probabilities were observed for these emotions. Furthermore, fear is also associated with panic, where people act blindly and excessively out of self-preservation, possibly endangering the survival of others [2]; this was experienced via incidents like panic buying, where people unreasonably stocked up on essential items for impending self-isolation. The surfacing of fear and sadness emotions represents increased anxiety, which impacts people’s mental health [73].

    The temporal emotion analysis showed that, out of the negative emotions, anger and disgust were more strongly expressed toward the latter part of the pandemic due to the security breaches and restrictions. Terms such as “covidiot,” “quarantine issues,” “restrictions,” and “another lockdown” were mostly mentioned in the Twitter conversations related to these emotions. This indicates that starting from the negativity of fear and sadness, public emotions have transferred to more anger and disgust at a later stage.

    Positive emotion expressions were also captured by the analysis. It was noted that joy was more strongly expressed during the easing restriction stage and least during the second lockdown stage. This behavior aligned with real-world behaviors, as it was expected that people would express disappointment for having to experience another lockdown. The emotions trust and anticipation demonstrated higher intensities during the first lockdown and easing restriction stages; however, they have subsided over time (Figure 3).

    We conducted a comparative study based on the two lockdown stages to identify the differences in emotion intensity (Table 3). The differences were compared using the Pearson chi-square test [74,75], and the confidence intervals were calculated based on the recommended method given by Altman et al [76]. A significance level of P<.05 was used to determine if the differences in the intensities were significant.

    The comparison of negative emotions shows that, except for sadness, all other negative emotion intensities were significantly higher in the second lockdown (P<.05). This evidently demonstrates the increased disappointment and negativity of having to experience a second lockdown. The emotion sadness was persistent in both lockdowns.

    The positive emotion comparison shows that, apart from anticipation, all other positive emotions were strongly expressed in the first lockdown (P<.05). This suggests that people had elicited more positive thoughts during the first lockdown compared to the second. The increase in the intensity of negative emotions and the reduction of positive emotions with further lockdowns confirm the increased levels of distress and deterioration of mental health. Based on this, relevant strategies should be initiated in order to eliminate the spread of negativity among people in further lockdowns.

    ‎
    Figure 3. Emotion intensity fluctuations over time.
    View this figure
    Table 3. Comparison of emotion intensities between the first and second lockdowns.
    EmotionNormalized emotion intensity during the first lockdown (n=36,317), %Normalized emotion intensity during the second lockdown (n=10,604), %Difference, % (95% CI)P value
    Negative emotions

    		Anger13.8415.972.13 (1.3567 to 2.9218)<.001

    		Sadness31.3631.010.35 (–0.6563 to 1.3461).49

    		Disgust9.8112.062.25 (1.5691 to 2.9518)<.001

    		Fear17.5819.662.08 (1.2365 to 2.9399)<.001
    Positive emotions

    		Joy16.9412.564.38 (3.6308 to 5.1097)<.001

    		Surprise1.320.740.58 (0.3662 to 0.7708)<.001

    		Trust6.565.630.93 (0.4108 to 1.4260)<.001

    		Anticipation2.592.370.22 (–0.1257 to 0.5406).21

    Emotion Transitions

    Mapping and quantifying the transition of emotions over conversations can be useful to predict the likelihood of emotion state changes as well as to observe temporal change in emotions and intensities over time. In a social media setting, this represents collective emotions in a community over time. The emotion state transitions were modeled separately for each stage, which enabled temporal analysis of emotions from the prepandemic stage to the second lockdown stage. For generating emotion transitions, the emotion space was defined as a set of discrete emotions—anger, fear, sadness, disgust, joy, anticipation, trust, and surprise—from the emotion extraction described earlier. Based on the temporal emotion sequence extracted from the emotion extraction module, a matrix was formed with the frequency of transitions of emotions from one state to another, and the matrix was used to create the emotion state transition diagrams shown in Figure 4; the shift and emergence of emotions are demonstrated via the arrows connecting the emotion states. The probabilities denote the likelihood of changed emotion states (Table 4).

    Transitions at each stage demonstrate the temporal analysis of emotions during the pandemic. This emotion flow is useful when determining how people’s emotions have changed over time. Health care practitioners and the government can use this information and analysis to understand the mental health status of the population.

    Regarding the outcomes, it was observed that most of the emotion transitions involved the sadness state in the prepandemic stage. Sadness to fear was also prominent relative to other emotion changes. By the first lockdown, there was a shift toward positive emotions, as the emotion states anticipation and joy were seen to be prominent. This pattern deviated slightly in the third stage, where positive emotions had amplified and negative emotions had reduced. The emotion propagation made a drastic change in the second lockdown, where positive emotion behaviors had decreased and negative emotions had been amplified, eliciting more tendencies toward sadness, fear, and disgust. This propagation of emotions denoted the emotion flow among people as they voiced their thoughts on social media.

    The emergence and shift of emotions over different points in time were seen to be associated with the different stages across different time points in the COVID-19 pandemic. It is apparent that people experienced diverse emotions and had expressed them on social media reflecting their individual interests, opinions, and priorities. Further analysis was conducted using profiling techniques to investigate the intensity (ie, valence) of these emotions and the variation across different stages, which provided insights into who these people were and their priorities and opinions during the pandemic.

    ‎
    Figure 4. Emotion transitions of the four stages of the pandemic. Each arrow represents a transferring of one emotional state to another. The numbers on the arrows represent the likelihoods (ie, probabilities) of changed emotion states.
    View this figure
    Table 4. Top five emotion transitions and probabilities for each pandemic stage.
    Emotion transitionProbability
    Prepandemic

    		Anger → sadness0.483

    		Joy → sadness0.378

    		Anticipation → sadness0.358

    		Sadness → sadness0.346

    		Sadness → fear0.335
    First lockdown

    		Anger → sadness0.445

    		Joy → sadness0.354

    		Sadness → sadness0.334

    		Anticipation → joy0.329

    		Sadness → anticipation0.288
    Easing restrictions

    		Anticipation → joy0.393

    		Surprise → joy0.389

    		Sadness → fear0.362

    		Anger → sadness0.340

    		Joy → joy0.295
    Second lockdown

    		Anger → sadness0.417

    		Sadness → sadness0.357

    		Sadness → fear0.341

    		Anticipation → sadness0.324

    		Fear → disgust0.322

    Group Profiles Based on Topics and Emotions

    The GSOM algorithm was used to generate profiles of individuals based on topics and emotions. This enabled the identification of different clusters of citizens that existed within the community in terms of their concerns and emotions. Figures 5 and 6 illustrate the profiles identified for the period of the first and second lockdowns, respectively. Seven profiles were identified for the first lockdown based on the topics and emotions expressed by individuals, and they were labeled based on the most prominent topics that have been discussed. The profiles were (1) children’s education, (2) family-oriented discussions, (3) work concerns, (4) panic buying, (5) lifestyle-oriented discussions, (6) travelers and traveling, and (7) higher education–focused discussions.

    These profiles demonstrated a distinct focus in their conversations, which resulted in grouping those individuals together. Among these profiles, the panic buying cluster demonstrated higher levels of fear, whereas sadness was prominent in all clusters. In addition, the family cluster demonstrated more joy, which can be aligned with spending more time with family.

    The second lockdown profiles shifted slightly from the profiles identified in the first lockdown. The identified behavior profiles were (1) government decisions, (2) health and mental health concerns, (3) community transmission, (4) safety measures, and (5) work concerns.

    Compared to the first lockdown, the profiles of the second lockdown exhibited more negative emotions, where increased levels of negative emotions can be seen in health and mental health concerns. It is noteworthy to observe that the profiles formed in the second lockdown were more focused toward broader aspects of the pandemic when compared to the more lifestyle-oriented profiles in the first lockdown. This difference indicates that as the pandemic progressed, people’s concerns shifted more toward broader aspects than concerns at the early stages, which may have been normalized with time.

    ‎
    Figure 5. Behavior profiles in the first lockdown. The mean strength of each concern is denoted as a blue line. GSOM: growing self-organizing map.
    View this figure
    ‎
    Figure 6. Behavior profiles in the second lockdown. The mean strength of each concern is denoted as a blue line. GSOM: growing self-organizing map.
    View this figure

    Principal Findings and Implications

    At a high level, the themes and topics that were seen at the individual level in the first stages of the pandemic had cascaded into broader and informed topics during the second lockdown. Individuals frequently discussed matters related to family, panic buying, and facilities during the first stage and transitioned into generic topics, such as the vaccine and economy, in the latter stages. Besides, several common concerns persisted over the course of time. Identification of such behaviors related to topics and topic associations from social media conversations can inform public health authorities regarding necessities and the issues that disrupt people’s quality of life during this global pandemic. However, it is crucial to investigate the underlying emotions in these conversations to gauge the status of people’s mental health.

    Limitations and Future Work

    One of the limitations of this study is the use of a single data source. As Twitter users are not representative of the whole of Australia, we believe that this could pose a limitation when modeling the emotions and concerns of the Australian population. However, given the importance of Twitter as a social media platform and the substantial amount of data retrieved (73,000 conversations), we were able to capture a broad cross-section of the community. As future work, the study could be extended by using multiple data sources, such as online forums and other social media channels. This will create a holistic view of the community. Moreover, the investigation could be conducted using data from another country, as this would provide different insights into how people have reacted toward different restriction measures and pandemic stages.

    Comparison With Prior Work

    Our findings regarding emotions are consistent with and align with public health studies related to the COVID-19 pandemic. The emergence of the sadness and fear emotions during the initial stages resemble the increased anxiety and depression faced by Australians in the first few months of the pandemic [31,32]. Past research indicates that public health emergencies often trigger negative emotions, as people tend to develop aversion, anxiety, and fear [77,78]. The emotion fear is often associated with such emergencies and pandemics, as it is one of the core emotions linked with survival. It has been reported that the human mind’s instinctive defensive systems for combating ecological threats are often wired with negative emotions resulting from a threat that can be contagious [78]. Moreover, fear is also associated with panic behaviors, where humans act egoistically, endangering others’ lives as well. These behaviors were captured via the emotion and topic analysis of this study, demonstrating that social media reflected people’s underlying emotions and mental health concerns.

    Studies investigating social media sentiments and topics during the pandemic align with our study [20,79,80]. In addition, our study provides an extension to the current body of COVID-19 social media research by modeling the emotion state transitions over time to represent the shift of emotions over time. The profiling of citizens based on emotions and concerns also enables the identification of clusters in the community during different stages of the pandemic. Recent research related to identifying the role of social media in the COVID-19 pandemic suggested further exploration of how social media can be utilized to inform health care practices [23]. We believe that our study contributes to providing outcomes that can enhance health care practices related to community understanding and uplifting of mental health care during a similar crisis.

    Conclusions

    As the entire world is still grappling with the direct or indirect effects of the COVID-19 pandemic, it is pertinent that we explore and analyze human emotions and related expressions in order to aid postpandemic recovery, as well as to prepare for future crises. Given the current social isolation settings, people are experiencing increased mental health issues and emotion changes that are affecting their quality of life [81,82]. During times of social isolation, the abundance of data available in digital spaces provides a reflection of people’s behaviors, which is otherwise challenging to assess via clinical trials or interviews. This enables emotion analysis using digital spaces to act as an adjunct to real-world behavior analysis [83].

    This study focused on capturing human emotions and concerns, which were reflected through social media conversations as the COVID-19 pandemic progressed. Our work is targeted at detecting and helping to understand the underlying mental health– and well-being–related issues via emotions expressed and topics discussed on social media by different segments of society. Although prevention of spread with social distancing and masking, as well as attempts at the elimination of COVID-19 with vaccine programs, have been widely discussed, less attention has been paid to the less visible issue of emotional distress and mental health. Governments and health care organizations are now realizing the deep and long-term effects of these issues; funding is being allocated to address these issues and programs for understanding the implications are now being established [81]. The work described here is focused on this less discussed area. We proposed an AI-based approach and designed and developed a supporting technical framework to learn about the emotion and content representations from unstructured, voluminous social media data, which are otherwise challenging to assess manually. The outcomes from this study aligned with theories of social sharing, emotional responses during pandemics, and collective emotion theories, and they represent insights that can be used to improve the mental health of citizens.

    The outcomes of the emotion analysis demonstrated a high intensity of fear and sadness emotions during the announcement stage of the COVID-19 pandemic, and the emotion transition model also demonstrated higher probabilities of expressing fear and sadness during the prepandemic stage among Australian citizens. Noteworthy insights were produced as the outcomes of the temporal emotion analysis. The emotion transition models across the pandemic demonstrated how the emotions had changed over time. It was noted that, although fear and sadness emotions were more prominent over the first few months, they had eventually transitioned into anger and disgust as people expressed their dissatisfaction and frustration with the continuing pandemic and restrictions. The temporal emotion analysis during the pandemic using emotion transition models was one of the main contributions of this study, as it demonstrates how people’s emotions change over time. On the other hand, positive emotions such as trust and anticipation were observed in the first lockdown and easing restrictions stages; however, they had diminished over time as the pandemic progressed. This emergence and shift of different emotions is a strong indicator of people’s varying emotions, and it signals the underlying mental health of the population. The decreased intensity of sadness and fear in the latter stage showed the normalization of emotions as people tended to realize the nature of the pandemic. However, the increased emotion transitions toward anger and disgust as well as broader concerns represented the dissatisfaction toward the management of the pandemic. These emotion behaviors presented insights into people’s emotions, which were challenging to quantify and measure using traditional surveys or clinical trials, given the social isolation settings. Social media has been effective in helping us understand human emotion expressions during times of crisis, and it can be utilized to improve health care practices in similar crises.

    On the other hand, emotions expressed on social media can be contagious, as negative content attracts more attention and propagation on social media [84,85]. The increased intensity of negative emotions and the persistence of negativity over time that was captured across different stages in this study aids this phenomenon. Negative emotions were observed in both first and second lockdowns and were significantly higher during the second lockdown. Therefore, it is crucial to observe this negative content to avoid further escalation. These swirls of emotions generated at one point in time have the ability to propagate through time using social media as the medium, as social media holds the potential for cascading and going viral to transmit emotional content more rapidly and broadly than other media [86,87]. These behaviors can also be aligned with the social sharing of emotion, which states that collective emotional events are anticipated to trigger a social process of emotion propagation [12]. It has been stated that emotion strength, presence, and valence can affect and influence citizen engagement through social media; therefore, it is imperative to moderate and monitor such emotional behaviors on digital platforms [88]. As potential implications, the proposed emotion behavior analysis in the study can be used to moderate online negativity and even detect distress among people during the pandemic to uplift their mental health as well as in postpandemic times as measures of precautions.

    Another key observation of this study was the profiling of people based on similar emotions and thematic expressions on social media. It was observed that while certain groups of people were more enthused about family matters, some groups expressed more concerns related to employment issues. The deviation of profiles from the first lockdown to the second lockdown also shows how the identified profiles took different shapes depending on the concerns at that point in time. Such behavior profiling enables the understanding of different groups of people with different concerns and emotions. Such groups elicit significant insights and enable us to look at the community from different perspectives. Based on these behavior changes, governance and health care policy-making practices could be tailored to consider different groups of concerns emerging from different sectors within the community. Insights can also inform future planning to uplift citizens’ mental health during disastrous events similar to the current pandemic.

    Apart from the aforementioned implications, one of the key outcomes of this study was to showcase the applicability and capability of using social media conversations to identify, analyze, and understand human behaviors at scale during a global crisis. Digital avatars created in social media have become a derivative of reality. This study showcases that in the age of social media, it is helpful, or even essential, to study human behaviors and emotions using their digital representations.

    Conflicts of Interest

    None declared.

    1. Johns Hopkins Coronavirus Resource Center.   URL: https://coronavirus.jhu.edu/ [accessed 2020-12-16]
    2. Bavel JJV, Baicker K, Boggio PS, Capraro V, Cichocka A, Cikara M, et al. Using social and behavioural science to support COVID-19 pandemic response. Nat Hum Behav 2020 May;4(5):460-471. [CrossRef] [Medline]
    3. Gao J, Zheng P, Jia Y, Chen H, Mao Y, Chen S, et al. Mental health problems and social media exposure during COVID-19 outbreak. PLoS One 2020;15(4):e0231924 [FREE Full text] [CrossRef] [Medline]
    4. Valdez D, Ten Thij M, Bathina K, Rutter L, Bollen J. Social media insights into US mental health during the COVID-19 pandemic: Longitudinal analysis of Twitter data. J Med Internet Res 2020 Dec 14;22(12):e21418 [FREE Full text] [CrossRef] [Medline]
    5. Goel A, Gupta L. Social media in the times of COVID-19. J Clin Rheumatol 2020 Jun 11;26(6):220-223. [CrossRef]
    6. Alexander DE. Social media in disaster risk reduction and crisis management. Sci Eng Ethics 2014 Sep;20(3):717-733. [CrossRef] [Medline]
    7. Gao H, Barbier G, Goolsby R. Harnessing the crowdsourcing power of social media for disaster relief. IEEE Intell Syst 2011 May;26(3):10-14. [CrossRef]
    8. Palen L, Hughes A. Social media in disaster communication. In: Rodríguez H, Donner W, Trainor J, editors. Handbook of Disaster Research. Cham, Switzerland: Springer; 2018:497-518.
    9. Abd-Alrazaq A, Alhuwail D, Househ M, Hamdi M, Shah Z. Top concerns of tweeters during the COVID-19 pandemic: Infoveillance study. J Med Internet Res 2020 Apr 21;22(4):e19016 [FREE Full text] [CrossRef] [Medline]
    10. Panayiotou G, Panteli M, Leonidou C. Coping with the invisible enemy: The role of emotion regulation and awareness in quality of life during the COVID-19 pandemic. J Contextual Behav Sci 2021 Jan;19:17-27. [CrossRef]
    11. Restubog SLD, Ocampo ACG, Wang L. Taking control amidst the chaos: Emotion regulation during the COVID-19 pandemic. J Vocat Behav 2020 Jun;119:103440 [FREE Full text] [CrossRef] [Medline]
    12. Rimé B. Emotion elicits the social sharing of emotion: Theory and empirical review. Emot Rev 2009;1(1):60-85. [CrossRef]
    13. Jin Y, Liu BF, Anagondahalli D, Austin L. Scale development for measuring publics’ emotions in organizational crises. Public Relat Rev 2014 Sep;40(3):509-518. [CrossRef]
    14. Rimé B, Páez D, Basabe N, Martínez F. Social sharing of emotion, post-traumatic growth, and emotional climate: Follow-up of Spanish citizen's response to the collective trauma of March 11th terrorist attacks in Madrid. Eur J Soc Psychol 2009 Sep 09;40(6):1029-1045. [CrossRef]
    15. Jin Y, Fraustino JD, Liu BF. The scared, the outraged, and the anxious: How crisis emotions, involvement, and demographics predict publics’ conative coping. Int J Strateg Commun 2016 Apr 19;10(4):289-308. [CrossRef]
    16. Adikari A, Gamage G, de Silva D, Mills N, Wong SJ, Alahakoon D. A self structuring artificial intelligence framework for deep emotions modeling and analysis on the social web. Future Gener Comput Syst 2021 Mar;116:302-315. [CrossRef]
    17. Adikari A, Alahakoon D. Understanding citizens’ emotional pulse in a smart city using artificial intelligence. IEEE Trans Industr Inform 2021 Apr;17(4):2743-2751. [CrossRef]
    18. Adikari A, de Silva D, Ranasinghe WKB, Bandaragoda T, Alahakoon O, Persad R, et al. Can online support groups address psychological morbidity of cancer patients? An artificial intelligence based investigation of prostate cancer trajectories. PLoS One 2020;15(3):e0229361 [FREE Full text] [CrossRef] [Medline]
    19. Xue J, Chen J, Hu R, Chen C, Zheng C, Su Y, et al. Twitter discussions and emotions about the COVID-19 pandemic: Machine learning approach. J Med Internet Res 2020 Nov 25;22(11):e20550 [FREE Full text] [CrossRef] [Medline]
    20. Chandrasekaran R, Mehta V, Valkunde T, Moustakas E. Topics, trends, and sentiments of tweets about the COVID-19 pandemic: Temporal infoveillance study. J Med Internet Res 2020 Oct 23;22(10):e22624 [FREE Full text] [CrossRef] [Medline]
    21. Hung M, Lauren E, Hon ES, Birmingham WC, Xu J, Su S, et al. Social network analysis of COVID-19 sentiments: Application of artificial intelligence. J Med Internet Res 2020 Aug 18;22(8):e22590 [FREE Full text] [CrossRef] [Medline]
    22. Al-Rawi A, Siddiqi M, Morgan R, Vandan N, Smith J, Wenham C. COVID-19 and the gendered use of emojis on Twitter: Infodemiology study. J Med Internet Res 2020 Nov 05;22(11):e21646 [FREE Full text] [CrossRef] [Medline]
    23. Al-Dmour H, Masa'deh R, Salman A, Abuhashesh M, Al-Dmour R. Influence of social media platforms on public health protection against the COVID-19 pandemic via the mediating effects of public health awareness and behavioral changes: Integrated model. J Med Internet Res 2020 Aug 19;22(8):e19996 [FREE Full text] [CrossRef] [Medline]
    24. Choudhury M, De S. Mental health discourse on reddit: Self-disclosure, social support, and anonymity. In: Proceedings of the 8th International AAAI Conference on Weblogs and Social Media. 2014 Presented at: 8th International AAAI Conference on Weblogs and Social Media; June 1-4, 2014; Ann Arbor, MI p. 71-80   URL: https://ojs.aaai.org/index.php/ICWSM/article/view/14526/14375
    25. De Choudhury M, Counts S, Horvitz E. Predicting postpartum changes in emotion and behavior via social media. In: Proceedings of the 31st SIGCHI Conference on Human Factors in Computing Systems. New York, NY: Association for Computing Machinery; 2013 Presented at: 31st SIGCHI Conference on Human Factors in Computing Systems; April 27-May 2, 2013; Paris, France p. 3267-3276. [CrossRef]
    26. De CM, Counts S, Horvitz E. Social media as a measurement tool of depression in populations. In: Proceedings of the 5th Annual ACM Web Science Conference. New York, NY: Association for Computing Machinery; 2013 Presented at: 5th Annual ACM Web Science Conference; May 2-4, 2013; Paris, France p. 47-56. [CrossRef]
    27. Adikari A, Gamage G, de Silva D, Mills N, Wong SJ, Alahakoon D. A self structuring artificial intelligence framework for deep emotions modeling and analysis on the social web. Future Gener Comput Syst 2021 Mar;116:302-315. [CrossRef]
    28. Cha M, Haddadi H, Benevenuto F, Gummadi K. Measuring user influence in Twitter: The million follower fallacy. In: Proceedings of the 4th International AAAI Conference on Weblogs and Social Media. 2010 Presented at: 4th International AAAI Conference on Weblogs and Social Media; May 23-26, 2010; Washington, DC p. 10-17   URL: https://aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1538/1826
    29. Yosufzai R. Australia among top 10 countries ranked for pandemic response, NZ tops list. SBS News. 2021 Jan 28.   URL: https:/​/www.​sbs.com.au/​news/​australia-among-top-10-countries-ranked-for-pandemic-response-nz-tops-list [accessed 2020-12-16]
    30. Hong J, Chang R, Varley K. The Covid resilience ranking: These are the best and worst places to be during the Covid pandemic. Bloomberg. 2020.   URL: https://www.bloomberg.com/graphics/covid-resilience-ranking/ [accessed 2020-12-16]
    31. Fisher J, Tran T, Hammarberg K, Sastry J, Nguyen H, Rowe H, et al. Mental health of people in Australia in the first month of COVID-19 restrictions: A national survey. Med J Aust 2020 Nov;213(10):458-464. [CrossRef] [Medline]
    32. Pachana N, Beattie E, Byrne G, Brodaty H. COVID-19 and psychogeriatrics: The view from Australia. Int Psychogeriatr 2020 May 12;32(10):1135-1141. [CrossRef]
    33. Rice ES, Haynes E, Royce P, Thompson SC. Social media and digital technology use among Indigenous young people in Australia: A literature review. Int J Equity Health 2016 May 25;15:81 [FREE Full text] [CrossRef] [Medline]
    34. Balakrishnan V, Teoh KK, Pourshafie T, Liew TK. Social media and their use in learning: A comparative analysis between Australia and Malaysia from the learners’ perspectives. Australas J Educ Technol 2016 Nov 11;33:81-97. [CrossRef]
    35. Ferguson C, Inglis SC, Newton PJ, Cripps PJ, MacDonald PS, Davidson PM. Collegian 2014;21(2):89-93. [CrossRef] [Medline]
    36. Anikeeva O, Steenkamp M, Arbon P. The future of social media use during emergencies in Australia: Insights from the 2014 Australian and New Zealand Disaster and Emergency Management conference social media workshop. Aust J Emerg Manag 2015 Jan;30(1):22-26 [FREE Full text]
    37. Flew T, Bruns A, Burgess J, Crawford K, Shaw F. Social media and its impact on crisis communication: Case studies of Twitter use in emergency management in Australia and New Zealand. In: Proceedings of the Communication and Social Transformation, ICA Regional Conference. 2013 Presented at: Communication and Social Transformation, ICA Regional Conference; November 8-10, 2013; Shanghai, China   URL: https://eprints.qut.edu.au/63707/15/ICA_Shanghai_paper_Social_Media_and_Crisis_Comms.pdf
    38. De Silva D, Ranasinghe W, Bandaragoda T, Adikari A, Mills N, Iddamalgoda L, et al. Machine learning to support social media empowered patients in cancer care and cancer treatment decisions. PLoS One 2018;13(10):e0205855 [FREE Full text] [CrossRef] [Medline]
    39. Esuli A, Sebastiani F. SENTIWORDNET: A publicly available lexical resource for opinion mining. In: Proceedings of the 5th International Conference on Language Resources and Evaluation.: European Language Resources Association (ELRA); 2006 Presented at: 5th International Conference on Language Resources and Evaluation; May 22-28, 2006; Genoa, Italy p. 417-422   URL: http://www.lrec-conf.org/proceedings/lrec2006/pdf/384_pdf.pdf
    40. Baccianella S, Esuli A, Sebastiani F. SENTIWORDNET 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of the 7th International Conference on Language Resources and Evaluation. 2010 Presented at: 7th International Conference on Language Resources and Evaluation; May 17-23, 2010; Valletta, Malta p. 2200-2204   URL: http://www.lrec-conf.org/proceedings/lrec2010/pdf/769_Paper.pdf
    41. Strapparava C, Valitutti A. WordNet-Affect: An affective extension of WordNet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation. 2010 Presented at: 4th International Conference on Language Resources and Evaluation; May 26-28, 2004; Lisbon, Portugal p. 1083-1086   URL: http://www.lrec-conf.org/proceedings/lrec2004/pdf/369.pdf
    42. Franzoni V, Biondi G, Milani A. A web-based system for emotion vector extraction. In: Proceedings of the 17th International Conference on Computational Science and its Applications. Cham, Switzerland: Springer International Publishing; 2017 Presented at: 17th International Conference on Computational Science and its Applications; July 3-6, 2017; Trieste, Italy p. 653-668. [CrossRef]
    43. Araque O, Corcuera-Platas I, Sánchez-Rada JF, Iglesias CA. Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Syst Appl 2017 Jul;77:236-246. [CrossRef]
    44. Basiri ME, Nemati S, Abdar M, Cambria E, Acharya UR. ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis. Future Gener Comput Syst 2021 Feb;115:279-294. [CrossRef]
    45. Cambria E, Schuller B, Xia Y, Havasi C. New avenues in opinion mining and sentiment analysis. IEEE Intell Syst 2013 Mar;28(2):15-21. [CrossRef]
    46. Giatsoglou M, Vozalis MG, Diamantaras K, Vakali A, Sarigiannidis G, Chatzisavvas KC. Sentiment analysis leveraging emotions and word embeddings. Expert Syst Appl 2017 Mar;69:214-224. [CrossRef]
    47. Franzoni V, Vallverdù J, Milani A. Errors, biases and overconfidence in artificial emotional modeling. In: Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence. New York, NY: Association for Computing Machinery; 2019 Presented at: IEEE/WIC/ACM International Conference on Web Intelligence; October 14-17, 2019; Thessaloniki, Greece p. 86-90. [CrossRef]
    48. Ekman P. An argument for basic emotions. Cogn Emot 2008 Jan 07;6(3-4):169-200. [CrossRef]
    49. Mohammad SM. Sentiment analysis: Detecting valence, emotions, and other affectual states from text. In: Meiselman HL, editor. Emotion Measurement. Cambridge, UK: Woodhead Publishing; 2016:201-237.
    50. Plutchik R. A psychoevolutionary theory of emotions. Soc Sci Inf 2017 Feb 06;21(4-5):529-553. [CrossRef]
    51. Binali H, Wu C, Potdar V. Computational approaches for emotion detection in text. In: Proceedings of the 4th IEEE International Conference on Digital Ecosystems and Technologies. 2010 Presented at: 4th IEEE International Conference on Digital Ecosystems and Technologies; April 12-15, 2010; Dubai, United Arab Emirates p. 172-177. [CrossRef]
    52. Mashal S, Asnani K. Emotion intensity detection for social media data. In: Proceedings of the International Conference on Computing Methodologies and Communication. 2017 Presented at: International Conference on Computing Methodologies and Communication; July 18-19, 2017; Erode, India p. 155-158. [CrossRef]
    53. Rao Y, Li Q, Wenyin L, Wu Q, Quan X. Affective topic model for social emotion detection. Neural Netw 2014 Oct;58:29-37. [CrossRef] [Medline]
    54. Herzig J, Shmueli-Scheuer M, Konopnicki D. Emotion detection from text via ensemble classification using word embeddings. In: Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. New York, NY: Association for Computing Machinery; 2017 Presented at: ACM SIGIR International Conference on Theory of Information Retrieval; October 1-4, 2017; Amsterdam, the Netherlands p. 269-272. [CrossRef]
    55. Zhang L, Wang S, Liu B. Deep learning for sentiment analysis: A survey. WIREs Data Min Knowl Discov 2018 Mar 30;8(4):1. [CrossRef]
    56. Felbo B, Mislove A, Søgaard A, Rahwan I, Lehmann S. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017 Presented at: 2017 Conference on Empirical Methods in Natural Language Processing; September 7-11, 2017; Copenhagen, Denmark p. 1616-1626   URL: https://www.aclweb.org/anthology/D17-1169.pdf [CrossRef]
    57. Majumder N, Poria S, Hazarika D, Mihalcea R, Gelbukh A, Cambria E. DialogueRNN: An attentive RNN for emotion detection in conversations. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019 Jul Presented at: 33rd AAAI Conference on Artificial Intelligence; January 27-February 1, 2019; Honolulu, HI p. 6818-6825. [CrossRef]
    58. Rathnayaka P, Abeysinghe S, Samarajeewa C, Manchanayake I, Walpola M, Nawaratne R, et al. Gated recurrent neural network approach for multilabel emotion detection in microblogs. ArXiv. Preprint posted online on July 17, 2019 [FREE Full text]
    59. Wood ID, Ruder S. Emoji as emotion tags for tweets. In: Proceedings of the 10th International Conference on Language Resources and Evaluation, Emotion and Sentiment Analysis Workshop. 2016 Presented at: 10th International Conference on Language Resources and Evaluation, Emotion and Sentiment Analysis Workshop; May 23-28, 2016; Portorož, Slovenia p. 76-79   URL: http://www.lrec-conf.org/proceedings/lrec2016/workshops/LREC2016Workshop-ESA_Proceedings.pdf
    60. Akhtar MS, Ekbal A, Cambria E. How intense are you? Predicting intensities of emotions and sentiments using stacked ensemble [application notes]. IEEE Comput Intell Mag 2020 Feb;15(1):64-75. [CrossRef]
    61. Cambria E. Affective computing and sentiment analysis. IEEE Intell Syst 2016 Mar;31(2):102-107. [CrossRef]
    62. O’Sullivan D, Rahamathulla M, Pawar M. The impact and implications of COVID-19: An Australian perspective. Int J Community Soc Dev 2020 Jul 01;2(2):134-151. [CrossRef]
    63. Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. In: Proceedings of the 1st International Conference on Learning Representations. 2013 Presented at: 1st International Conference on Learning Representations; May 2-4, 2013; Scottsdale, AZ   URL: https://arxiv.org/pdf/1301.3781
    64. Picard RW. Affective Computing. Cambridge, MA: The MIT Press; 1997.
    65. Han J, Xie L, Liu X, Wang Z. Emotional state transfer model based on FSM. In: Proceedings of the International Conference on Multisensor Fusion and Information Integration for Intelligent Systems. 2014 Presented at: International Conference on Multisensor Fusion and Information Integration for Intelligent Systems; September 28-30, 2014; Beijing, China p. 1-5. [CrossRef]
    66. Meng Q, Wu W. Artificial emotional model based on finite state machine. J Cent South Univ Technol 2008 Oct 1;15(5):694-699. [CrossRef]
    67. Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res 2003;3:993-1022 [FREE Full text]
    68. Rose S, Engel D, Cramer N, Cowley W. Automatic keyword extraction from individual documents. In: Berry MW, Kogan J, editors. Text Mining: Applications and Theory. Chichester, UK: John Wiley & Sons; 2010:3-20.
    69. Alahakoon D, Halgamuge SK, Srinivasan B. Dynamic self-organizing maps with controlled growth for knowledge discovery. IEEE Trans Neural Netw 2000;11(3):601-614. [CrossRef] [Medline]
    70. Nawaratne R, Alahakoon D, De Silva D, Kumara H, Yu X. Hierarchical two-stream growing self-organizing maps with transience for human activity recognition. IEEE Trans Industr Inform 2020 Dec;16(12):7756-7764. [CrossRef]
    71. Nawaratne R, Alahakoon D, De SD, Yu X. HT-GSOM: Dynamic self-organizing map with transience for human activity recognition. In: Proceedings of the IEEE 17th International Conference on Industrial Informatics. 2019 Presented at: IEEE 17th International Conference on Industrial Informatics; July 22-25, 2019; Helsinki-Espoo, Finland p. 270. [CrossRef]
    72. Rafea A, Gaballah N. Topic extraction in social media. In: Proceedings of the 2013 International Conference on Collaboration Technologies and Systems. 2013 Presented at: 2013 International Conference on Collaboration Technologies and Systems; May 20-24, 2013; San Diego, CA. [CrossRef]
    73. Podlogar MC, Rogers ML, Stanley IH, Hom MA, Chiurliza B, Joiner TE. Anxiety, depression, and the suicidal spectrum: A latent class analysis of overlapping and distinctive features. Cogn Emot 2018 Nov;32(7):1464-1477. [CrossRef] [Medline]
    74. Campbell I. Chi-squared and Fisher-Irwin tests of two-by-two tables with small sample recommendations. Stat Med 2007 Aug 30;26(19):3661-3675. [CrossRef] [Medline]
    75. Richardson JTE. The analysis of 2 × 2 contingency tables--Yet again. Stat Med 2011 Apr 15;30(8):890; author reply 891-890; author reply 892. [CrossRef] [Medline]
    76. Altman D, Machin D, Bryant T, Gardner M, editors. Statistics with Confidence: Confidence Intervals and Statistical Guidelines. 2nd edition. London, UK: BMJ Books; 2000.
    77. Li S, Wang Y, Xue J, Zhao N, Zhu T. The impact of COVID-19 epidemic declaration on psychological consequences: A study on active Weibo users. Int J Environ Res Public Health 2020 Mar 19;17(6):2032 [FREE Full text] [CrossRef] [Medline]
    78. Pakpour AH, Griffiths MD. The fear of COVID-19 and its role in preventive behaviors. J Concurr Disord 2020;2(1):58-63 [FREE Full text]
    79. Boon-Itt S, Skunkan Y. Public perception of the COVID-19 pandemic on Twitter: Sentiment analysis and topic modeling study. JMIR Public Health Surveill 2020 Nov 11;6(4):e21978 [FREE Full text] [CrossRef] [Medline]
    80. Rajput N, Grover B, Rathi V. Word frequency and sentiment analysis of Twitter messages during Coronavirus pandemic. ArXiv. Preprint posted online on April 8, 2020 [FREE Full text]
    81. Pfefferbaum B, North CS. Mental health and the Covid-19 pandemic. N Engl J Med 2020 Aug 06;383(6):510-512. [CrossRef] [Medline]
    82. Rajkumar RP. COVID-19 and mental health: A review of the existing literature. Asian J Psychiatr 2020 Aug;52:102066 [FREE Full text] [CrossRef] [Medline]
    83. Meshi D, Tamir DI, Heekeren HR. The emerging neuroscience of social media. Trends Cogn Sci 2015 Dec;19(12):771-782. [CrossRef] [Medline]
    84. Peng W, Adikari A, Alahakoon D, Gero J. Discovering the influence of sarcasm in social media responses. WIREs Data Min Knowl Discov 2019 Jul 29;9(6):1. [CrossRef]
    85. Naveed N, Gottron T, Kunegis J, Alhadi A. Bad news travel fast: A content-based analysis of interestingness on Twitter. In: Proceedings of the 3rd International Web Science Conference. New York, NY: Association for Computing Machinery; 2011 Presented at: 3rd International Web Science Conference; June 15-17, 2011; Koblenz, Germany. [CrossRef]
    86. Berger J, Milkman KL. Social Transmission, Emotion, and the Virality of Online Content. Marketing Science Institute Working Paper Series 2010 Report No. 10-114. New York, NY: Marketing Science Institute; 2010.   URL: https://www.msi.org/wp-content/uploads/2020/06/MSI_Report_10-114.pdf [accessed 2021-04-22]
    87. Hansen L, Arvidsson A, Nielsen F, Colleoni E, Etter M. Good friends, bad news - Affect and virality in Twitter. In: Proceedings of the 6th International Conference on Future Information Technology. 2011 Presented at: 6th International Conference on Future Information Technology; June 28-30, 2011; Crete, Greece p. 34-43. [CrossRef]
    88. Chen Q, Min C, Zhang W, Wang G, Ma X, Evans R. Unpacking the black box: How to promote citizen engagement through government social media during the COVID-19 crisis. Comput Human Behav 2020 Sep;110:106380 [FREE Full text] [CrossRef] [Medline]

    ‎
    AI: artificial intelligence
    GSOM: growing self-organizing map
    NLU: natural language understanding

    Edited by C Basch; submitted 21.01.21; peer-reviewed by E Cambria, V Franzoni; comments to author 15.02.21; revised version received 03.03.21; accepted 01.04.21; published 30.04.21

    Copyright

    ©Achini Adikari, Rashmika Nawaratne, Daswin De Silva, Sajani Ranasinghe, Oshadi Alahakoon, Damminda Alahakoon. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.04.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.