Published on in Vol 23, No 5 (2021): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26618, first published .
Understanding Public Perceptions of COVID-19 Contact Tracing Apps: Artificial Intelligence–Enabled Social Media Analysis

Understanding Public Perceptions of COVID-19 Contact Tracing Apps: Artificial Intelligence–Enabled Social Media Analysis

Understanding Public Perceptions of COVID-19 Contact Tracing Apps: Artificial Intelligence–Enabled Social Media Analysis

Original Paper

1Usher Institute, The University of Edinburgh, Edinburgh, United Kingdom

2School of Computing, Edinburgh Napier University, Edinburgh, United Kingdom

3University of Engineering and Technology, Lahore, Pakistan

4Edinburgh Medical School, The University of Edinburgh, Edinburgh, United Kingdom

5Department of Computer Science, University of Bristol, Bristol, United Kingdom

6Institute for the Study of Science, Technology and Innovation, The University of Edinburgh, Edinburgh, United Kingdom

Corresponding Author:

Kathrin Cresswell, BSc, MSc, PhD

Usher Institute

The University of Edinburgh

Teviot Place

Edinburgh, EH8 9AG

United Kingdom

Phone: 44 (0)131 651 4151

Email: kathrin.cresswell@ed.ac.uk


Background: The emergence of SARS-CoV-2 in late 2019 and its subsequent spread worldwide continues to be a global health crisis. Many governments consider contact tracing of citizens through apps installed on mobile phones as a key mechanism to contain the spread of SARS-CoV-2.

Objective: In this study, we sought to explore the suitability of artificial intelligence (AI)–enabled social media analyses using Facebook and Twitter to understand public perceptions of COVID-19 contact tracing apps in the United Kingdom.

Methods: We extracted and analyzed over 10,000 relevant social media posts across an 8-month period, from March 1 to October 31, 2020. We used an initial filter with COVID-19–related keywords, which were predefined as part of an open Twitter-based COVID-19 dataset. We then applied a second filter using contract tracing app–related keywords and a geographical filter. We developed and utilized a hybrid, rule-based ensemble model, combining state-of-the-art lexicon rule-based and deep learning–based approaches.

Results: Overall, we observed 76% positive and 12% negative sentiments, with the majority of negative sentiments reported in the North of England. These sentiments varied over time, likely influenced by ongoing public debates around implementing app-based contact tracing by using a centralized model where data would be shared with the health service, compared with decentralized contact-tracing technology.

Conclusions: Variations in sentiments corroborate with ongoing debates surrounding the information governance of health-related information. AI-enabled social media analysis of public attitudes in health care can help facilitate the implementation of effective public health campaigns.

J Med Internet Res 2021;23(5):e26618

doi:10.2196/26618

Keywords



SARS-CoV-2 is a major global threat to public health. Many governments worldwide are using mobile apps for contact tracing of citizens to contain the spread of SARS-CoV-2 [1]. The contact tracing applications used vary within and between countries, but they generally track movements of individuals and aggregate data to communicate to users when they may have been exposed to the virus, and if they need to be tested and/or self-isolated.

The higher the user base of contact tracing apps, the larger the anticipated impact on reducing the estimated effective reproduction number of the virus [2,3]. However, there is inconsistent and incomplete uptake of these technologies, especially in individualistic societies where intended users may perceive such apps to be of limited personal benefit or infringing on personal privacy and where the use of such apps is difficult to mandate [4,5]. To understand what public health measures may promote the use of contact tracing apps in the United Kingdom, policy makers need to understand the reasons why members of the public may be hesitant to use them. The empirical literature shows how attitudes of prospective users influenced by their individual backgrounds, technological features of the app in question, and perceived benefits and trade-offs can be a key barrier to the effective implementation of such apps (see Textbox 1) [6-9].

More than half of the world’s population, and around two-thirds of the UK population, currently use social media platforms, with significantly increased engagement levels during the ongoing pandemic [10]. We here sought to explore how analyzing public attitudes on these forums can provide insights into low levels of adoption of contact tracing apps in the United Kingdom. Our recent work has demonstrated the value of artificial intelligence (AI)–based sentiment analysis of social media data [11].

Factors likely to impact the use of contact tracing apps identified in the literature.

  • Individual backgrounds: demographics, health status, involvement with COVID-19, and previous experiences (technology, health, and other factors associated with COVID-19).
  • Technological features: effectiveness, privacy, security, cost, trust in system vendor, performance and reliability (eg, false-positive or false-negative alerts), compatibility with installed base, interoperability, and architecture of systems (eg, centralized vs decentralized).
  • Perceived benefits: self, society, research, epidemiology, mediators (eg, if the user or others will take action based on risk, if they care about the environment or society, if they believe data will be used effectively), and tension between perceived benefit to the user and altruistic benefit to the whole population.
  • Perceived trade-offs, risks, and limiting factors: data collected (who will know what about the user), risk of marginalization of certain demographic groups, level of control of who sees what aspects of data and who can retain it for how long, and transparency of app.
Textbox 1. Factors likely to impact the use of contact tracing apps identified in the literature.

Data Sources

To assess the potential of an AI-based sentiment analysis to understand public views and concerns, we analyzed data from two popular social media platforms—Facebook and Twitter. Facebook posts were extracted using the CrowdTangle platform [12], and Twitter posts were extracted from the COVID-19 Twitter chatter dataset, constructed by Panacea Lab (using a publicly available Twitter application programming interface) [13]. English-language Facebook posts and tweets posted in the United Kingdom from March 1 to October 31, 2020, were extracted, and thematically filtered using a two-step process, for keywords related to both COVID-19 and contact tracing apps. The Twitter dataset was already filtered with predefined COVID-19–related keywords as detailed previously [13], and the same keywords were used to apply an initial filter to the Facebook data. The following contract tracing app–related keywords for the second step filtering were selected by our interdisciplinary team: “covid app,” “tracing app,” “contact tracing,” “privacy,” “security,” “app security,” “app privacy,” “contain virus spread,” “movement tracking,” “tracking,” and “surveillance.” Following geographical filtering for the United Kingdom, a total of 2000 tweets and 8000 Facebook posts were obtained for the analysis.

Analysis

A novel ensemble-based AI model developed by the authors in a recent study [14] was adapted and utilized for this study, by combining lexicon rule-based and deep learning (DL)–based approaches. Specifically, an average-weighting ensemble [15] of lexicon-based models, including Valence Aware Dictionary and Sentiment Reasoner (VADER) [16] and TextBlob [17], was combined with a state-of-the-art DL-based model Bidirectional Encoder Representations from Transformers (BERT) [18]. The overall hybrid rule-based ensemble method is illustrated in Figure 1.

The hybrid rule-based ensemble model was hyperparameter tuned based on manual validation by using 1000 randomly selected posts and labelled by the team. Specificity and sensitivity analyses showed that the lexicon-based model demonstrated better performance accuracy for positive sentiment and that the DL-based BERT model demonstrated enhanced accuracy for neutral and negative sentiments, as shown in the confusion matrices (Multimedia Appendix 1). The TextBlob lexicon-based model utilizes a higher weight (0.52) than the VADER model, as it demonstrated marginally better accuracy (VADER × 0.48 + TextBlob × 0.52) for positive sentiments. For the ensemble, weighted averaging was utilized to combine the two lexicon-based models. The output of the weighted average lexicons was combined with the DL-based BERT model through a rule-based approach. For positive sentiments, the weighted average output of the lexicon models was selected as the final output. For neutral and negative sentiments, the output of the DL-based BERT model was chosen as the final output. If-else logical constructs were used to determine the final sentiment class based on the base model outputs.

Figure 1. Hybrid rule-based ensemble model for sentiment classification of social media data. AI: artificial intelligence; BERT: Bidirectional Encoder Representations from Transformers.
View this figure

Ethics

No ethical review was necessary since the data analyzed was fully in the public domain. A thorough assessment of the privacy risk to individuals posed by our research was conducted to ensure compliance with relevant sections of the General Data Protection Regulation (GDPR). We have also striven to comply with best practices for user protection to ensure no nonpublic material was included in our dataset.

Data Availability

The code and data used for data analysis and generation of figures are openly available on GitLab [19] for reproducibility and transparency of the analysis. Due to the computationally expensive nature of the code, we recommend using a high-performance computing resource.


Figure 2 shows the average weekly sentiment trends on using contact tracing apps on Facebook and Twitter. The sentiments were computed by utilizing our ensemble-based AI model to predict the overall polarity of each post as positive, negative, or neutral. Due to the relatively small number, sentiments extracted from Facebook and Twitter posts were combined using weighted weekly averages. Sentiment word clouds were used to explore the topics being discussed at different time points of interest (Figure 2). Overall, the average positive sentiments were found to far outnumber the negative sentiments. We observed a six-fold difference between the sentiment type, with 76% positive and 12% negative sentiments. These were found to broadly corroborate with findings from independent surveys that show strong support for contact tracing apps. For example, an Ipsos MORI survey of 1983 UK adults conducted in May 2020 found that two-thirds (67%) of the population was in support and 12% was against the government’s plans to use a smartphone app to track and trace new cases of COVID-19 [20]. However, the public’s attitudes were found to be heavily influenced by demographics and the digital divide [12,21]. Several new survey studies are currently underway to explore public attitudes towards contact tracing apps.

Figure 2 illustrates some of the key events that are likely to have influenced changes in the public’s sentiments toward using contact tracing apps. Following the successful implementation of contact tracing strategies in several East Asian countries and high-profile research reporting that app-based contact tracing was likely to be effective in curbing the spread of the virus [22-24], Apple and Google partnered on developing a decentralized COVID-19 contact tracing technology [25]. Shortly after, the UK government released contact tracing guidance with plans to develop a home-grown app and deploy it through a centralized model, implying that individual information would be shared with health services [26,27]. This model was controversial, primarily driven by public concerns around privacy and security of the app, its incompatibility with iPhones, and potential breaches of the Data Protection Law [28]. In June 2020, the centralized UK app was abandoned [29]. In August 2020, the UK government decided to implement the decentralized contact tracing system developed by Apple and Google [30], launching the app in September 2020 [31]. Since then, there have been concerns around high rates of false-positive COVID-19 cases reported via the app, hindering its uptake among the public [32].

We also carried out spatial mapping of the sentiments extracted from geo-tagged tweets to counties in the United Kingdom (see Figure 3). Most tweets had an overall positive sentiment toward COVID-19 contact tracing. The counties with the most positive sentiment included Lincolnshire, Norfolk, Nottinghamshire, Leicestershire, and Northamptonshire in England, and Stirling, Fife, Dumfries and Galloway, East Ayrshire, and West Lothian in Scotland. Counties with the most negative sentiment were Suffolk, Somerset, Devon, and North Yorkshire—all in England.

Figure 2. Average weekly sentiments extracted from Facebook and Twitter posts (combined) in the United Kingdom.
View this figure
Figure 3. Spatial mapping of (averaged) UK public sentiment related to COVID-19 contact tracing apps on Twitter.
View this figure

Principal Findings

This study has provided insights into some of the underlying issues and concerns surrounding contact tracing apps, and particularly how decisions surrounding system architecture (eg, centralized vs decentralized) can influence public attitudes.

More fine-grained spatiotemporal variation in sentiments across the United Kingdom, and globally, should be further studied. Failure to pay timely attention to negative attitudes and resistance to emerging technologies in population health can have potentially damaging consequences, whereas an understanding of barriers can facilitate the development of effective interventions.

Limitations of our preliminary analysis include that social media platform users are largely not representative of the total UK population (eg, users are younger, more left-wing, and have higher incomes) [33]. Only about 15% of UK adults are regular Twitter users, with activity levels varying among passive and active users, and the demographics of users do not reflect the demographics of the general population [34]. It has also been shown that sentiments of tweets can vary across geographical locations and may therefore reflect the demographics of the population being studied [35]. Variations in sentiments can occur at an individual level, influenced by personal experiences and changes not necessarily related to the subject of the tweet [36]. The developed ensemble-based sentiment analysis approach can be optimized by using additional labelled data for transfer learning and fine-tuning the BERT base model. Adaptive neuro-fuzzy inferencing [37] can also be utilized to replace the current nonadaptive (weighted-average and logical rule-based) constructs.

The current focus on selected social media platforms, outstanding issues around the accuracy of AI techniques (eg, around deceptive language), and the limited number of tweets and Facebook posts due to the specific search strategy used, as well as given that only geo-tagged tweets were used, may limit the generalizability of these findings. Thus, there is a need for a more refined and comprehensive search strategy using multiple social media and web platforms, as well as linked analysis with data from external trustworthy sources such as polls, census, surveys, and clinical notes. Considering the limitations, the proposed approach should currently only be used alongside other methods to assess public sentiment, including public consultations, surveys, and qualitative studies.

In future, mobility trends [38] can be integrated into this approach and demographic determinants can also be inferred [39] and included to provide further insights. Furthermore, subgroup analysis could explore reasons for low uptake in certain populations or communities (eg, anti-vaxxers [40]), which can inform the design of vaccine deployment strategies, including public messaging campaigns. The approach may help promote a learning public health policy cycle, where ideas can be tested against public attitudes before they are implemented, thereby maximizing their effectiveness and real-world applicability [41].

Conclusions

We advocate for future work on AI-enabled social media analysis of public attitudes in health care, which we believe has the potential to help facilitate the implementation of effective public health campaigns. Through this preliminary analysis, we show how such innovative methods may complement findings using conventional methods to monitor public sentiments (eg, surveys) while also providing greater spatiotemporal granularity.

Acknowledgments

This work is supported by the Scottish Government Chief Scientist Office under its COVID-19 priority research program (grant COV/NAP/20/07), and by BREATHE - The Health Data Research Hub for Respiratory Health [MC_PC_19004]. BREATHE is funded through the UK Research and Innovation Industrial Strategy Challenge Fund and delivered through Health Data Research, United Kingdom. AH is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) (grants EP/T021063/1 and EP/T024917/1). KC is supported by a Scottish Government Chief Scientist Office Research Grant exploring the potential of next-generation health information technology.

Conflicts of Interest

AS is a member of the Scottish Government’s Chief Medical Officer’s COVID-19 Advisory Group and the UK Government’s New and Emerging Respiratory Virus Threats (NERVTAG) Risk Stratification Subgroup. The other authors have no conflicts to declare. The views expressed are those of the authors and does not represent the views of the Scottish or UK governments.

Multimedia Appendix 1

Normalized confusion matrices for VADER- and TextBlob-based lexicon models, deep learning-based BERT model, and the hybrid rule-based ensemble approach. BERT: Bidirectional Encoder-Representations from Transformers; VADER: Valence Aware Dictionary and Sentiment Reasoner.

PNG File , 338 KB

References

  1. Digital contact tracing can slow or even stop coronavirus transmission and ease us out of lockdown. University of Oxford. 2020 Apr 16.   URL: https:/​/www.​research.ox.ac.uk/​Article/​2020-04-16-digital-contact-tracing-can-slow-or-even- stop-coronavirus-transmission-and-ease-us-out-of-lockdown [accessed 2020-11-09]
  2. Kretzschmar M, Rozhnova G, Bootsma M, van Boven M, van de Wijgert JHHM, Bonten M. Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study. Lancet Public Health 2020 Aug;5(8):e452-e459 [FREE Full text] [CrossRef] [Medline]
  3. Braithwaite I, Callender T, Bullock M, Aldridge RW. Automated and partly automated contact tracing: a systematic review to inform the control of COVID-19. Lancet Digit Health 2020 Nov;2(11):e607-e621 [FREE Full text] [CrossRef] [Medline]
  4. Kubba S. The Importance of Culture in Societal Responses to COVID-19. Harvard Political Review. 2020 Oct 14.   URL: https://harvardpolitics.com/culture-response-covid-19/ [accessed 2020-11-09]
  5. Fendos J. How surveillance technology powered South Korea's COVID-19 response. Brookings. 2020 Apr 29.   URL: https:/​/www.​brookings.edu/​techstream/​how-surveillance-technology-powered-south-koreas-covid-19-response/​ [accessed 2020-11-09]
  6. Redmiles EM. User Concerns & Tradeoffs in technology-facilitated contact tracing. Digit. Gov.: Res. Pract 2021 Jan 11;2(1):1-12 [FREE Full text] [CrossRef]
  7. Hargittai E, Redmiles E. Will Americans Be Willing to Install COVID-19 Tracking Apps? Scientific American. 2020 Apr 28.   URL: https://tinyurl.com/3u87bkjr [accessed 2021-05-04]
  8. Kaptchuk G, Hargittai E, Redmiles E. How good is good enough for COVID19 apps? The influence of benefits, accuracy, and privacy on willingness to adopt. arXiv Preprint posted online on May 9, 2020.
  9. Kent C. A comedy of errors: the UK’s contact-tracing apps. Medical Device Network. 2020 Oct 6.   URL: https://www.medicaldevice-network.com/features/uk-contact-tracing-app-problems/ [accessed 2021-03-16]
  10. Perez S. Report: WhatsApp has seen a 40% increase in usage due to COVID-19 pandemic. Tech Crunch. 2020 Mar 26.   URL: https:/​/techcrunch.​com/​2020/​03/​26/​report-whatsapp-has-seen-a-40-increase-in-usage-due-to-covid-19-pandemic/​ [accessed 2020-11-09]
  11. Al-Ghadir AI, Azmi AM, Hussain A. A novel approach to stance detection in social media tweets by fusing ranked lists and sentiments. Information Fusion 2021 Mar;67:29-40. [CrossRef]
  12. Altmann S, Milsom L, Zillessen H, Blasone R, Gerdon F, Bach R, et al. Acceptability of app-based contact tracing for COVID-19: Cross-country survey study. JMIR Mhealth Uhealth 2020 Aug 28;8(8):e19857 [FREE Full text] [CrossRef] [Medline]
  13. COVID-19 AI-powered dashboard for sentiment and opinion mining in social media platforms. COVID Tracker Cloud.: Edinburgh Napier University   URL: http://covidtracker.cloud [accessed 2020-11-09]
  14. Hussain A, Tahir A, Hussain Z, Sheikh Z, Gogate M, Dashtipour K, et al. Artificial intelligence-enabled analysis of UK and US public attitudes on Facebook and Twitter towards COVID-19 vaccinations. J Med Internet Res 2021 Apr 05;23(4):e26627 [FREE Full text] [CrossRef] [Medline]
  15. Yoo BH, Kim J, Lee B, Hoogenboom G, Kim KS. A surrogate weighted mean ensemble method to reduce the uncertainty at a regional scale for the calculation of potential evapotranspiration. Sci Rep 2020 Jan 21;10(1):870 [FREE Full text] [CrossRef] [Medline]
  16. Hutto CJ, Gilbert E. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In: Proceedings of the International AAAI Conference on Web and Social Media. 2014 May 16 Presented at: Eighth International AAAI Conference on Weblogs and Social Media; June 1-4, 2014; Ann Arbor, MI   URL: http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf
  17. Loria S, Keen P, Honnibal M, Yankovsky R, Karesh D, Dempsey E. TextBlob: simplified text processing. Secondary TextBlob: Simplified Text Processing. 2014.   URL: https://textblob.readthedocs.io/en/dev/ [accessed 2020-08-10]
  18. Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. : Association for Computational Linguistics; 2019 Presented at: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers); June 2019; Minneapolis, MN p. 4171-4186   URL: https://www.aclweb.org/anthology/N19-1423 [CrossRef]
  19. Edinburgh Napier University - covid-app: Data and code for using artificial intelligence-enabled analysis of social media data to understand public perceptions of COVID-19 contact tracing apps. GitLab.   URL: https://gitlab.com/edinburgh-napier-university/covid-app [accessed 2021-05-04]
  20. Survey on COVID-19 track and trace smartphone app for The Health Foundation. IPSOS Mori. 2020 Jun 3.   URL: https:/​/www.​ipsos.com/​ipsos-mori/​en-uk/​survey-covid-19-track-and-trace-smartphone-app-health-foundation [accessed 2020-12-01]
  21. Taylor E, Jackson J, Yesberg J, Pósch K. Coronavirus: survey reveals what the public wants from a contact-tracing app. The Conversation. 2020 May 15.   URL: https:/​/theconversation.​com/​coronavirus-survey-reveals-what-the-public-wants- from-a-contact-tracing-app-138574 [accessed 2020-12-01]
  22. Kennedy J. What can the UK learn from South Korea's response to covid-19? Centre for Health and the Public Interest (CHPI). 2020 Apr 7.   URL: https://chpi.org.uk/blog/what-can-the-uk-learn-from-south-koreas-response-to-covid-19/ [accessed 2020-11-09]
  23. Utomo D. TraceTogether: Things you might not know about Singapore's contact tracing app. TimeOut. 2020 May 12.   URL: https:/​/www.​timeout.com/​singapore/​news/​tracetogether-things-you-might-not-know-about-singapores-contact- tracing-app-051220 [accessed 2020-11-09]
  24. Kucharski A, Klepac P, Conlan A, Kissler S, Tang M, Fry H, CMMID COVID-19 working group. Effectiveness of isolation, testing, contact tracing, and physical distancing on reducing transmission of SARS-CoV-2 in different settings: a mathematical modelling study. Lancet Infect Dis 2020 Oct;20(10):1151-1160 [FREE Full text] [CrossRef] [Medline]
  25. Apple and Google partner on COVID-19 contact tracing technology. Apple Newsroom. 2020 Apr 10.   URL: https:/​/www.​apple.com/​newsroom/​2020/​04/​apple-and-google-partner-on-covid-19-contact-tracing-technology/​ [accessed 2020-11-09]
  26. NHS Test and Trace: how it works. Gov.UK.   URL: https://www.gov.uk/guidance/nhs-test-and-trace-how-it-works [accessed 2020-11-09]
  27. Coronavirus test, track and trace plan launched on Isle of Wight. Gov.UK. 2020 May 4.   URL: https://www.gov.uk/government/news/coronavirus-test-track-and-trace-plan-launched-on-isle-of-wight [accessed 2020-11-09]
  28. UK contact-tracing app could fall foul of privacy law, government told.   URL: https:/​/www.​theguardian.com/​world/​2020/​may/​07/​uk-coronavirus-contract-tracing-app-could-fall-foul-of-privacy-law-government-told [accessed 2020-11-09]
  29. Murphy S, Sabbagh D, Hern A. Piloted in May, ditched in June: the failure of England's Covid-19 app. The Guardian. 2020 Jun 18.   URL: https:/​/www.​theguardian.com/​world/​2020/​jun/​18/​piloted-in-may-ditched-in-june-the-failure-of- englands-covid-19-app [accessed 2020-11-09]
  30. Downey A. Covid-19: New trial for England’s revamped NHS contact-tracing app. Digital Health. 2020 Aug 13.   URL: https://www.digitalhealth.net/2020/08/covid-19-new-trial-nhs-contact-tracing-app/ [accessed 2020-11-09]
  31. Downey A. NHS Covid-19 contact-tracing app launched in England and Wales. Digital Health.   URL: https://www.digitalhealth.net/2020/09/nhs-covid-19-contact-tracing-app-launched-england-wales/ [accessed 2020-11-09]
  32. Manthorpe R. Coronavirus: NHS contact tracing app to stop sending 'false alarms' after update. Sky News. 2020 Oct 29.   URL: https:/​/news.​sky.com/​story/​coronavirus-nhs-contact-tracing-app-to-stop-sending-false-alarms-after-update-12117882 [accessed 2020-11-09]
  33. Blank G. The digital divide among Twitter users and its implications for social research. Social Science Computer Review 2016 Oct 12;35(6):679-697. [CrossRef]
  34. Smith A, Brenner J. Twitter Use 2012. Pew Research Center. 2012 May 31.   URL: https://www.pewresearch.org/internet/2012/05/31/twitter-use-2012/ [accessed 2021-05-04]
  35. Gore RJ, Diallo S, Padilla J. You are what you tweet: connecting the geographic variation in America's obesity rate to Twitter content. PLoS One 2015;10(9):e0133505 [FREE Full text] [CrossRef] [Medline]
  36. Padilla JJ, Kavak H, Lynch CJ, Gore RJ, Diallo SY. Temporal and spatiotemporal investigation of tourist attraction visit sentiment on Twitter. PLoS One 2018;13(6):e0198857 [FREE Full text] [CrossRef] [Medline]
  37. Kour H, Manhas J, Sharma V. Usage and implementation of neuro-fuzzy systems for classification and prediction in the diagnosis of different types of medical disorders: a decade review. Artif Intell Rev 2020 Feb 04;53(7):4651-4706 [FREE Full text] [CrossRef]
  38. Drake TM, Docherty AB, Weiser TG, Yule S, Sheikh A, Harrison EM. The effects of physical distancing on population mobility during the COVID-19 pandemic in the UK. The Lancet Digital Health 2020 Aug;2(8):e385-e387. [CrossRef]
  39. Brandt J, Buckingham K, Buntain C, Anderson W, Ray S, Pool J, et al. Identifying social media user demographics and topic diversity with computational social science: a case study of a major international policy forum. J Comput Soc Sc 2020 Jan 07;3(1):167-188. [CrossRef]
  40. Francia M, Gallinucci E, Golfarelli M. Social BI to understand the debate on vaccines on the Web and social media: unraveling the anti-, free, and pro-vax communities in Italy. Soc Netw Anal Min 2019 Aug 10;9:46. [CrossRef]
  41. Hussain A, Sheikh A. Opportunities for artificial intelligence–enabled social media analysis of public attitudes toward Covid-19 vaccines. NEJM Catal Innov Care Deliv 2021 Feb 05;2(1). [CrossRef]


AI: artificial intelligence
BERT: Bidirectional Encoder-Representations from Transformers
DL: deep learning
GDPR: General Data Protection Regulation
VADER: Valence Aware Dictionary and Sentiment Reasoner


Edited by C Basch; submitted 19.12.20; peer-reviewed by J Li, R Gore, S Lalmuanawma; comments to author 15.03.21; revised version received 29.03.21; accepted 17.04.21; published 17.05.21

Copyright

©Kathrin Cresswell, Ahsen Tahir, Zakariya Sheikh, Zain Hussain, Andrés Domínguez Hernández, Ewen Harrison, Robin Williams, Aziz Sheikh, Amir Hussain. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.05.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.