This paper is in the following e-collection/theme issue:

JMIR Theme Issue: COVID-19 Special Issue (2388) Outbreak and Pandemic Preparedness and Management (1673) Anxiety and Stress Disorders (1003) Surveillance Systems (427) JMH Theme Issue: COVID-19 and Mental Health: Impact and Interventions (372) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1482) Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (785) mHealth for Data Collection and Research (910)

Published on in Vol 23, No 2 (2021): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25118, first published .
Efficiency and Quality of Data Collection Among Public Mental Health Surveys Conducted During the COVID-19 Pandemic: Systematic Review

Efficiency and Quality of Data Collection Among Public Mental Health Surveys Conducted During the COVID-19 Pandemic: Systematic Review

Efficiency and Quality of Data Collection Among Public Mental Health Surveys Conducted During the COVID-19 Pandemic: Systematic Review

Authors of this article:

Yu-Hsuan Lin1, 2, 3, 4 Author Orcid Image ;   Chung-Yen Chen5, 6 Author Orcid Image ;   Shiow-Ing Wu1 Author Orcid Image
Article Authors Cited by (15) Tweetations (7) Metrics

    Review

    1Institute of Population Health Sciences, National Health Research Institutes, Miaoli County, Taiwan

    2Department of Psychiatry, National Taiwan University Hospital, Taipei, Taiwan

    3Department of Psychiatry, College of Medicine, National Taiwan University, Taipei, Taiwan

    4Institute of Health Behaviors and Community Sciences, College of Public Health, National Taiwan University, Taipei, Taiwan

    5Department of Environmental and Occupational Medicine, National Taiwan University Hospital, Taipei, Taiwan

    6Institute of Environmental and Occupational Health Sciences, College of Public Health, National Taiwan University, Taipei, Taiwan

    *these authors contributed equally

    Corresponding Author:

    Yu-Hsuan Lin, MD, PhD

    Institute of Population Health Sciences

    National Health Research Institutes

    35 Keyan Road, Zhunan

    Miaoli County, 35053

    Taiwan

    Phone: 886 37 206166 ext 36383

    Email: yuhsuanlin@nhri.edu.tw


    Background: The World Health Organization has recognized the importance of assessing population-level mental health during the COVID-19 pandemic. During a global crisis such as the COVID-19 pandemic, a timely surveillance method is urgently needed to track the impact on public mental health.

    Objective: This brief systematic review focused on the efficiency and quality of data collection of studies conducted during the COVID-19 pandemic.

    Methods: We searched the PubMed database using the following search strings: ((COVID-19) OR (SARS-CoV-2)) AND ((Mental health) OR (psychological) OR (psychiatry)). We screened the titles, abstracts, and texts of the published papers to exclude irrelevant studies. We used the Newcastle-Ottawa Scale to evaluate the quality of each research paper.

    Results: Our search yielded 37 relevant mental health surveys of the general public that were conducted during the COVID-19 pandemic, as of July 10, 2020. All these public mental health surveys were cross-sectional in design, and the journals efficiently made these articles available online in an average of 18.7 (range 1-64) days from the date they were received. The average duration of recruitment periods was 9.2 (range 2-35) days, and the average sample size was 5137 (range 100-56,679). However, 73% (27/37) of the selected studies had Newcastle-Ottawa Scale scores of <3 points, which suggests that these studies are of very low quality for inclusion in a meta-analysis.

    Conclusions: The studies examined in this systematic review used an efficient data collection method, but there was a high risk of bias, in general, among the existing public mental health surveys. Therefore, following recommendations to avoid selection bias, or employing novel methodologies considering both a longitudinal design and high temporal resolution, would help provide a strong basis for the formation of national mental health policies.

    J Med Internet Res 2021;23(2):e25118

    doi:10.2196/25118

    Keywords

    COVID-19; mental health; Newcastle-Ottawa Scale; review; data collection; survey; surveillance; literature; research 


    The World Health Organization has recognized the importance of assessing population-level mental health during the COVID-19 pandemic. More than 463 articles on the mental health impact of COVID-19 have been published in 2020, and several more mental health surveys are underway [1]. Most of these studies evaluated the psychiatric symptoms among the general public and health care workers, and some others evaluated patients with COVID-19 or those with psychiatric disorders [2]. In a global crisis such as the COVID-19 pandemic, time-sensitive policy decision-making underscores the importance of fostering an agile empirical approach that can monitor population-level mental health in a timely manner. Any change in the mental health impact of COVID-19 is likely to be dynamic. For example, the 20% decrease in suicide rate in Japan observed during the early stage of the pandemic seemed to have reversed in August 2020, when a 7.7% rise was reported [3]. Therefore, a timely surveillance method is urgently needed to track the impact of COVID-19 on public mental health. This brief systematic review focused on studies on the general public conducted during the COVID-19 pandemic and analyzed the efficiency and quality of data collection in the existing literature.


    We used the following search strings to select relevant articles in the PubMed database: ((COVID-19) OR (SARS-CoV-2)) AND ((Mental health) OR (psychological) OR (psychiatry)). We screened the titles, abstracts, and texts of the published articles to exclude irrelevant studies. We also excluded articles published on preprint platforms such as bioRxiv, because we were unable to analyze the time between the receipt of the articles and their availability online. A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram detailing the study retrieval process is shown in Figure 1.

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of article selection for the systematic review.
    View this figure

    We used the Newcastle-Ottawa Scale to evaluate the quality of each research article. The Newcastle-Ottawa Scale includes four criteria for selection bias, with scores ranging between 0 and 5; scores <3 points are regarded as high risk for selection bias in a cross-sectional study [1,4]. The four quality assessment criteria were as follows: sample representativeness (1 point), predetermined and satisfactory sample size (1 point), comparability between respondents and nonrespondents (1 point), and ascertainment of the measurement tool (2 points for the validated measurement tool; 1 point for the nonvalidated measurement tool, but the tool is available or described).


    In all, we found 37 relevant mental health surveys of the general public during the COVID-19 pandemic by searching the PubMed database, as of July 10, 2020. All these public mental health survey studies examined had a cross-sectional study design. The journals efficiently made these articles available online in an average of 18.7 (range 1-64) days from the date they were received. The average duration of recruitment period of these studies was 9.2 (range 2-35) days, and the average sample size was 5137 (range 100-56,679). The correlation coefficient (r=0.164, P=.40) between these durations and the corresponding sample sizes suggested that the samples in the studies selected from the existing literature were collected in an efficient manner; even larger sample sizes did not require proportionately longer to collect the data. Although there was a potential publication bias in the recruitment or data collection period, it is noteworthy that approximately 92% (34/37) of the studies reported the use of a web-based or an app-based survey, thereby promoting the efficiency of large-scale data collection, especially during the COVID-19 pandemic.

    Unlike most studies on health care workers, which have a low risk of selection bias [1], our review showed that 73% (27/37) of the selected studies evaluating the mental health impact of COVID-19 on the general public had Newcastle-Ottawa Scale scores of <3 points, which suggests that these studies were of very low quality to be considered for inclusion in a meta-analysis. All of the selected studies met the criterion of validated measurement tools, but only 8 studies met the criterion of sample representativeness, 5 met the criterion of justified sample size, and 1 met the criterion of comparability between respondents and nonrespondents (Table 1).

    Table 1. Public mental health surveys conducted during the COVID-19 pandemic.
    PublicationSample size (n)Recruitment period (2020)Duration of recruitment period (days)Duration between article submission and online publication (days)Newcastle-Ottawa Quality Assessment Scale scores





    		Sample representativenessSample sizeNonrespondentsAscertainment of exposureTotal score
    Ahmed et al [5]1074—a—3200022
    Bryan et al [6]10,625March 18 to April 418—11024
    Cao et al [7]7143——610124
    Castelli et al [8]1321March 19 to April 518—00022
    Choi et al [9]500April 24 to May 3101811024
    Cortes-Alvarez et al [10]1105March 30 to April 57—00022
    Fitzpatrick et al [11]10,368March 23-297—10023
    Gao et al [12]4827January 31 to February 234300022
    Huang and Zhao [13]7236February 3-17152300022
    Lei et al [14]1593February 4-1072500022
    Li et al [15]4607February 2-98—00022
    Liang et al [16]584January 30 to unknown——00022
    Liu et al [17]285January 30 to February 810100022
    Ma et al [18]728April 9-3022—00022
    Mazza et al [19]2766March 18-2251900022
    McGinty et al [20]1468April 7-137—10023
    Moccia et al [21]500April 10-134511024
    Moghanibashi-Mansourieh [22]10,754March 1-992200022
    Naser et al [23]4126March 22-2876401023
    Ozdin and Bayrak Özdin [24]343April 14-163—01023
    Qiu et al [25]52,730January 31 to February 1011700022
    Rehman et al [26]403April 3-644900022
    Roy et al [27]662March 22-243700011
    Smith et al [28]932March 17 to unknown—2500022
    Shi et al [29]56,679February 28 to March 1113—00022
    Sonderskov et al [30]2458March 31 to April 671310023
    Tan et al [31]673February 24-252600022
    Tian et al [32]1060January 31 to February 231500022
    Ustun [33]1115March 23 to April 312—00022
    Verma and Mishra [34]354April 4-1411—00022
    Wang et al [35]








    		1210January 31 to February 23400022

    		861February 28 to March 1
    Xiao et al [36]170——1100022
    Yuan et al [37]939——2200022
    Yuan et al [38]100———00022
    Zhang and Ma [39]263January 28 to February 592000022
    Zhang et al [40]369February 20-2121010023
    Zhu et al [41]2279February 12 to March 1735100022

    a‑Data not available.


    The general public surveys examined in this review most commonly used the web-based format to promote efficiency of large-scale data collection, especially in the countries with significant COVID-19 outbreaks that had introduced social distancing or lockdown measures. However, the web-based survey approach has an inherent methodological limitation to show the comparability between respondents and nonrespondents. The sample representativeness of the target population is critical for selection bias. Web-based survey platforms using sampling methods could avoid such selection bias. Two published surveys [6,11] used Qualtrics Panels, a web-based survey platform that uses quota sampling methods to identify participants who meet a given study’s eligibility criteria, with a target to recruit a sample that is demographically similar to the 2010 United States census distributions for age, sex, and race or ethnicity (with a ±10% variation). Qualtrics Panels maintains a database of several million US residents who have volunteered to participate in periodic survey-based research. Another study [20] used AmeriSpeak, a probability-based research panel designed to be representative of the adult US population. AmeriSpeak data is sourced from National Opinion Research Center’s area probability sample and a US postal service address–based sample covering approximately 97% of all US households. In this national survey study [20], a higher proportion (13.6%) of the US adult sample reported symptoms of serious psychological distress in 2020 than in 2018 (3.9%).

    The limitations of cross-sectional studies should also be noted when interpreting the role of the COVID-19 outbreak on the mental health of the general public. These epidemiological studies identified general risk factors during the COVID-19 outbreak, rather than the specific impacts of COVID-19 on the mental health of the public. For example, a cross-sectional study of 4827 participants [12] showed that lower education levels and higher social media exposure result in a high risk of individuals developing depressive symptoms, but these findings did not suggest that COVID-19 had an impact on the public’s mental health. There was only one longitudinal study [35], which conducted two-wave surveys during the initial COVID-19 outbreak (ie, January 31 to February 2, 2020) and during the peak 4 weeks later (ie, February 28 to March 1). Although this study recruited 1210 and 861 participants within 3 days by using a web-based survey approach, no significant longitudinal changes in stress, anxiety, or depression levels were observed. These findings might result from a lack of baseline data, few follow-ups with the same respondents (333/1210, 27.52%), or insufficient temporal resolution to detect mental health changes. Only one study compared the prevalence of symptoms of serious psychological distress before and during the COVID-19 pandemic by using an identical measure. This survey, which used AmeriSpeak for sample recruitment, showed that the prevalence of serious psychological distress symptoms among US adults was higher during the COVID-19 pandemic in 2020 than that reported in the 2018 National Health Interview Survey [20]. However, this study did not monitor the changes in population-level mental health through the unfolding pandemic. Timely mental health survey studies with a high temporal resolution, such as internet search data (eg, Google Trends) [42] and sentiment analysis on social media (eg, Weibo posts) [43], are warranted in the future to monitor the long-term impacts of the fast-moving COVID-19 outbreak.

    In conclusion, this systematic review found that the data collection methods used in the public mental health surveys in the existing literature were efficient but generally had a high risk of bias. Therefore, following recommendations to avoid selection bias, or to apply novel methodologies considering both longitudinal design and high temporal resolution, would help provide a strong basis for the formation of national mental health policies.

    Acknowledgments

    The authors thank Hsiao-Han Lin and Ting-Chen Hsu for their excellent technical assistance during this study. This study was supported by grants from National Health Research Institute in Taiwan (10A1-PHPP22-014; 10A1-PHPP16-014).

    Authors' Contributions

    YHL, CYC, and SIW conceptualized the study, and analyzed and interpreted the data. YHL and CYC drafted the manuscript. All authors have read and approved the final version of the manuscript.

    Conflicts of Interest

    None declared.

    1. Pappa S, Ntella V, Giannakas T, Giannakoulis VG, Papoutsi E, Katsaounou P. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: A systematic review and meta-analysis. Brain, Behavior, and Immunity 2020 Aug;88:901-907. [CrossRef]
    2. Vindegaard N, Benros ME. COVID-19 pandemic and mental health consequences: Systematic review of the current evidence. Brain, Behavior, and Immunity 2020 Oct;89:531-542. [CrossRef]
    3. Ueda M, Nordström R, Matsubayashi T. Suicide and mental health during the COVID-19 pandemic in Japan. medRxiv. Preprint posted online on December 20, 2020. [CrossRef]
    4. Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol 2010 Jul 22;25(9):603-605. [CrossRef] [Medline]
    5. Ahmed MZ, Ahmed O, Aibao Z, Hanbin S, Siyu L, Ahmad A. Epidemic of COVID-19 in China and associated psychological problems. Asian J Psychiatr 2020 Jun;51:102092 [FREE Full text] [CrossRef] [Medline]
    6. Bryan CJ, Bryan AO, Baker JC. Associations among state-level physical distancing measures and suicidal thoughts and behaviors among U.S. adults during the early COVID-19 pandemic. Suicide Life Threat Behav 2020 Dec;50(6):1223-1229 [FREE Full text] [CrossRef] [Medline]
    7. Cao W, Fang Z, Hou G, Han M, Xu X, Dong J, et al. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res 2020 May;287:112934 [FREE Full text] [CrossRef] [Medline]
    8. Castelli L, Di Tella M, Benfante A, Romeo A. The spread of COVID-19 in the Italian population: anxiety, depression, and post-traumatic stress symptoms. Can J Psychiatry 2020 Oct;65(10):731-732. [CrossRef] [Medline]
    9. Choi EPH, Hui BPH, Wan EYF. Depression and anxiety in Hong Kong during COVID-19. Int J Environ Res Public Health 2020 May 25;17(10):3740 [FREE Full text] [CrossRef] [Medline]
    10. Cortés-Álvarez NY, Piñeiro-Lamas R, Vuelvas-Olmos CR. Psychological effects and associated factors of COVID-19 in a Mexican sample. Disaster Med Public Health Prep 2020 Jun;14(3):413-424 [FREE Full text] [CrossRef] [Medline]
    11. Fitzpatrick KM, Harris C, Drawve G. How bad is it? Suicidality in the middle of the COVID-19 pandemic. Suicide Life Threat Behav 2020 Dec;50(6):1241-1249 [FREE Full text] [CrossRef] [Medline]
    12. Gao J, Zheng P, Jia Y, Chen H, Mao Y, Chen S, et al. Mental health problems and social media exposure during COVID-19 outbreak. PLoS One 2020;15(4):e0231924 [FREE Full text] [CrossRef] [Medline]
    13. Huang Y, Zhao N. Generalized anxiety disorder, depressive symptoms and sleep quality during COVID-19 outbreak in China: a web-based cross-sectional survey. Psychiatry Res 2020 Jun;288:112954 [FREE Full text] [CrossRef] [Medline]
    14. Lei L, Huang X, Zhang S, Yang J, Yang L, Xu M. Comparison of prevalence and associated factors of anxiety and depression among people affected by versus people unaffected by quarantine during the COVID-19 epidemic in Southwestern China. Med Sci Monit 2020 Apr 20;26:e924609. [CrossRef]
    15. Li G, Miao J, Wang H, Xu S, Sun W, Fan Y, et al. Psychological impact on women health workers involved in COVID-19 outbreak in Wuhan: a cross-sectional study. J Neurol Neurosurg Psychiatry 2020 Aug;91(8):895-897. [CrossRef] [Medline]
    16. Liang L, Ren H, Cao R, Hu Y, Qin Z, Li C, et al. The effect of COVID-19 on youth mental health. Psychiatr Q 2020 Sep;91(3):841-852 [FREE Full text] [CrossRef] [Medline]
    17. Liu N, Zhang F, Wei C, Jia Y, Shang Z, Sun L, et al. Prevalence and predictors of PTSS during COVID-19 outbreak in China hardest-hit areas: Gender differences matter. Psychiatry Res 2020 May;287:112921 [FREE Full text] [CrossRef] [Medline]
    18. Ma ZF, Zhang Y, Luo X, Li X, Li Y, Liu S, et al. Increased stressful impact among general population in mainland China amid the COVID-19 pandemic: A nationwide cross-sectional study conducted after Wuhan city's travel ban was lifted. Int J Soc Psychiatry 2020 Dec;66(8):770-779 [FREE Full text] [CrossRef] [Medline]
    19. Mazza C, Ricci E, Biondi S, Colasanti M, Ferracuti S, Napoli C, et al. A nationwide survey of psychological distress among Italian people during the COVID-19 pandemic: immediate psychological responses and associated factors. Int J Environ Res Public Health 2020 May 02;17(9):3165 [FREE Full text] [CrossRef] [Medline]
    20. McGinty EE, Presskreischer R, Han H, Barry CL. Psychological distress and loneliness reported by US adults in 2018 and April 2020. JAMA 2020 Jul 07;324(1):93. [CrossRef]
    21. Moccia L, Janiri D, Pepe M, Dattoli L, Molinaro M, De Martin V, et al. Affective temperament, attachment style, and the psychological impact of the COVID-19 outbreak: an early report on the Italian general population. Brain Behav Immun 2020 Jul;87:75-79 [FREE Full text] [CrossRef] [Medline]
    22. Moghanibashi-Mansourieh A. Assessing the anxiety level of Iranian general population during COVID-19 outbreak. Asian J Psychiatr 2020 Jun;51:102076 [FREE Full text] [CrossRef] [Medline]
    23. Naser AY, Dahmash EZ, Al-Rousan R, Alwafi H, Alrawashdeh HM, Ghoul I, et al. Mental health status of the general population, healthcare professionals, and university students during 2019 coronavirus disease outbreak in Jordan: A cross-sectional study. Brain Behav 2020 Aug;10(8):e01730 [FREE Full text] [CrossRef] [Medline]
    24. Özdin S, Bayrak Özdin Ş. Levels and predictors of anxiety, depression and health anxiety during COVID-19 pandemic in Turkish society: The importance of gender. Int J Soc Psychiatry 2020 Aug;66(5):504-511 [FREE Full text] [CrossRef] [Medline]
    25. Qiu J, Shen B, Zhao M, Wang Z, Xie B, Xu Y. A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: implications and policy recommendations. Gen Psychiatr 2020;33(2):e100213 [FREE Full text] [CrossRef] [Medline]
    26. Rehman U, Shahnawaz MG, Khan NH, Kharshiing KD, Khursheed M, Gupta K, et al. Depression, anxiety and stress among Indians in times of Covid-19 lockdown. Community Ment Health J 2021 Jan;57(1):42-48 [FREE Full text] [CrossRef] [Medline]
    27. Roy D, Tripathy S, Kar SK, Sharma N, Verma SK, Kaushal V. Asian J Psychiatr 2020 Jun;51:102083 [FREE Full text] [CrossRef] [Medline]
    28. Smith L, Jacob L, Yakkundi A, McDermott D, Armstrong NC, Barnett Y, et al. Correlates of symptoms of anxiety and depression and mental wellbeing associated with COVID-19: a cross-sectional study of UK-based respondents. Psychiatry Res 2020 Sep;291:113138 [FREE Full text] [CrossRef] [Medline]
    29. Shi L, Lu Z, Que J, Huang X, Liu L, Ran M, et al. Prevalence of and risk factors associated with mental health symptoms among the general population in China During the coronavirus disease 2019 pandemic. JAMA Netw Open 2020 Jul 01;3(7):e2014053 [FREE Full text] [CrossRef] [Medline]
    30. Sønderskov KM, Dinesen PT, Santini ZI, Østergaard SD. The depressive state of Denmark during the COVID-19 pandemic. Acta Neuropsychiatr 2020 Aug;32(4):226-228 [FREE Full text] [CrossRef] [Medline]
    31. Tan W, Hao F, McIntyre RS, Jiang L, Jiang X, Zhang L, et al. Is returning to work during the COVID-19 pandemic stressful? A study on immediate mental health status and psychoneuroimmunity prevention measures of Chinese workforce. Brain Behav Immun 2020 Jul;87:84-92 [FREE Full text] [CrossRef] [Medline]
    32. Tian F, Li H, Tian S, Yang J, Shao J, Tian C. Psychological symptoms of ordinary Chinese citizens based on SCL-90 during the level I emergency response to COVID-19. Psychiatry Res 2020 Jun;288:112992 [FREE Full text] [CrossRef] [Medline]
    33. Ustun G. Determining depression and related factors in a society affected by COVID-19 pandemic. Int J Soc Psychiatry 2020 Jun 30:20764020938807 [FREE Full text] [CrossRef] [Medline]
    34. Verma S, Mishra A. Depression, anxiety, and stress and socio-demographic correlates among general Indian public during COVID-19. Int J Soc Psychiatry 2020 Dec;66(8):756-762. [CrossRef] [Medline]
    35. Wang C, Pan R, Wan X, Tan Y, Xu L, McIntyre RS, et al. A longitudinal study on the mental health of general population during the COVID-19 epidemic in China. Brain Behav Immun 2020 Jul;87:40-48 [FREE Full text] [CrossRef] [Medline]
    36. Xiao H, Zhang Y, Kong D, Li S, Yang N. Social capital and sleep quality in individuals who self-isolated for 14 days during the coronavirus disease 2019 (COVID-19) outbreak in January 2020 in China. Med Sci Monit 2020 Mar 20;26:e923921 [FREE Full text] [CrossRef] [Medline]
    37. Yuan S, Liao Z, Huang H, Jiang B, Zhang X, Wang Y, et al. Comparison of the indicators of psychological stress in the population of Hubei Province and non-endemic provinces in China during two weeks during the coronavirus disease 2019 (COVID-19) outbreak in February 2020. Med Sci Monit 2020 Apr 15;26:e923767 [FREE Full text] [CrossRef] [Medline]
    38. Yuan R, Xu Q, Xia C, Lou C, Xie Z, Ge Q, et al. Psychological status of parents of hospitalized children during the COVID-19 epidemic in China. Psychiatry Res 2020 Jun;288:112953 [FREE Full text] [CrossRef] [Medline]
    39. Zhang Y, Ma ZF. Impact of the COVID-19 pandemic on mental health and quality of life among local residents in Liaoning Province, China: a cross-sectional study. Int J Environ Res Public Health 2020 Mar 31;17(7):2381 [FREE Full text] [CrossRef] [Medline]
    40. Zhang SX, Wang Y, Rauch A, Wei F. Unprecedented disruption of lives and work: health, distress and life satisfaction of working adults in China one month into the COVID-19 outbreak. Psychiatry Res 2020 Jun;288:112958 [FREE Full text] [CrossRef] [Medline]
    41. Zhu S, Wu Y, Zhu C, Hong W, Yu Z, Chen Z, et al. The immediate mental health impacts of the COVID-19 pandemic among people with or without quarantine managements. Brain Behav Immun 2020 Jul;87:56-58 [FREE Full text] [CrossRef] [Medline]
    42. Lin Y, Liu C, Chiu Y. Google searches for the keywords of "wash hands" predict the speed of national spread of COVID-19 outbreak among 21 countries. Brain Behav Immun 2020 Jul;87:30-32 [FREE Full text] [CrossRef] [Medline]
    43. Li S, Wang Y, Xue J, Zhao N, Zhu T. The impact of COVID-19 epidemic declaration on psychological consequences: a study on active Weibo users. Int J Environ Res Public Health 2020 Mar 19;17(6):2032 [FREE Full text] [CrossRef] [Medline]

    Edited by G Eysenbach; submitted 19.10.20; peer-reviewed by LA Lee, P Banik; comments to author 23.12.20; revised version received 29.12.20; accepted 16.01.21; published 10.02.21

    Copyright

    ©Yu-Hsuan Lin, Chung-Yen Chen, Shiow-Ing Wu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 10.02.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.