This paper is in the following e-collection/theme issue:

JMIR Theme Issue: COVID-19 Special Issue (2388) eHealth Literacy / Digital Literacy (351) Hygiene and Infection Prevention (297) Infectious Diseases (non-STD/STI) (1478) Outbreak and Pandemic Preparedness and Management (1673) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1481)

Published on in Vol 22, No 11 (2020): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/22894, first published .
Health Literacy, eHealth Literacy, Adherence to Infection Prevention and Control Procedures, Lifestyle Changes, and Suspected COVID-19 Symptoms Among Health Care Workers During Lockdown: Online Survey

Health Literacy, eHealth Literacy, Adherence to Infection Prevention and Control Procedures, Lifestyle Changes, and Suspected COVID-19 Symptoms Among Health Care Workers During Lockdown: Online Survey

Health Literacy, eHealth Literacy, Adherence to Infection Prevention and Control Procedures, Lifestyle Changes, and Suspected COVID-19 Symptoms Among Health Care Workers During Lockdown: Online Survey

Authors of this article:

Binh N Do1, 2, 3 Author Orcid Image ;   Tien V Tran1, 2, 3 Author Orcid Image ;   Dung T Phan4, 5 Author Orcid Image ;   Hoang C Nguyen6, 7 Author Orcid Image ;   Thao T P Nguyen8, 9 Author Orcid Image ;   Huu C Nguyen10, 11 Author Orcid Image ;   Tung H Ha12 Author Orcid Image ;   Hung K Dao13 Author Orcid Image ;   Manh V Trinh14 Author Orcid Image ;   Thinh V Do15 Author Orcid Image ;   Hung Q Nguyen16 Author Orcid Image ;   Tam T Vo17 Author Orcid Image ;   Nhan P T Nguyen18 Author Orcid Image ;   Cuong Q Tran19, 20 Author Orcid Image ;   Khanh V Tran21 Author Orcid Image ;   Trang T Duong22 Author Orcid Image ;   Hai X Pham23 Author Orcid Image ;   Lam V Nguyen24, 25 Author Orcid Image ;   Kien T Nguyen26 Author Orcid Image ;   Peter W S Chang27, 28 Author Orcid Image ;   Tuyen Van Duong29 Author Orcid Image
Article Authors Cited by (73) Tweetations (13) Metrics

    Original Paper

    1Department of Infectious Diseases, Vietnam Military Medical University, Hanoi, Vietnam

    2Division of Military Science, Military Hospital 103, Hanoi, Vietnam

    3Director Office, Military Hospital 103, Hanoi, Vietnam

    4Faculty of Nursing, Hanoi University of Business and Technology, Hanoi, Vietnam

    5Nursing Office, Viet Duc University Hospital, Hanoi, Vietnam

    6Director Office, Thai Nguyen National Hospital, Thai Nguyen, Vietnam

    7President Office, Thai Nguyen University of Medicine and Pharmacy, Thai Nguyen, Vietnam

    8Health Management Training Institute, Hue University of Medicine and Pharmacy, Thua Thien Hue, Vietnam

    9Department of Health Economics, Corvinus University of Budapest, Budapest, Hungary

    10Department of Thoracic and Cardiovascular Surgery, E Hospital, Hanoi, Vietnam

    11Director Office, E Hospital, Hanoi, Vietnam

    12Director Office, General Hospital of Agricultural, Hanoi, Vietnam

    13Director Office, Bac Ninh Obstetrics and Pediatrics Hospital, Bac Ninh, Vietnam

    14Director Office, Quang Ninh General Hospital, Quang Ninh, Vietnam

    15Director Office, Bai Chay Hospital, Quang Ninh, Vietnam

    16Director Office, Quang Ninh Obstetrics and Pediatrics Hospital, Quang Ninh, Vietnam

    17Director Office, Trieu Phong District Health Center, Quang Tri, Vietnam

    18General Planning Department, Da Nang Oncology Hospital, Da Nang, Vietnam

    19Director Office, Thu Duc District Health Center, Ho Chi Minh, Vietnam

    20Faculty of Health, Mekong University, Vinh Long, Vietnam

    21Director Office, Hospital District 2, Ho Chi Minh, Vietnam

    22Nursing Office, Tan Phu District Hospital, Ho Chi Minh, Vietnam

    23Director Office, District 9 Health Center, Ho Chi Minh, Vietnam

    24President Office, Can Tho University of Medicine and Pharmacy, Can Tho, Vietnam

    25Aesthetic Plastic Surgery & Skin Care Center, Can Tho University of Medicine and Pharmacy Hospital, Can Tho, Vietnam

    26Department of Health Education, Faculty of Social Sciences, Behavior and Health Education, Hanoi University of Public Health, Hanoi, Vietnam

    27School of Medicine, Chung Shan Medical University, Taichung, Taiwan

    28Department of Public Health and Community Medicine, Tufts University School Of Medicine, Boston, MA, United States

    29School of Nutrition and Health Sciences, Taipei Medical University, Taipei, Taiwan

    *these authors contributed equally

    Corresponding Author:

    Tuyen Van Duong, BNSc, MNSc, PhD

    School of Nutrition and Health Sciences

    Taipei Medical University

    250 Wuxing St

    Taipei, 11031

    Taiwan

    Phone: 886 2 2736 1661 ext 6545

    Email: tvduong@tmu.edu.tw


    Background: The COVID-19 pandemic has imposed a heavy burden on health care systems and governments. Health literacy (HL) and eHealth literacy (as measured by the eHealth Literacy Scale [eHEALS]) are recognized as strategic public health elements but they have been underestimated during the pandemic. HL, eHEALS score, practices, lifestyles, and the health status of health care workers (HCWs) play crucial roles in containing the COVID-19 pandemic.

    Objective: The aim of this study is to evaluate the psychometric properties of the eHEALS and examine associations of HL and eHEALS scores with adherence to infection prevention and control (IPC) procedures, lifestyle changes, and suspected COVID-19 symptoms among HCWs during lockdown.

    Methods: We conducted an online survey of 5209 HCWs from 15 hospitals and health centers across Vietnam from April 6 to April 19, 2020. Participants answered questions related to sociodemographics, HL, eHEALS, adherence to IPC procedures, behavior changes in eating, smoking, drinking, and physical activity, and suspected COVID-19 symptoms. Principal component analysis, correlation analysis, and bivariate and multivariate linear and logistic regression models were used to validate the eHEALS and examine associations.

    Results: The eHEALS had a satisfactory construct validity with 8 items highly loaded on one component, with factor loadings ranked from 0.78 to 0.92 explaining 76.34% of variance; satisfactory criterion validity as correlated with HL (ρ=0.42); satisfactory convergent validity with high item-scale correlations (ρ=0.80-0.84); and high internal consistency (Cronbach α=.95). HL and eHEALS scores were significantly higher in men (unstandardized coefficient [B]=1.01, 95% CI 0.57-1.45, P<.001; B=0.72, 95% CI 0.43-1.00, P<.001), those with a better ability to pay for medication (B=1.65, 95% CI 1.25-2.05, P<.001; B=0.60, 95% CI 0.34-0.86, P<.001), doctors (B=1.29, 95% CI 0.73-1.84, P<.001; B 0.56, 95% CI 0.20-0.93, P=.003), and those with epidemic containment experience (B=1.96, 95% CI 1.56-2.37, P<.001; B=0.64, 95% CI 0.38-0.91, P<.001), as compared to their counterparts, respectively. HCWs with higher HL or eHEALS scores had better adherence to IPC procedures (B=0.13, 95% CI 0.10-0.15, P<.001; B=0.22, 95% CI 0.19-0.26, P<.001), had a higher likelihood of healthy eating (odds ratio [OR] 1.04, 95% CI 1.01-1.06, P=.001; OR 1.04, 95% CI 1.02-1.07, P=.002), were more physically active (OR 1.03, 95% CI 1.02-1.03, P<.001; OR 1.04, 95% CI 1.03-1.05, P<.001), and had a lower likelihood of suspected COVID-19 symptoms (OR 0.97, 95% CI 0.96-0.98, P<.001; OR 0.96, 95% CI 0.95-0.98, P<.001), respectively.

    Conclusions: The eHEALS is a valid and reliable survey tool. Gender, ability to pay for medication, profession, and epidemic containment experience were independent predictors of HL and eHEALS scores. HCWs with higher HL or eHEALS scores had better adherence to IPC procedures, healthier lifestyles, and a lower likelihood of suspected COVID-19 symptoms. Efforts to improve HCWs’ HL and eHEALS scores can help to contain the COVID-19 pandemic and minimize its consequences.

    J Med Internet Res 2020;22(11):e22894

    doi:10.2196/22894

    Keywords

    COVID-19; health literacy; eHealth literacy; health care workers; personal protective equipment; handwashing; masks; disposing; lifestyle; Vietnam; eHealth; adherence; infection prevention; control 


    COVID-19, the disease caused by SARS-CoV-2, has created unprecedented challenges worldwide [1-6], with significant socioeconomic burdens [7], morbidity, and mortality [8,9]. Multidisciplinary and multidimensional approaches are required to contain the pandemic [10-12]. Social and behavioral changes [13,14] and improving health literacy (HL) [15] and eHealth literacy (as measured by the eHealth Literacy Scale [eHEALS]) [16] are highly recommended to control this global health crisis while effective treatments and vaccines are unavailable.

    Lockdown measures were applied in many countries, including Vietnam [17]. Lockdown is a necessary public health approach to contain COVID-19. During the lockdown period, health care workers (HCWs) still needed to work and provide health services to patients, which made them more vulnerable to COVID-19 infection. In addition, the pandemic itself and social distancing measures have had negative impacts on psychological health [18,19] and health-related behavior (eg, unhealthy eating habits and behaviors, less physical activity, and more smoking [20-23] and drinking to cope amid the COVID-19 pandemic [24]).

    Online consultations from hospitals and health care centers were found to be a safe and effective way to reduce the negative effects of the pandemic [25]. In addition, social media can provide insights for effective communication among public health authorities [26], researchers [27], and the public [28-30]. However, social media spreads panic and anxiety among the public [31]. Disinformation and misinformation have been raised as highly concerning issues for the public [32-34]. HL plays an important role in evaluating online health information [35]. HL is critical for people who interact with the digital world, with its diverse information and sources [36], especially during the COVID-19 pandemic. HCWs play an important role in supporting the public to combat misinformation and disinformation [28,37]. The HCWs’ consultations cannot be altered by social media networks [37,38]. Continuous training and education have been recognized as effective approaches to improving HCWs’ HL, further improving health care delivery [39,40], communication [39,41,42], shared decision making [43], and patient health outcomes [40]. Furthermore, raising awareness of behavioral pitfalls could support appropriate behavioral changes and containment of the crisis [44]. HL and eHealth literacy are more important than ever due to the COVID-19 pandemic [16,45]. However, these issues have been underestimated during the pandemic [15].

    We evaluated the psychometric properties of the eHEALS and examined the predictors of HL and eHEALS scores. We also examined the associations between HL and eHEALS scores with adherence to infection prevention and control (IPC) measures, lifestyle changes, and suspected COVID-19 symptoms among HCWs during the lockdown period in Vietnam.


    Study Design and Settings

    A cross-sectional study was conducted with HCWs April 6-19, 2020, using online-based questionnaires (Text 1 in Multimedia Appendix 1). The HCWs were recruited from 12 hospitals and 3 health centers across Vietnam, including 8 hospitals in the nation’s north, 1 hospital and 1 health center in the central region, and 3 hospitals and 2 health centers in the country’s south.

    Study Participants and Data Collection

    No HCWs (doctors and nurses) in our study had provided any direct care or had contact with patients with COVID-19. A total sample of 5209 HCWs (out of 11,517 possible participants) completed an online survey. The studied and possible participants from public hospitals and health centers are presented in Table 1.

    Table 1. Participants from the studied hospitals and health centers by geographic location.
    Geographic location and hospital/health centerPossible participantsStudied participants
    North

    		Ha Noi city


    		Military Hospital 1031660177


    		E hospital1125335


    		General Hospital of Agricultural555424

    		Thai Nguyen province


    		Thai Nguyen National Hospital1186988

    		Bac Ninh city


    		Bac Ninh Obstetrics and Pediatrics Hospital391364

    		Quang Ninh province


    		Quang Ninh General Hospital922675


    		Bai Chay Hospital819476


    		Quang Ninh Obstetrics and Pediatrics Hospital478290
    Center

    		Quang Tri province


    		Trieu Phong District Health Center271203

    		Da Nang city


    		Da Nang Oncology Hospital555134
    South

    		Ho Chi Minh city


    		Tan Phu District Hospital530241


    		Hospital District 2812318


    		District 9 Health Center170102


    		Thu Duc District Health Center302291

    		Can Tho city


    		Can Tho University of Medicine and Pharmacy Hospital424191
    Total11,5175209

    Vietnam applied a nationwide lockdown measure April 1-15, 2020 [17], which was extended to April 22, 2020 [46,47]. In this study, HCWs took the online survey during the lockdown period. We used Google Forms to design and conduct the survey. The online survey links were sent to HCWs by researchers via email, Messenger, or Zalo. QR codes were also displayed in different departments of hospitals and health centers. It took about 15 minutes to complete the questionnaire. All survey questions were mandatory; therefore, there is no missing data in our study. All responses were exported to Google Sheets and saved on Google Drive. Finally, the data was coded, cleaned, and analyzed confidentially by researchers.

    Measures

    Sociodemographics

    HCWs reported their age (21-40 years versus 41-60 years), gender (woman versus man), marital status (never married versus ever married), ability to pay for medication (very or fairly difficult versus very or fairly easy), social status (low versus middle to high), profession (doctor, nurse, or other, the last of which included medical technicians, midwives, pharmacists, pharmacy technicians, administrative staff, catering staff, and cleaners), type of health care facility (second-line versus frontline, the latter of which includes the outpatient department, emergency department, quarantine and isolation areas, medical imaging and laboratory diagnosis department, and patient administration areas), and previous epidemic (eg, SARS, tuberculosis, influenza A) containment experience (no versus yes). Additionally, comorbidity was assessed using the Charlson comorbidity index items [48,49].

    Health Literacy

    A 12-item short-form health literacy questionnaire (HLS-SF12) was used. The questionnaire was validated and used in Asian countries [50], including Vietnam [51-54]. HCWs rated their perceived difficulty of items based on a 4-point Likert scale from 1=Very difficult to 4=Very easy. The health literacy index score was standardized to an unified metric from 0 to 50, with higher scores representing better health literacy [50,55].

    eHealth Literacy

    The widely used eHealth literacy scale (eHEALS) with 8 items was used to assess HCWs’ eHealth literacy skills [56]. The questionnaire was translated into Vietnamese by the researchers. The content was then validated by an expert panel (28 medical doctors, 7 nurses, and 9 nutrition and public health professionals). The expert panel suggested retaining the original rating and scoring system. HCWs rated their experiences using the internet for health information based on a 5-point Likert scale from 1=Strongly disagree to 5=Strongly agree. Scores range from 8 to 40, with high eHEALS scores representing better eHealth literacy.

    Adherence to IPC Procedures

    Participants reported the practices and activities performed related to COVID-19 IPC during health care interactions. The questionnaire was adapted from the interim guidance of the World Health Organization [57]. The IPC measures included the following: (1) wearing single-use gloves, (2) wearing a medical mask, (3) wearing a face shield or goggles/protective glasses, (4) wearing a disposable gown, (5) removing and replacing personal protective equipment (PPE) according to protocol, (6) performing hand hygiene before and after touching patients, (7) performing hand hygiene before and after performing any clean or aseptic procedure, (8) performing hand hygiene after exposure to body fluid, (9) performing hand hygiene after touching patients’ surroundings, and (10) decontaminating high-touch surfaces. HCWs quantified the frequency of performance of the above IPC procedures as recommended, where 0=Never, 1=Rarely (<20% of the time), 2=Occasionally (20% to <50% of the time), 3=Most of the time (50% to <95% of the time), and 4=Always (≥95% of the time). The total performance score of the 10 activities ranged from 0 to 40, with higher scores indicating better adherence to IPC procedures.

    Lifestyle Changes

    HCWs reported their current smoking (never/stopped/less versus unchanged or more), drinking (never/stopped/less versus unchanged or more), physical activity (never/stopped/less versus unchanged or more), and eating (less healthy versus unchanged or healthier) behaviors as compared with that before the pandemic [54].

    Suspected COVID-19 Symptoms

    HCWs were screened for suspected COVID-19 symptoms [58], including the common symptoms of fever, cough, and dyspnea, along with the less common symptoms of myalgia, fatigue, sputum production, confusion, headache, sore throat, rhinorrhea, chest pain, hemoptysis, diarrhea, and nausea/vomiting. HCWs with any of those symptoms were classified as having symptoms indicative of COVID-19.

    Ethical Consideration

    The study was reviewed and approved by the Institutional Ethical Review Committee of Hanoi University of Public Health, Vietnam (Institutional Review Board number 133/2020/YTCC-HD3). The HCWs voluntarily took the survey.

    Data Analysis

    Psychometric Properties of the eHEALS
    Construct Validity

    The construct validity of the eHEALS was examined using principal component analysis (PCA). An Kaiser-Meyer Olkin (KMO) value ≥0.6 was set to measure sampling adequacy and a Bartlett Test of Sphericity value <0.05 was set to determine the suitability of the data for PCA [59]. The oblique rotation (Promax) method was used.

    Convergent Validity

    The Spearman correlation was used to check the correlations between the eHEALS and its eight items.

    Criterion Validity

    The Pearson correlation between eHEALS and HLS-SF12 was estimated to provide evidence of criterion validity [60].

    Floor and Ceiling Effects

    The percentages of the lowest and highest score among HCWs were calculated. Minimal percentages (<15%) were recommended to eliminate floor and ceiling effects [61].

    Reliability Analysis

    The internal consistency of the eHEALS was estimated using the Cronbach α test. A Cronbach α value ≥.70 was designated as satisfactory reliability [62].

    Health Literacy, eHealth Literacy, and Associated Factors

    The distributions of HL and eHEALS scores in different categories of studied variables were explored using a one-way analysis of variance (ANOVA) test. In addition, bivariate and multivariate linear regression models were used to examine predictors of HL and eHEALS scores and to investigate the associations of HL and eHEALS scores with adherence to IPC measures. Next, bivariate and multivariate logistic regression models were used to examine the associations of HL and eHEALS scores with lifestyle changes and suspected COVID-19 symptoms. The factors that demonstrated associations with outcome variables at P values <.20 in the bivariate models were included in multivariate models [63]. To exclude colliders, which may cause multicollinearity, we checked the correlations between them using the Spearman correlation test. The representative factors were selected in the multivariate analysis. The regression coefficient (B), odds ratio (OR), and 95% confidence interval (95% CI) were reported appropriately.

    Data were analyzed using IBM SPSS (Version 20.0; IBM Corp). The significance level was set at a P value <.05.


    Characteristics of HCWs

    Table 2 indicates that, of 5209 HCWs, 905 (17.4%) were aged 41-60 years, 1714 (32.9%) were men, 3915 (75.2%) were ever married, 2458 (47.2%) could very or fairly easily pay for medication, 4506 (86.5%) had a middle or high social status, 2540 (48.8%) were nurses, 1453 (27.9%) were doctors, 2299 (44.1%) were frontline HCWs, 1999 (38.4%) had epidemic containment experience, 270 (5.2%) had comorbidities, 769 (14.8%) had had suspected COVID-19 symptoms, 5042 (96.8%) ate at an “unchanged or healthier” level, 228 (4.4%) smoked at an “unchanged or more” level, 234 (4.5%) drank at an “unchanged or more” level, and 3553 (68.2%) had an “unchanged or more” physical activity level during the pandemic. The mean (SD) scores of adherence to IPC measures, HL, and eHEALS were 30.6 (6.2), 36.2 (7.3), and 33.1 (4.8), respectively. The HL scores varied by age, gender, ability to pay for medication, social status, type of health care personnel, epidemic containment experience, BMI, suspected COVID-19 symptoms, dietary intake, smoking, and physical activity (P<.05). The eHEALS scores varied by gender, ability to pay for medication, type of health care personnel, epidemic containment experience, BMI, suspected COVID-19 symptoms, dietary intake, smoking, and physical activity (Table 2).

    Table 2. Characteristics, health literacy, and eHealth literacy among health care workers.
    VariablesTotal (N=5209)Health literacy
    		eHealth Literacy Scale

    		Participants, n (%)Mean (SD)P valueaMean (SD)P valuea
    Age (years).02N/Ab.88

    		21-404304 (82.6)36.1 (7.2)N/A33.1 (4.7)N/A

    		41-60905 (17.4)36.7 (7.5)N/A33.1 (5.0)N/A
    Gender<.001N/A<.001

    		Women3495 (67.1)35.7 (7.0)N/A32.8 (4.5)N/A

    		Men1714 (32.9)37.1 (7.8)N/A33.8 (5.3)N/A
    Marital status.04N/A.29

    		Never married1294 (24.8)35.8 (7.0)N/A33.0 (4.8)N/A

    		Ever married3915 (75.2)36.3 (7.4)N/A33.2 (4.7)N/A
    Ability to pay for medication<.001N/A<.001

    		Very or fairly difficult2751 (52.8)35.3 (7.4)N/A32.8 (4.9)N/A

    		Very or fairly easy2458 (47.2)37.2 (7.0)N/A33.5 (4.6)N/A
    Social status<.001N/A.27

    		Low703 (13.5)35.0 (7.7)N/A32.9 (4.7)N/A

    		Middle or high4506 (86.5)36.3 (7.2)N/A33.2 (4.8)N/A
    Type of health care personnel<.001N/A<.001

    		Other1216 (23.3)35.7 (7.6)N/A33.1 (5.1)N/A

    		Nurse2540 (48.8)35.7 (7.2)N/A32.7 (4.6)N/A

    		Doctor1453 (27.9)37.3 (7.1)N/A33.9 (4.8)N/A
    Type of health care facilityc.77N/A.46

    		Non-frontline2910 (55.9)36.2 (7.4)N/A33.2 (4.7)

    		Frontline2299 (44.1)36.1 (7.1)N/A33.1 (4.8)N/A
    Epidemic containment experience<.001N/A<.001

    		No3210 (61.6)35.4 (7.2)N/A32.9 (4.6)N/A

    		Yes1999 (38.4)37.4 (7.3)N/A33.5 (5.0)N/A
    Comorbidity.38N/A.37

    		None4939 (94.8)36.1 (7.3)N/A33.1 (4.8)N/A

    		One or more270 (5.2)36.5 (7.6)N/A32.9 (5.0)N/A
    Suspected COVID-19 symptomsd<.001N/A<.001

    		No4440 (85.2)36.4 (7.3)N/A33.3 (4.7)N/A

    		Yes769 (14.8)34.7 (7.2)N/A32.3 (5.1)N/A
    Dietary intakee<.001N/A.002

    		Eat less healthy167 (3.2)34.2 (6.9)N/A32.0 (5.9)N/A

    		Unchanged or healthier5042 (96.8)36.2 (7.3)N/A33.2 (4.7)N/A
    Smoking tobaccof.046N/A.03

    		Never, stopped, or less4981 (95.6)36.1 (7.2)N/A33.1 (4.7)N/A

    		Unchanged or more228 (4.4)37.1 (8.6)N/A33.8 (5.5)N/A
    Drinking alcoholf.18N/A.53

    		Never, stopped, or less4975 (95.5)36.1 (7.2)N/A33.1 (4.7)N/A

    		Unchanged or more234 (4.5)36.8 (8.3)N/A33.3 (5.7)N/A
    Physical activityf<.001N/A<.001

    		Never, stopped, or less1656 (31.8)35.2 (7.6)N/A32.5 (5.2)N/A

    		Unchanged or more3553 (68.2)36.6 (7.1)N/A33.4 (4.6)N/A
    Adherence to infection prevention and control measures, mean (SD)30.6 (6.2)N/AN/AN/AN/A
    Health literacy index, mean (SD)36.2 (7.3)N/AN/AN/AN/A
    eHealth Literacy Scale score, mean (SD)33.1 (4.8)N/AN/AN/AN/A

    aP value: result of one-way analysis of variance (ANOVA) test.

    bN/A: not applicable.

    cFrontline areas include the outpatient department, emergency department, isolation areas, imaging and laboratory diagnosis department, and patient administration areas.

    dSuspected COVID-19 symptoms include common symptoms (fever, cough, dyspnea) and less common symptoms (myalgia, fatigue, sputum production, confusion, headache, sore throat, rhinorrhea, chest pain, hemoptysis, diarrhea, and nausea/vomiting).

    fHealth care workers were asked whether their lifestyle behaviors got worse, better, or were unchanged during the COVID-19 pandemic as compared to before the pandemic.

    Psychometric Properties of eHealth Literacy

    As shown in Table 3, the KMO value of the overall scale was 0.93 and the values of its 8 items ranged from 0.92 to 0.95. The Bartlett Test of Sphericity value was <0.001, which is a satisfactory level. In addition, the average communality value of 0.76 was satisfactory, demonstrating the accuracy of the approach [59]. Overall, the 8 items of the eHEALS were strongly loaded on one component and explained 76.34% of the scale variance. The factor loading values of 8 items were ranked from 0.78 to 0.92, as shown in Table 3. The correlations between each item and the scale range from 0.80 to 0.84, indicating satisfactory convergent validity [50,51]. In addition, the correlation between eHEALS and HL scores was moderate (ρ=0.42), providing evidence of criterion validity [60]. Furthermore, the Cronbachα value of .95 indicated a high level of internal consistency. There was no significant floor effect, with 0.70% of participants having the lowest potential response. A marginal ceiling effect was found, with 16.1% of participants having the highest potential response (slightly higher than the recommended percentage of <15%; Table 3).

    Table 3. Construct, convergent, and criterion validity, internal consistency, and floor and ceiling effects of the 8-item eHealth Literacy Scale (N=5209).
    Construct validity, factor loadingsValues
    eHealth Literacy scale items

    		I know what health resources are available on the internet0.85

    		I know where to find helpful health resources on the internet0.90

    		I know how to find helpful health resources on the internet0.92

    		I know how to use the internet to answer my questions about health0.90

    		I know how to use the health information I find on the internet to help me0.92

    		I have the skills I need to evaluate the health resources I find on the internet0.89

    		I have the skills needed to tell high-quality health resources from low-quality health resources on the internet0.82

    		I feel confident in using information from the internet to make health decisions0.78
    Percentage of variance, %76.34
    Item-scale convergent validity, mean of ρa (range)0.83 (0.80-0.84)
    Criterion validity, correlation with health literacy, ρb0.42
    Internal consistency, Cronbach α.95
    Floor effect, %0.70
    Ceiling effect, %16.10

    aρ: Spearman correlation coefficient.

    bρ: Pearson correlation coefficient.

    Determinants of HL and eHealth Literacy

    The week correlations among independent variables (ρ <0.30) suggest that there is no collider which might affect the results (Table 1 in Multimedia Appendix 1). Results of the multivariate analysis shown in Table 4 indicated that HCWs with higher HL scores were men (unstandardized regression coefficient [B]=1.01, 95% CI 0.57-1.45, P<.001), those with a very or fairly easy ability to pay for medication (B=1.65, 95% CI 1.25-2.05, P<.001), those with a middle or high social status (B=0.586, 95% CI 0.003-1.169, P=.049), doctors (B=1.29, 95% CI 0.73-1.84, P<.001), and those with epidemic containment experience (B=1.96, 95% CI 1.56-2.37, P<.001), as compared to their counterparts, respectively.

    HCWs with higher eHEALS scores were men (B=0.72, 95% CI 0.43-1.00, P<.001), those with a very or fairly easy ability to pay for medication (B=0.60, 95% CI 0.34-0.86, P<.001), doctors (B=0.56, 95% CI 0.20-0.93, P=.003), and those with epidemic containment experience (B=0.64, 95% CI 0.38-0.91, P<.001), as compared to their counterparts, respectively (Table 4).

    Table 4. Determinants of health literacy and eHealth literacy among health care workers (N=5209).
    VariablesHealth literacy


    		eHealth literacy



    		Bivariate
    		Multivariate
    		Bivariate
    		Multivariate

    		Ba (95% CI)P valueB (95% CI)P valueB (95% CI)P valueB (95% CI)P value
    Age (years)

    		21-40ReferenceN/AbReferenceN/AReferenceN/AReferenceN/A

    		41-600.62 (0.10 to 1.14).02–0.05 (–0.58 to 0.49).86–0.03 (–0.37 to 0.31).88–0.30 (–0.65 to 0.04).08
    Gender

    		FemaleReferenceN/AReferenceN/AReferenceN/AReferenceN/A

    		Male1.36 (0.94 to 1.78)<.0011.01 (0.57 to 1.45)<.0010.99 (0.72 to 1.27)<.0010.72 (0.43 to 1.00)<.001
    Marital status

    		Never marriedReferenceN/AReferenceN/AReferenceN/AN/AN/A

    		Ever married0.47 (0.01 to 0.93).040.34 (–0.12 to 0.81).150.16 (–0.14 to 0.46).29N/AN/A
    Ability to pay for medication

    		Very or fairly difficultReferenceN/AReferenceN/AReferenceN/AReferenceN/A

    		Very or fairly easy1.9 (1.51 to 2.29)<.0011.65 (1.25 to 2.05)<.0010.72 (0.46 to 0.98)<.0010.60 (0.34 to 0.86)<.001
    Social status

    		LowReferenceN/AReferenceN/AReferenceN/AN/AN/A

    		Middle or high1.29 (0.72 to 1.87)<.0010.586 (0.003 to 1.169).0490.22 (–0.16 to 0.59).27N/AN/A
    Type of health care personnel

    		OtherReferenceN/AReferenceN/AReferenceN/AReferenceN/A

    		Nurse0.09 (–0.41 to 0.58).730.18 (–0.32 to 0.68).48–0.40 (–0.72 to –0.07).02–0.32 (–0.65 to 0.01).06

    		Doctor1.68 (1.13 to 2.24)<.0011.29 (0.73 to 1.84)<.0010.76 (0.39 to 1.12)<.0010.56 (0.20 to 0.93).003
    Type of health care facilityc

    		Non-frontlineReferenceN/AN/AN/AReferenceN/AN/AN/A

    		Frontline–0.06 (–0.46 to 0.34).77N/AN/A–0.10 (–0.36 to 0.16).46N/AN/A
    Epidemic containment experience

    		NoReferenceN/AReferenceN/AReferenceN/AReferenceN/A

    		Yes1.95 (1.54 to 2.35)<.0011.96 (1.56 to 2.37)<.0010.55 (0.29 to 0.82)<.0010.64 (0.38 to 0.91)<.001
    Comorbidity

    		NoneReferenceN/AReferenceN/AReferenceN/AN/AN/A

    		One or more0.40 (–0.49 to 1.29).378N/AN/A–0.27 (–0.85 to 0.32).369N/AN/A

    aB: unstandardized regression coefficient.

    bN/A: not applicable.

    cFrontline areas are the outpatient department, emergency department, isolation areas, imaging and laboratory diagnosis department, and patient administration areas.

    Association Between HL and eHealth Literacy and Adherence to IPC Procedures

    According to the results of a multivariate linear regression analysis (Table 5), higher HL (B=0.13, 95% CI 0.10-0.15, P<.001) and eHEALS scores (B=0.22, 95% CI 0.19-0.26, P<.001) were found to be associated with better adherence to IPC measures after adjusting for age, gender, ability to pay for medication, social status, profession, type of health care facility, epidemic containment experience, and comorbidities. These adjusted factors showed the associations with adherence to IPC measures at P<.20 (Table 2 in Multimedia Appendix 1).

    Table 5. Associations of health literacy and eHealth literacy with adherence to infection prevention and control measures, lifestyle changes, and suspected COVID-19 symptoms among health care workers (N=5209).
    VariablesAdherence to IPC measuresa
    		Dietary intakeb
    		Smoking tobaccoc
    		Drinking alcohold
    		Physical activitye
    		Suspected COVID-19 symptomsf

    		Bg (95% CI)P valueORh (95% CI)P valueOR (95% CI)P valueOR (95% CI)P valueOR (95% CI)P valueOR (95% CI)P value
    Health literacy, 1-score increment

    		Bivariate model0.13 (0.11-0.15)<.0011.04 (1.02-1.06)<.0011.02 (1.00-1.04).0461.01 (0.99-1.03).181.03 (1.02-1.04)<.0010.97 (0.96-0.98)<.001

    		Multivariate model0.13 (0.10-0.15)<.0011.04 (1.01-1.06).0011.01 (0.99-1.03).341.01 (0.99-1.02).471.03 (1.02-1.03)<.0010.97 (0.96-0.98)<.001
    eHealth Literacy Scale, 1-score increment

    		Bivariate model0.22 (0.18-0.25)<.0011.04 (1.02-1.07).0021.03 (1-1.07).031.01 (0.98-1.04).531.04 (1.03-1.05)<.0010.96 (0.95-0.98)<.001

    		Multivariate model0.22 (0.19-0.26)<.0011.04 (1.02-1.07).0021.01 (0.99-1.04).300.99 (0.97-1.02).691.04 (1.03-1.05)<.0010.96 (0.95-0.98)<.001

    aAdherence to infection prevention and control procedures; adjusted for age, gender, ability to pay for medication, social status, type of health care personnel, type of health care facility, epidemic containment experience, and comorbidity in the multivariate model.

    bAdjusted for age, gender, marital status, ability to pay for medication, and social status in the multivariate model. The reference group is “less healthy diet” and the test group is “unchanged or healthier diet.”

    cAdjusted for age, gender, marital status, social status, and type of health care personnel in the multivariate model. The reference group is “never, stopped, or less smoking,” and the test group is “unchanged or more smoking.”

    dAdjusted for age, gender, social status, type of health care personnel, and epidemic containment experience in the multivariate model. The reference group is “never, stopped, or less drinking,” and the test group is “unchanged or more drinking.”

    eAdjusted for age, gender, ability to pay for medication, social status, type of health care personnel, type of health care facility, and epidemic containment experience in the multivariate model. The reference group is “never, stopped, or less physical activity,” and the test group is “unchanged or more physical activity.”

    fSuspected COVID-19 symptoms include common symptoms (fever, cough, dyspnea) and less common symptoms (myalgia, fatigue, sputum production, confusion, headache, sore throat, rhinorrhea, chest pain, hemoptysis, diarrhea, and nausea/vomiting). Adjusted for age, gender, marital status, ability to pay for medications, social status, type of health care personnel, and comorbidity in the multivariate model.

    gB: unstandardized regression coefficient.

    hOR: odds ratio.

    Association Between HL and eHealth Literacy and Lifestyle Changes

    The results of the multivariate logistic regression analysis shown in Table 5 indicated that HCWs with higher HL (OR 1.04, 95% CI 1.01-1.06, P=.001) and higher eHEALS scores (OR 1.04, 95% CI 1.02-1.07, P=.002) had a higher likelihood of eating at an “unchanged or healthier” level after being adjusted for age, gender, marital status, ability to pay for medication, and social status. These adjusted factors had associations with eating behavior changes at P<.20 (Table 2 in Multimedia Appendix 1). In addition, HCWs with higher HL scores (OR 1.03, 95% CI 1.02-1.03, P<.001) and higher eHEALS scores (OR 1.04, 95% CI 1.03-1.05, P<.001) had a higher likelihood of doing physical activity at an “unchanged or more” level after adjusting for age, gender, ability to pay for medication, social status, profession, type of health care facility, and epidemic containment experience. These adjusted factors had associations with physical activity at P<.20 (Table 2 in Multimedia Appendix 1). No association was found between HL and eHEALS scores and smoking or drinking behaviors.

    Associations of HL and eHealth Literacy With Suspected COVID-19 Symptoms

    As shown in Table 5, HCWs with higher HL scores (OR 0.97, 95% CI 0.96-0.98, P<.001) and higher eHEALS scores (OR 0.96, 95% CI 0.95-0.98, P<.001) had a lower likelihood of having suspected COVID-19 symptoms after adjusting for age, gender, marital status, ability to pay for medication, social status, profession, and comorbidities. These adjusted factors had associations with suspected COVID-19 symptoms at P<.20 (Table 2 in Multimedia Appendix 1).


    Principal Findings

    The eHEALS questionnaire was found to be valid and reliable for assessing the eHealth literacy of HCWs. The tool was found to have satisfactory construct validity, convergent validity, criterion validity, and reliability. There was no flooring effect. A marginal ceiling effect was found. A high percentage of participants with the highest possible score was also observed in a previous study that validated the digital health literacy scale [64]. This could be because HCWs have been recognized as a group that has a higher standard of health knowledge and skills than the average person. Therefore, it is reasonable that many participants had a higher or maximum score.

    In this study, male HCWs had higher HL and eHEALS scores. This finding is inconsistent with a previous study conducted among the general population in Europe [65], in which men had lower HL scores than women. Another study conducted among the general population in Taiwan showed that men also had lower scores of HL and digital health diet literacy than women [66]. In addition, eHealth literacy was found to be higher in women than men in the general population in Hungary [67] and in health care professionals in Ethiopia [68]. This inconsistency could be explained by the fact that although the total HCW population had fewer males (32.9% versus 67.1% female), the proportion of male doctors (797/1453 or 54.9%) was higher than that of female doctors (656/1453 or 45.1%).

    Doctors had higher HL and eHEALS scores than other HCWs in this study. Similarly, HCWs with epidemic containment experience had higher HL and eHEALS scores in this study. Among HCWs, doctors receive longer professional training and they have been recognized as the group with the highest level of ability to find, understand, justify, and use health-related information. In addition, they are involved in educating and counseling other HCWs and patients [69]. HCWs who were involved in containing previous outbreaks or epidemics logically had better health knowledge and skills that made their HL and eHEALS scores higher. HL was found to be a strategic approach for improving communication between patients and HCWs [41], further contributing to the containment of the pandemic and its consequences [45].

    HCWs with a better ability to pay for medication had higher HL and eHEALS scores. A positive association between the ability to pay for medication was prominently found in many previous studies [51,52,66,70,71]. In addition, HCWs with middle or high social status had higher HL scores. This was similar to the findings of previous studies conducted in Asia [51,52,66,70-72] and Europe [65]. This implies that socioeconomic factors are independent indictors that should be taken into consideration in intervention development to improve HL and eHEALS scores.

    In this study, HCWs with higher HL or eHEALS scores had better adherence to IPC procedures. This is the first study investigating these associations. In the literature, higher HL scores were found to be associated with better adherence to therapies among people with chronic diseases [73-76]. Receiving COVID-19–related information online during the pandemic could be beneficial for various preventive behaviors [77]. Therefore, more efforts are needed to increase the distribution of accurate information and encourage appropriate behaviors [78]. HCWs should be aware of and strictly adhere to the guidelines for IPC to contain the COVID-19 pandemic.

    Our study found that HCWs with higher HL and eHEALS scores had a lower likelihood of suspected COVID-19 symptoms. It has been noted that individuals with higher HL scores had better health status [71,79] and well-being [80,81]. This could be further explained by the fact that higher HL and eHEALS scores were associated with healthier eating and physical activity behaviors in this study. In a previous study, we also found that medical and nursing students with higher digital healthy diet literacy had healthier eating behaviors during the pandemic [82]. In addition, eHealth literacy was found to be associated with key issues related to seeking out exercise regime information and maintaining a healthy diet [83]. In addition, it has been reported that people with higher HL scores had healthier behaviors (eg, exercise, balanced diet) [66,84], further protecting and improving their health and well-being [85].

    This study has some limitations. First, the study was conducted online and the suspected COVID-19 symptoms were self-reported; therefore, we cannot confirm COVID-19 cases and exclude them from our study. Fortunately, there were no new confirmed cases during the data collection period [86]. Second, during the lockdown period, people were ordered to stay at home and only go out to buy food or medicine, for emergencies, or to work at businesses producing and providing essential goods and services. This may cause a measurement bias in the application domain of HL, as the questionnaire cannot distinguish between an individual’s ability and restrictions during lockdown (eg, on questions such as “on a scale from very easy to very difficult, how easy would you say it is to join a sports club or exercise class if you want to?”). Third, a cross-sectional design was used, which cannot provide evidence of the causal relationships among studied variables. Furthermore, the convenient sampling technique used may limit the generalizability of the findings. In addition, the differences between some categories were not examined due to small sample sizes (eg, widowed/separated/divorced participants [140/5209, 2.7%] or those with high social status 63/5209, 1.2%]). However, we surveyed a relatively large sample of HCWs from 15 hospitals and health centers across Vietnam, which can help with exploring associations in future research and provides evidence for potential interventions.

    Conclusions

    The eHEALS questionnaire is a valid and reliable tool for assessing eHealth literacy among HCWs. HL and eHealth literacy were significantly higher in men, those with better ability to pay for medication, doctors, and those with previous epidemic containment experience. Both HL and eHealth literacy were associated with better adherence to IPC procedures, healthier lifestyles (eg, healthier eating behavior and more physical activity during the pandemic), and a lower likelihood of having suspected COVID-19 symptoms. Integrative and multidisciplinary approaches are required to improve HCWs’ HL and eHealth literacy, which could help improve adherence to IPC measures, promote healthy behaviors, and protect the health of HCWs. This would further contribute to containing the COVID-19 pandemic and minimizing its consequences.

    Acknowledgments

    We appreciate and acknowledge the participation of HCWs from the selected hospitals and health centers. This work was supported by Military Hospital 103, Vietnam, and Taipei Medical University, Taiwan (108-3805-022-400).

    Authors' Contributions

    BND, TVT, and TVD analyzed the data and drafted the manuscript. BND, TVT, DTP, HCN, TTPN, HCN, THH, HKD, MVT, TVD, HQN, TTN, NPTN, CQT, KVT, TTD, HXP, LVN, KTN, PWC, and TVD contributed to conceptualization, investigation, methodology, validation, and manuscript revision. BND, TVT, DTP, HCN, TTPN, HCN, THH, HKD, MVT, TVD, HQN, TTN, NPTN, CQT, KVT, TTD, HXP, LVN, KTN, and TVD conducted data curation. All authors gave final approval of the manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Supplementary data.

    DOCX File , 38 KB
    1. Greenberg N, Docherty M, Gnanapragasam S, Wessely S. Managing mental health challenges faced by healthcare workers during covid-19 pandemic. BMJ 2020 Mar 26;368:m1211. [CrossRef] [Medline]
    2. Bassetti M, Vena A, Giacobbe DR. The novel Chinese coronavirus (2019-nCoV) infections: Challenges for fighting the storm. Eur J Clin Invest 2020 Mar;50(3):e13209. [CrossRef] [Medline]
    3. Phelan AL, Katz R, Gostin LO. The Novel Coronavirus Originating in Wuhan, China: Challenges for Global Health Governance. JAMA 2020 Feb 25;323(8):709-710 [FREE Full text] [CrossRef] [Medline]
    4. Rubin R. The Challenge of Preventing COVID-19 Spread in Correctional Facilities. JAMA 2020 May 12;323(18):1760-1761. [CrossRef] [Medline]
    5. Xiang Y, Jin Y, Cheung T. Joint International Collaboration to Combat Mental Health Challenges During the Coronavirus Disease 2019 Pandemic. JAMA Psychiatry 2020 Oct 01;77(10):989-990. [CrossRef] [Medline]
    6. Campion J, Javed A, Sartorius N, Marmot M. Addressing the public mental health challenge of COVID-19. The Lancet Psychiatry 2020 Aug;7(8):657-659. [CrossRef]
    7. Nicola M, Alsafi Z, Sohrabi C, Kerwan A, Al-Jabir A, Iosifidis C, et al. The socio-economic implications of the coronavirus pandemic (COVID-19): A review. Int J Surg 2020 Jun;78:185-193 [FREE Full text] [CrossRef] [Medline]
    8. Clark A, Jit M, Warren-Gash C, Guthrie B, Wang HHX, Mercer SW, Centre for the Mathematical Modelling of Infectious Diseases COVID-19 working group. Global, regional, and national estimates of the population at increased risk of severe COVID-19 due to underlying health conditions in 2020: a modelling study. Lancet Glob Health 2020 Aug;8(8):e1003-e1017 [FREE Full text] [CrossRef] [Medline]
    9. Banerjee A, Pasea L, Harris S, Gonzalez-Izquierdo A, Torralbo A, Shallcross L, et al. Estimating excess 1-year mortality associated with the COVID-19 pandemic according to underlying conditions and age: a population-based cohort study. Lancet 2020 May 30;395(10238):1715-1725 [FREE Full text] [CrossRef] [Medline]
    10. Moradian N, Ochs HD, Sedikies C, Hamblin MR, Camargo CA, Martinez JA, et al. The urgent need for integrated science to fight COVID-19 pandemic and beyond. J Transl Med 2020 May 19;18(1):205 [FREE Full text] [CrossRef] [Medline]
    11. Holmes EA, O'Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry 2020 Jun;7(6):547-560 [FREE Full text] [CrossRef] [Medline]
    12. Nazir M, Hussain I, Tian J, Akram S, Mangenda Tshiaba S, Mushtaq S, et al. A Multidimensional Model of Public Health Approaches Against COVID-19. Int J Environ Res Public Health 2020 May 26;17(11):3780 [FREE Full text] [CrossRef] [Medline]
    13. Betsch C. How behavioural science data helps mitigate the COVID-19 crisis. Nat Hum Behav 2020 May 27;4(5):438-438 [FREE Full text] [CrossRef] [Medline]
    14. Bavel JJV, Baicker K, Boggio PS, Capraro V, Cichocka A, Cikara M, et al. Using social and behavioural science to support COVID-19 pandemic response. Nat Hum Behav 2020 May 30;4(5):460-471. [CrossRef] [Medline]
    15. Paakkari L, Okan O. COVID-19: health literacy is an underestimated problem. Lancet Public Health 2020 May;5(5):e249-e250 [FREE Full text] [CrossRef] [Medline]
    16. Chong YY, Cheng HY, Chan HYL, Chien WT, Wong SYS. COVID-19 pandemic, infodemic and the role of eHealth literacy. Int J Nurs Stud 2020 Aug;108:103644 [FREE Full text] [CrossRef] [Medline]
    17. PM orders strict nationwide social distancing rules, starting April 1. Vietnam Law Magazine. 2020.   URL: https:/​/vietnamlawmagazine.​vn/​pm-orders-strict-nationwide-social-distancing-rules-starting-april-1-27108.​html [accessed 2020-03-31]
    18. Hao F, Tan W, Jiang L, Zhang L, Zhao X, Zou Y, et al. Do psychiatric patients experience more psychiatric symptoms during COVID-19 pandemic and lockdown? A case-control study with service and research implications for immunopsychiatry. Brain Behav Immun 2020 Jul;87:100-106 [FREE Full text] [CrossRef] [Medline]
    19. Ozamiz-Etxebarria N, Idoiaga Mondragon N, Dosil Santamaría M, Picaza Gorrotxategi M. Psychological Symptoms During the Two Stages of Lockdown in Response to the COVID-19 Outbreak: An Investigation in a Sample of Citizens in Northern Spain. Front Psychol 2020;11:1491 [FREE Full text] [CrossRef] [Medline]
    20. Di Renzo L, Gualtieri P, Pivari F, Soldati L, Attinà A, Cinelli G, et al. Eating habits and lifestyle changes during COVID-19 lockdown: an Italian survey. J Transl Med 2020 Jun 08;18(1):229 [FREE Full text] [CrossRef] [Medline]
    21. Ammar A, Brach M, Trabelsi K, Chtourou H, Boukhris O, Masmoudi L, et al. Effects of COVID-19 Home Confinement on Eating Behaviour and Physical Activity: Results of the ECLB-COVID19 International Online Survey. Nutrients 2020 May 28;12(6):1583 [FREE Full text] [CrossRef] [Medline]
    22. Sidor A, Rzymski P. Dietary Choices and Habits during COVID-19 Lockdown: Experience from Poland. Nutrients 2020 Jun 03;12(6):1657 [FREE Full text] [CrossRef] [Medline]
    23. Rodríguez-Pérez C, Molina-Montes E, Verardo V, Artacho R, García-Villanova B, Guerra-Hernández EJ, et al. Changes in Dietary Behaviours during the COVID-19 Outbreak Confinement in the Spanish COVIDiet Study. Nutrients 2020 Jun 10;12(6):1730 [FREE Full text] [CrossRef] [Medline]
    24. Wardell J, Kempe T, Rapinda K, Single A, Bilevicius E, Frohlich J, et al. Drinking to Cope During COVID-19 Pandemic: The Role of External and Internal Factors in Coping Motive Pathways to Alcohol Use, Solitary Drinking, and Alcohol Problems. Alcohol Clin Exp Res 2020 Sep 01:1. [CrossRef] [Medline]
    25. Gong K, Xu Z, Cai Z, Chen Y, Wang Z. Internet Hospitals Help Prevent and Control the Epidemic of COVID-19 in China: Multicenter User Profiling Study. J Med Internet Res 2020 Apr 14;22(4):e18908 [FREE Full text] [CrossRef] [Medline]
    26. Sesagiri Raamkumar A, Tan S, Wee H. Measuring the Outreach Efforts of Public Health Authorities and the Public Response on Facebook During the COVID-19 Pandemic in Early 2020: Cross-Country Comparison. J Med Internet Res 2020 May 19;22(5):e19334 [FREE Full text] [CrossRef] [Medline]
    27. Shen C, Chen A, Luo C, Zhang J, Feng B, Liao W. Using Reports of Symptoms and Diagnoses on Social Media to Predict COVID-19 Case Counts in Mainland China: Observational Infoveillance Study. J Med Internet Res 2020 May 28;22(5):e19421 [FREE Full text] [CrossRef] [Medline]
    28. Abd-Alrazaq A, Alhuwail D, Househ M, Hamdi M, Shah Z. Top Concerns of Tweeters During the COVID-19 Pandemic: Infoveillance Study. J Med Internet Res 2020 Apr 21;22(4):e19016 [FREE Full text] [CrossRef] [Medline]
    29. Liu Q, Zheng Z, Zheng J, Chen Q, Liu G, Chen S, et al. Health Communication Through News Media During the Early Stage of the COVID-19 Outbreak in China: Digital Topic Modeling Approach. J Med Internet Res 2020 Apr 28;22(4):e19118 [FREE Full text] [CrossRef] [Medline]
    30. Liao Q, Yuan J, Dong M, Yang L, Fielding R, Lam WWT. Public Engagement and Government Responsiveness in the Communications About COVID-19 During the Early Epidemic Stage in China: Infodemiology Study on Social Media Data. J Med Internet Res 2020 May 26;22(5):e18796 [FREE Full text] [CrossRef] [Medline]
    31. Ahmad AR, Murad HR. The Impact of Social Media on Panic During the COVID-19 Pandemic in Iraqi Kurdistan: Online Questionnaire Study. J Med Internet Res 2020 May 19;22(5):e19556 [FREE Full text] [CrossRef] [Medline]
    32. Bao Y, Sun Y, Meng S, Shi J, Lu L. 2019-nCoV epidemic: address mental health care to empower society. Lancet 2020 Feb 22;395(10224):e37-e38 [FREE Full text] [CrossRef] [Medline]
    33. Shimizu K. 2019-nCoV, fake news, and racism. Lancet 2020 Feb 29;395(10225):685-686 [FREE Full text] [CrossRef] [Medline]
    34. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar 14;395(10227):912-920 [FREE Full text] [CrossRef] [Medline]
    35. Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res 2015 May 07;17(5):e112 [FREE Full text] [CrossRef] [Medline]
    36. Norman CD, Skinner HA. eHealth Literacy: Essential Skills for Consumer Health in a Networked World. J Med Internet Res 2006 Jun 16;8(2):e9 [FREE Full text] [CrossRef] [Medline]
    37. Park HW, Park S, Chong M. Conversations and Medical News Frames on Twitter: Infodemiological Study on COVID-19 in South Korea. J Med Internet Res 2020 May 05;22(5):e18897. [CrossRef] [Medline]
    38. Ko N, Lu W, Chen Y, Li D, Wang P, Hsu S, et al. COVID-19-related information sources and psychological well-being: An online survey study in Taiwan. Brain Behav Immun 2020 Jul;87:153-154 [FREE Full text] [CrossRef] [Medline]
    39. Mackert M, Ball J, Lopez N. Health literacy awareness training for healthcare workers: improving knowledge and intentions to use clear communication techniques. Patient Educ Couns 2011 Dec;85(3):e225-e228. [CrossRef] [Medline]
    40. Yunyongying P, Rich M, Jokela J. Patient-Centered Performance Metrics. JAMA 2019 May 14;321(18):1829. [CrossRef] [Medline]
    41. McKinn S, Linh DT, Foster K, McCaffery K. Communication Between Health Workers and Ethnic Minorities in Vietnam. Health Lit Res Pract 2017 Oct 01;1(4):e163-e172 [FREE Full text] [CrossRef] [Medline]
    42. Tavakoly Sany SB, Behzhad F, Ferns G, Peyman N. Communication skills training for physicians improves health literacy and medical outcomes among patients with hypertension: a randomized controlled trial. BMC Health Serv Res 2020 Jan 23;20(1):60 [FREE Full text] [CrossRef] [Medline]
    43. Wang M, Hung L, Lo Y. Glycemic control in type 2 diabetes: role of health literacy and shared decision-making. Patient Prefer Adherence 2019;13:871-879 [FREE Full text] [CrossRef] [Medline]
    44. Redelmeier DA, Shafir E. Pitfalls of judgment during the COVID-19 pandemic. Lancet Public Health 2020 Jun;5(6):e306-e308. [CrossRef] [Medline]
    45. Sentell T, Vamos S, Okan O. Interdisciplinary Perspectives on Health Literacy Research Around the World: More Important Than Ever in a Time of COVID-19. Int J Environ Res Public Health 2020 Apr 26;17(9):3010 [FREE Full text] [CrossRef] [Medline]
    46. Govt extends social distancing for at least one week in 28 localities. VGP News. 2020.   URL: http:/​/news.​chinhphu.vn/​Home/​Govt-extends-social-distancing-for-at-least-one-week-in-28-localities/​20204/​39735.​vgp [accessed 2020-04-15]
    47. Govt to gradually ease COVID-19 control measure in cautious manner. VGP News. 2020.   URL: http:/​/primeminister.​chinhphu.vn/​Home/​Govt-to-gradually-ease-COVID19-control-measure-in-cautious-manner/​20204/​3759.​vgp [accessed 2020-04-20]
    48. Quan H, Li B, Couris CM, Fushimi K, Graham P, Hider P, et al. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol 2011 Mar 15;173(6):676-682. [CrossRef] [Medline]
    49. Charlson ME, Pompei P, Ales KL, MacKenzie C. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40(5):373-383. [CrossRef] [Medline]
    50. Duong TV, Aringazina A, Kayupova G, Nurjanah, Pham TV, Pham KM, et al. Development and Validation of a New Short-Form Health Literacy Instrument (HLS-SF12) for the General Public in Six Asian Countries. Health Lit Res Pract 2019 Apr;3(2):e91-e102 [FREE Full text] [CrossRef] [Medline]
    51. Duong TV, Nguyen TTP, Pham KM, Nguyen KT, Giap MH, Tran TDX, et al. Validation of the Short-Form Health Literacy Questionnaire (HLS-SF12) and Its Determinants among People Living in Rural Areas in Vietnam. Int J Environ Res Public Health 2019 Sep 11;16(18):3346 [FREE Full text] [CrossRef] [Medline]
    52. Van Hoa H, Giang HT, Vu PT, Van Tuyen D, Khue PM. Factors Associated with Health Literacy among the Elderly People in Vietnam. Biomed Res Int 2020 Mar 27;2020:3490635-3490637. [CrossRef] [Medline]
    53. Nguyen HC, Nguyen MH, Do BN, Tran CQ, Nguyen TTP, Pham KM, et al. People with Suspected COVID-19 Symptoms Were More Likely Depressed and Had Lower Health-Related Quality of Life: The Potential Benefit of Health Literacy. J Clin Med 2020 Mar 31;9(4):965 [FREE Full text] [CrossRef] [Medline]
    54. Nguyen HT, Do BN, Pham KM, Kim GB, Dam HT, Nguyen TT, et al. Fear of COVID-19 Scale-Associations of Its Scores with Health Literacy and Health-Related Behaviors among Medical Students. Int J Environ Res Public Health 2020 Jun 11;17(11):4164. [CrossRef] [Medline]
    55. HLS-EU Consortium. Comparative report of health literacy in eight EU member states. The European Health Literacy Project. 2009.   URL: https://www.healthliteracyeurope.net/hls-eu [accessed 2012-10-22]
    56. Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [CrossRef] [Medline]
    57. World Health Organization. Risk assessment and management of exposure of health care workers in the context of COVID-19: Interim guidance. 2020.   URL: https:/​/www.​who.int/​publications/​i/​item/​risk-assessment-and-management-of-exposure-of-health-care-workers-in-the-context-of-covid-19-interim-guidance [accessed 2020-03-19]
    58. Editorial Team. Overview of novel coronavirus (2019-nCoV). BMJ Best Practice. 2020.   URL: https://bestpractice.bmj.com/topics/en-gb/3000165 [accessed 2020-02-10]
    59. Field A. Discovering statistics using IBM SPSS statistics. Washington DC, USA: Sage Publications Ltd; 2013.
    60. Kimberlin C, Winterstein A. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm 2008 Dec 01;65(23):2276-2284. [CrossRef] [Medline]
    61. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007 Jan;60(1):34-42. [CrossRef] [Medline]
    62. Cronbach LJ, Shavelson RJ. My Current Thoughts on Coefficient Alpha and Successor Procedures. Educational and Psychological Measurement 2016 Jul 02;64(3):391-418. [CrossRef]
    63. Maldonado G, Greenland S. Simulation study of confounder-selection strategies. Am J Epidemiol 1993 Dec 01;138(11):923-936. [CrossRef] [Medline]
    64. van der Vaart R, Drossaert C. Development of the Digital Health Literacy Instrument: Measuring a Broad Spectrum of Health 1.0 and Health 2.0 Skills. J Med Internet Res 2017 Jan 24;19(1):e27 [FREE Full text] [CrossRef] [Medline]
    65. Sørensen K, Pelikan JM, Röthlin F, Ganahl K, Slonska Z, Doyle G, HLS-EU Consortium. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health 2015 Dec 05;25(6):1053-1058 [FREE Full text] [CrossRef] [Medline]
    66. Van Duong T, Chiu C, Lin C, Chen YC, Wong TC, Chang P, et al. E-healthy diet literacy scale and its relationship with behaviors and health outcomes in Taiwan. Health Promot Int 2020 Apr 08:E. [CrossRef] [Medline]
    67. Zrubka Z, Hajdu O, Rencz F, Baji P, Gulácsi L, Péntek M. Psychometric properties of the Hungarian version of the eHealth Literacy Scale. Eur J Health Econ 2019 Jun 16;20(Suppl 1):57-69 [FREE Full text] [CrossRef] [Medline]
    68. Shiferaw KB, Mehari EA. Internet use and eHealth literacy among health-care professionals in a resource limited setting: a cross-sectional survey. Adv Med Educ Pract 2019;10:563-570 [FREE Full text] [CrossRef] [Medline]
    69. Palesy D, Jakimowicz S. Health literacy training for Australian home care workers: Enablers and barriers. Home Health Care Serv Q 2019 Apr 20;38(2):80-95. [CrossRef] [Medline]
    70. Duong T, Sørensen K, Pelikan JM, Van den Broucke S, Lin I, Lin Y, et al. Health-related behaviors moderate the association between age and self-reported health literacy among Taiwanese women. Women Health 2018 Jul 06;58(6):632-646. [CrossRef] [Medline]
    71. Kim D, Chang H. Key functional characteristics in designing and operating health information websites for user satisfaction: an application of the extended technology acceptance model. Int J Med Inform 2007;76(11-12):790-800. [CrossRef] [Medline]
    72. Duong TV, Aringazina A, Baisunova G, Nurjanah, Pham TV, Pham KM, et al. Measuring health literacy in Asia: Validation of the HLS-EU-Q47 survey tool in six Asian countries. J Epidemiol 2017 Feb;27(2):80-86 [FREE Full text] [CrossRef] [Medline]
    73. Elisabeth Stømer U, Klopstad Wahl A, Gunnar Gøransson L, Hjorthaug Urstad K. Health Literacy in Kidney Disease: Associations with Quality of Life and Adherence. J Ren Care 2020 Jun 16;46(2):85-94. [CrossRef] [Medline]
    74. Huang Y, Shiyanbola OO, Chan H. A path model linking health literacy, medication self-efficacy, medication adherence, and glycemic control. Patient Educ Couns 2018 Nov;101(11):1906-1913. [CrossRef] [Medline]
    75. Lor M, Koleck TA, Bakken S, Yoon S, Dunn Navarra A. Association Between Health Literacy and Medication Adherence Among Hispanics with Hypertension. J Racial Ethn Health Disparities 2019 Jun 3;6(3):517-524 [FREE Full text] [CrossRef] [Medline]
    76. Skoumalova I, Kolarcik P, Madarasova Geckova A, Rosenberger J, Majernikova M, Klein D, et al. Is Health Literacy of Dialyzed Patients Related to Their Adherence to Dietary and Fluid Intake Recommendations? Int J Environ Res Public Health 2019 Nov 05;16(21):4295 [FREE Full text] [CrossRef] [Medline]
    77. Li S, Feng B, Liao W, Pan W. Internet Use, Risk Awareness, and Demographic Characteristics Associated With Engagement in Preventive Behaviors and Testing: Cross-Sectional Survey on COVID-19 in the United States. J Med Internet Res 2020 Jun 16;22(6):e19782 [FREE Full text] [CrossRef] [Medline]
    78. Alsan M, Stantcheva S, Yang D, Cutler D. Disparities in Coronavirus 2019 Reported Incidence, Knowledge, and Behavior Among US Adults. JAMA Netw Open 2020 Jun 01;3(6):e2012403 [FREE Full text] [CrossRef] [Medline]
    79. Duong VT, Lin I, Sorensen K, Pelikan JM, Van Den Broucke S, Lin Y, et al. Health Literacy in Taiwan: A Population-Based Study. Asia Pac J Public Health 2015 Nov 29;27(8):871-880. [CrossRef] [Medline]
    80. Watson R. Europeans with poor "health literacy" are heavy users of health services. BMJ 2011 Nov 29;343(nov29 4):d7741-d7741. [CrossRef] [Medline]
    81. Greenhalgh T. Health literacy: towards system level solutions. BMJ 2015 Feb 24;350(feb24 13):h1026-h1026. [CrossRef] [Medline]
    82. Duong TV, Pham KM, Do BN, Kim GB, Dam HTB, Le VT, et al. Digital Healthy Diet Literacy and Self-Perceived Eating Behavior Change during COVID-19 Pandemic among Undergraduate Nursing and Medical Students: A Rapid Online Survey. Int J Environ Res Public Health 2020 Sep 30;17(19):7185 [FREE Full text] [CrossRef] [Medline]
    83. Britt RK, Collins WB, Wilson K, Linnemeier G, Englebert AM. eHealth Literacy and Health Behaviors Affecting Modern College Students: A Pilot Study of Issues Identified by the American College Health Association. J Med Internet Res 2017 Dec 19;19(12):e392 [FREE Full text] [CrossRef] [Medline]
    84. Mitsutake S, Shibata A, Ishii K, Oka K. Associations of eHealth Literacy With Health Behavior Among Adult Internet Users. J Med Internet Res 2016 Jul 18;18(7):e192 [FREE Full text] [CrossRef] [Medline]
    85. Owen L, Corfe B. The role of diet and nutrition on mental health and wellbeing. Proc Nutr Soc 2017 Nov;76(4):425-426. [CrossRef] [Medline]
    86. Ministry of Health. Coronavirus disease (COVID-19) outbreak in Vietnam. 2020.   URL: https://ncov.moh.gov.vn/ [accessed 2020-05-04]

    ‎
    B: unstandardized regression coefficient
    eHEALS: eHealth Literacy Scale
    HCWs: health care workers
    HL: health literacy
    HLS-SF12: 12-item short-form health literacy questionnaire
    IPC: infection prevention and control
    KMO: Kaiser-Meyer Olkin Measure
    PCA: principal component analysis
    PPE: personal protective equipment
    WHO: World Health Organization

    Edited by G Eysenbach; submitted 26.07.20; peer-reviewed by M Messer, N Werts, R Batten, A Stylianides; comments to author 23.09.20; revised version received 25.09.20; accepted 28.10.20; published 12.11.20

    Copyright

    ©Binh N Do, Tien V Tran, Dung T Phan, Hoang C Nguyen, Thao T P Nguyen, Huu C Nguyen, Tung H Ha, Hung K Dao, Manh V Trinh, Thinh V Do, Hung Q Nguyen, Tam T Vo, Nhan P T Nguyen, Cuong Q Tran, Khanh V Tran, Trang T Duong, Hai X Pham, Lam V Nguyen, Kien T Nguyen, Peter W S Chang, Tuyen Van Duong. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 12.11.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.