This paper is in the following e-collection/theme issue:

eHealth Literacy / Digital Literacy (390) Research Instruments, Questionnaires, and Tools (576) Development of Instruments and Surveys (86) Tools and Questionnaires in Human Factors Evaluation (51)

Published on in Vol 20, No 4 (2018): April

The Korean eHealth Literacy Scale (K-eHEALS): Reliability and Validity Testing in Younger Adults Recruited Online

The Korean eHealth Literacy Scale (K-eHEALS): Reliability and Validity Testing in Younger Adults Recruited Online

The Korean eHealth Literacy Scale (K-eHEALS): Reliability and Validity Testing in Younger Adults Recruited Online

Authors of this article:

SeonYoon Chung1 Author Orcid Image ;   Bu Kyung Park2 Author Orcid Image ;   Eun-Shim Nahm3 Author Orcid Image
Article Authors Cited by (76) Tweetations (14) Metrics

    Original Paper

    1Mennonite College of Nursing, Illinois State University, Normal, IL, United States

    2College of Nursing, Research Institute of Nursing Science, Kyungpook National University, Daegu, Republic Of Korea

    3School of Nursing, University of Maryland, Baltimore, MD, United States

    Corresponding Author:

    Bu Kyung Park, RN, PhD

    College of Nursing, Research Institute of Nursing Science

    Kyungpook National University

    640 Gukchaebosang-ro Jung-gu

    Daegu, 41944

    Republic Of Korea

    Phone: 82 53 420 4929

    Fax:82 53 421 2758

    Email: bukpark@knu.ac.kr


    Background: In this digital era, eHealth literacy is an essential skill set to leverage health information available online to promote health outcomes. South Korea has an advanced health information technology infrastructure, including widespread use of the internet and mobile phones. A few studies have explored eHealth literacy in South Korea using translated versions of the eHEALS; however, they were not fully validated. A unified reliable and valid assessment tool is critical to assess and enhance the eHealth literacy level across the population.

    Objective: The aim was to develop a Korean version of eHealth Literacy Scale (K-eHEALS) and evaluate its reliability and validity employing healthy young adults in Korea.

    Methods: The K-eHEALS was developed based on eHEALS, a widely used tool that measures eHealth literacy, and was validated using a sample of 500 young adults recruited from a pool of a Korean internet survey panel. Content validity was assessed using the content validity index (CVI) for individual items and for scale. Construct validity was examined using exploratory factor analysis and hypothesis testing. The Cronbach alpha coefficient was used to determine the internal consistency and the Pearson correlation coefficient was used to evaluable the stability of the measure (n=55).

    Results: Both individual and scale CVIs were acceptable (individual CVIs>0.67; scale CVI=0.83). Single factors accounting for 50.3% of the variance in the scales were extracted revealing the unidimensional latent structure of K-eHEALS. Hypothesis testing showed significant association between eHealth literacy and hours of internet use per day, supporting the construct validity. Items of the K-eHEALS were internally consistent (Cronbach alpha=.88) and stable over a 1-month period (r=.754, P<.001).

    Conclusions: The findings of this study suggest that K-eHEALS is a valid and reliable measure of eHealth literacy in Korean young adults. Additional studies are needed with more diverse groups of adults in Korea.

    J Med Internet Res 2018;20(4):e138

    doi:10.2196/jmir.8759

    Keywords

    eHEALS; eHealth; literacy; reliability; validity 


    In this digital era, the internet and mobile devices are integral to our daily life and the majority of the population uses the internet and social media to find health information [1]. Literacy in eHealth is an essential skillset to leverage these resources and produce better outcomes [2]. The eHealth Literacy Scale (eHEALS) is a measure to assess eHealth literacy initially developed in Canada by Norman and Skinner [3]. It is based on the Lily model that outlines six core literacies: (1) traditional literacy, (2) health literacy, (3) information literacy, (4) scientific literacy, (5) media literacy, and (6) computer literacy. The validity and reliability of eHEALS have been evaluated in numerous age groups and it has been translated globally [4-12]. However, the existing Korean versions of eHEALS (K-eHEALS) are limited in the number of items translated for use or have not been tested for their reliability and validity.

    Korea has advanced health information technology infrastructure including widespread use of the internet and mobile phones [13,14]. In Korea, the concept of health literacy was introduced in the 2000s and a Korean health literacy assessment tool was first developed in 2005 [15]. This tool, Korean Health Literacy Assessment Tool, was developed by translating and modifying the Rapid Estimate of Adult Literacy in Medicine (REALM) [16] in the context of Korean culture [15]. More recently, the most frequently used tools were the Korean version of REALM, Test of Functional Health Literacy in Adults, and Newest Vital Sign [17]. These tools, however, are limited in assessing eHealth literacy.

    A few studies have explored eHealth literacy in Korea. For example, Lee et al [18] examined how eHealth literacy affects communication between patients and doctors. Cho et al [19] examined the effects of cognitive factors including eHealth literacy on the health app use. More recently, eHealth literacy was assessed among nursing students in Korea [20] and relationships between eHealth literacy and health behaviors were examined in Korean adults [21]. These studies used a translated version of eHEALS [3] to measure eHealth literacy. The translated version of the eHEALS used in those studies, however, was not fully validated [18,20,21]. Not having a reliable and valid Korean version of eHealth literacy scale can significantly limit eHealth literacy in Korea that is necessary to optimize the development of interventions aimed at promoting eHealth literacy. This highlights the importance of having a unified and reliable assessment tool to compare eHealth literacy levels across the population. Thus, the purpose of this study was to develop a full eight-item K-eHEALS and evaluate its reliability and validity among healthy young adults in Korea.


    Design

    In this study, we translated eHEALS and tested the psychometric aspects of the measure using a survey. The initial data collection was conducted from September 5 to 16, 2016. To test the stability of the measure, another wave of the survey was conducted from October 3 to 10, 2016.

    Participants

    The study participants were recruited (N=500) from a pool of registrants of Banana Lab, a Korean internet survey panel service agency. The survey was conducted using SurveyMonkey (SurveyMonkey Inc, Palo Alto, CA, USA). Banana Lab is an exclusive survey panel service agency working with SurveyMonkey Korea that had approximately 437,511 voluntarily registered participants in 2016. The sample size was determined based on previous studies that used a subject-to-item ratio greater than or equal to two or a sample size greater than or equal to 100 to validate a scale [22]. Potential participants who met inclusion criteria (younger adults aged 20-39 years in South Korea) received an email asking to respond to an online survey. A proportioned quota sampling by sex and age was used to minimize selection bias. For equal distribution of the younger adults, we recruited equal sample size for four sample groups by sex and age (age 20-29 male: 25%; age 20-29 female: 25%; age 30-39 male: 25%; age 30-39 female: 25%). The survey panel service agency randomly selected the potential participants from a list of registered participants with demographic information. The survey was sent out via email to potential participants in accordance to the proportion for stratified sampling, and this continued until it met our target number of participants in each quota.

    Among the participants who completed the K-eHEALS, a subset of participants was randomly selected and invited to complete the same survey for test-retest reliability testing after 1 month from the initial response. If randomly selected participants did not respond, the next randomly selected participant was invited until 55 participants completed the survey. After the targeted number of 55 participants voluntarily signed and completed the survey, the online survey was closed.

    Measurement

    Sociodemographic Attributes and Internet Use Behaviors

    The survey included items on demographics and internet use behaviors, including hours of internet use per day, purpose of internet use, types of health information searched, level of trust for health information, and its usefulness. Previous studies found internet usage was related to eHealth literacy; thus, hours of internet use was measured to validate construct validity of K-eHEALS.

    The eHEALS

    The eHEALS is composed of eight items measuring eHealth literacy on a 5-point Likert scale (1=strongly disagree, 5=strongly agree), with a total score that ranges from 8 to 40, in which a higher score indicates higher literacy. The eight items measure perceived knowledge, skills, and confidence in locating, evaluating, and using electronic health information to make health decisions. The measure also includes two additional questions that are not included in the total score. These two questions assess the perception of the internet as a tool to assess health information and make decisions about health.

    The K-eHEALS

    The K-eHEALS was developed following the process of translation and adaptation of instrument proposed by the World Health Organization [23]. After we acquired permission from the original developers of the eHEALS (Dr Cameron and Dr Norman), two bilingual professionals in nursing (two of the authors) conducted forward translation independently and compared the translated instrument. Next, a bilingual expert panel, consisting of four faculty members of nursing schools in Korea and the United States and two professionals in computer and information technology, evaluated the translated instrument. All six members of the expert panel rated each item of the K-eHEALS in terms of its relevance to the underlying construct on a 4-point scale (1=not relevant, 2=somewhat relevant, 3=quite relevant, 4=highly relevant) [24]. In addition, the expert panel commented on each item if they had any suggestions or questions. Through the expert panel discussion, the translated instrument was adjusted and the complete K-eHEALS was produced. Then, back-translation of the K-eHEALS was done by an independent translator and was compared against the original eHEALS. Multimedia Appendix 1 is the final version of the K-eHEALS.

    Ethical Considerations

    The study was reviewed and approved as an exempt study by the corresponding author’s Institutional Review Boards (1041078-201608-HRSB-151-01). The data collection was conducted by a third party, Banana Lab, a Korean internet survey panel service agency; the researchers were not in direct contact with the potential participants and received deidentified survey data. Eligible individuals who were interested in the study received an email including a link to the online consent form, which explained the study and other information listed in the face-to-face consent form. On review of the form, those who agreed to participate in the survey typed their name and clicked on the “I agree” button to consent to participate in the study and proceed to the survey.

    Data Analysis

    Validity
    Content Validity

    To determine the content validity index (CVI) for individual items, six members of the expert panel rated each item in terms of its relevance to the underlying construct on a 4-point scale. Then, individual CVI was computed for each item as the number of experts giving a rating of either 3 or 4, divided by the number of experts (the proportion of agreement about relevance). The CVI for the scale was calculated as the mean of the individual CVIs for all items on the scale [25]. An individual CVI higher than 0.78 was considered excellent; a scale CVI higher than 0.80 was considered acceptable [26].

    Construct Validity: Exploratory Factor Analysis

    After content validity was confirmed, the Korean eHEALS was administered to young adults through an online survey. Exploratory factor analysis was conducted to ensure the construct validity [27]. Sufficiency of the sample size relative to the number of items was determined using the Kaiser-Meyer-Olkin value (>0.70) and factorability of the data were evaluated based on the Bartlett test of sphericity. A scree plot and the eigenvalue (>1) were used to determine the number of factors to be extracted [28].

    Construct Validity: Hypothesis Testing

    Construct validity was also assessed using a hypothesis-testing approach [27]. Based on prior studies [2,12,29], we hypothesized that young adults who used the internet for more hours would have higher eHealth literacy scores. Both analysis of variance and Tukey post hoc analyses were used to test the association between duration of internet use and eHealth literacy.

    Reliability
    Interitem Consistency

    Interitem reliability was calculated by Cronbach alpha. A value of .70 or higher was considered acceptable [30].

    Stability of the Measure

    Test-retest reliability testing was conducting using Pearson correlation.


    Characteristics of the Young Adult Participants

    The total number of young adults included in the study was 500. Half of the participants were male (n=250). Half of the participants were in their twenties, whereas the other half were in their thirties. More than two-thirds of the participants were single (72.6%, 363/500) and had at least some university education (437/500, 87.4%) (Table 1). The highest percentage of participants used the internet for 1 to 3 hours per day (personal computer; PC: 38.6%, 193/500; portable device: 49.2%, 246/500), followed by 4 to 7 hours per day (PC: 29.6%, 148/500; portable device: 23.6%, 118/500). More than half of the participants (54.6%, 273/500) used the internet to search for information. Types of health information searched for included healthy lifestyle (45.0%, 225/500), disease (32.8%, 164/500), and treatment and medicine (15.8%, 79/500). More than half (53.6%, 268/500) of the participants neither trusted nor distrusted health information online, but 64.8% (324/500) reported the internet is useful in making decisions about health and 64.4% (322/500) reported the internet is important to have access to health resources. The mean total score on the K-eHEALS was 28.06 (SD 4.80, range 8-40) (Table 1). The mean of items in the K-eHEALS was 3.51 ranging from 3.31 to 3.68 (item range 1-5) (Table 2).

    Content Validity

    The individual CVI were excellent, scoring higher than 0.78 for all items except for item 3 (“I know how to find helpful health resources on the internet”) and item 6 (“I have the skills I need to evaluate the health resources I find on the internet”). Although scoring was lower than 0.78 for two items, the experts commented that the underlying construct for these two items were apparent but they gave them a low individual CVI score because of the low fluency of the translation. Therefore, the K-eHEALS was used after editing items 3 and 6 per the experts’ suggestions. The scale CVI was acceptable, scoring higher than 0.80 (scale CVI=0.83) (Table 3).

    Table 1. Characteristics of the young adult participants (N=500).
    Characteristicsn (%)
    Gender

    		Male250 (50.0)

    		Female250 (50.0)
    Age group (years)

    		20-29250 (50.0)

    		30-39250 (50.0)
    Education level

    		High school diploma166 (33.2)

    		University degree and above334 (66.8)
    Marital status

    		Single363 (72.6)

    		Married137 (27.4)
    Internet use on personal computer per day (hours)

    		<155 (11.0)

    		1-3193 (38.6)

    		4-7148 (29.6)

    		>8104 (20.8)
    Internet use on phone per day (hours)

    		l<177 (15.4)

    		1-3246 (49.2)

    		4-7118 (23.6)

    		>859 (11.8)
    Purpose of internet use

    		Social networking service (eg, Kakaotalk,a Instagram)171 (34.2)

    		Searching information273 (54.6)

    		Game30 (6.0)

    		Others26 (5.2)
    Health information searched for

    		Disease164 (32.8)

    		Healthy lifestyle225 (45.0)

    		Medicine36 (7.2)

    		Treatment43 (8.6)

    		Medical personnel4 (0.8)

    		Others28 (5.6)
    Engine used to search health information

    		Google38 (7.6)

    		Naver423 (84.6)

    		Daum32 (6.4)

    		YouTube5 (1.0)

    		Others2 (0.4)
    Level of trust for health information

    		Strongly trust11 (2.2)

    		Quite trust187 (37.4)

    		Neutral268 (53.6)

    		Quite distrust31 (6.2)

    		Never trust3 (0.6)
    How useful do you feel the internet is in helping you in making decisions about your health?

    		Not useful at all4 (0.8)

    		Not useful29 (5.8)

    		Unsure120 (24.0)

    		Useful324 (64.8)

    		Very useful23 (4.6)
    How important is it for you to be able to access health resources on the internet?

    		Not important at all4 (0.8)

    		Not important22 (4.4)

    		Unsure105 (21.0)

    		Important322 (64.4)

    		Very important47 (9.4)

    aKakaotalk is one of the most popular messenger apps in South Korea.

    Table 2. Total and item means for the K-eHEALS in young adult participants (N=500).
    K-eHEALS itemsMean (SD)
    1. I know what health resources are available on the internet3.53 (0.76)
    2. I know where to find helpful health resources on the internet3.47 (0.80)
    3. I know how to find helpful health resources on the internet3.59 (0.80)
    4. I know how to use the internet to answer my questions about health3.68 (0.77)
    5. I know how to use the health information I find on the internet to help me3.62 (0.77)
    6. I have the skills I need to evaluate the health resources I find on the internet3.31 (0.85)
    7. I can tell high quality from low quality health resources on the internet3.41 (0.87)
    8. I feel confident in using information from the internet to make health decisions3.44 (0.81)
    Total means3.51

    Construct Validity-Exploratory Factor Analysis

    The results supported the validity of the K-eHEALS. The Bartlett test of sphericity was significant (χ228=1859.0, P<.001) suggesting the factorability of the correlation matrix. The results of the Kaiser-Meyer-Olkin test (0.91) was high, showing adequate sampling relative to the number of items present. Based on the initial eigenvalue (4.52) and the scree plot that was suggestive of a unidimensional latent structure (Figure 1), a single factor was retained. In this single factor model, the sum of squared loadings of the eight items on the extracted factor based on maximum likelihood method was 4.02, explaining 50.3% of the variance in the scale (Table 4).

    Construct Validity-Hypothesis Testing

    The results from hypothesis testing further supported the construct validity of the K-eHEALS (Table 5). There was a significant association between eHealth literacy and the hours of internet use per day using PC (F4,106.3=5.608, P<.001). Post hoc analysis showed that the difference was evident between adults using internet on PC for less than 1 hour per day compared to other groups that used more than 1 hour: 1 to 3 hours, 4 to 7 hours, or 8 to 11 hours. Similarly, there was significant association between eHealth literacy and hours of internet use per day using a portable device (F4,98.0=4.610, P=.002). Post hoc analysis showed that the difference in mean eHealth literacy was evident between adults using internet on a portable device for less than 1 hour per day compared to other groups that used more than 1 hour: 1 to 3 hours, 4 to 7 hours, or more than 12 hours. The difference was also shown between those using the Web for 8 to 11 hours and more than 12 hours.

    Table 3. Individual content validity index (CVI) and scale CVI scores for the Korean eHEALS (K-eHEALS).
    K-eHEALS ItemsExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Individual CVI
    1. I know what health resources are available on the internet3333240.83
    2. I know where to find helpful health resources on the internet3433240.83
    3. I know how to find helpful health resources on the internet4424240.67
    4. I know how to use the internet to answer my questions about health4443240.83
    5. I know how to use the health information I find on the internet to help me3443441.00
    6. I have the skills I need to evaluate the health resources I find on the internet2423330.67
    7. I can tell high quality from low quality health resources on the internet3424440.83
    8. I feel confident in using information from the internet to make health decisions4434341.00
    Scale CVI





    		0.83
    ‎
    Figure 1. Scree Plot of the K-eHEALS.
    View this figure
    Table 4. Factor loadings and factor score coefficients for the K-eHEALS and single extracted factor (N=500).
    K-eHEALS ItemsFactor loadingsa
    1. I know what health resources are available on the internet0.758
    2. I know where to find helpful health resources on the internet0.737
    3. I know how to find helpful health resources on the internet0.762
    4. I know how to use the internet to answer my questions about health0.787
    5. I know how to use the health information I find on the internet to help me0.705
    6. I have the skills I need to evaluate the health resources I find on the internet0.623
    7. I can tell high quality from low quality health resources on the internet0.616
    8. I feel confident in using information from the internet to make health decisions0.664
    Sums of squared loadings4.022

    a Extraction method: maximum likelihood. One factor extracted, four iterations required.

    Table 5. Participant eHealth literacy by hours of internet use (N=500).
    Hours of internet useNMean (SD)P valueF (df1, df2)P value
    Internet use on personal computer (hours)50028.06 (4.81)
    		5.608 (4,106.30)<.001

    		<15525.60 (5.91)—


    		1-319327.68 (5.30).032a


    		4-714828.63 (3.89).001a


    		8-118329.34 (3.41)<.001a


    		>12 hours2128.90 (5.09).050a

    Internet use on phone (hours)50028.06 (4.81)
    		4.610 (4,98.03).002

    		<17726.30 (5.90)—


    		1-324628.13 (4.07).026b


    		4-711828.86 (4.43).002b


    		8-113226.81 (6.90).986b


    		>122730.48 (4.52).001b

    aVersus <1 hour internet use on personal computer.

    bVersus <1 hour internet use on phone.

    Reliability

    Regarding interitem reliability, the calculated Cronbach alpha coefficient was .88, suggesting that the K-eHEALS was internally consistent. The measure also showed stability over time as evidenced by high test-retest reliability (r=.754, P<.001).


    Principal Findings

    This study is the first attempt to translate a full version of eHEALS into Korean and to test its psychometric aspects. Although additional psychometric testing is necessary to further establish validity of this measure in this population, results of this study are promising and support the validity and reliability of K-eHEALS. Content validity was acceptable (individual CVIs>0.67, scale CVI=0.83), and so was construct validity as supported by unidimensional latent structure of K-eHEALS and significant association between eHealth literacy and hours of internet use per day. In terms of reliability, the items of K-eHEALS were internally consistent (Cronbach alpha=.88) and stable over a 1-month period (r=.754, P<.001). Therefore, the results of this study reveal that the translated K-eHEALS is reliable.

    The eHEALS is a measure that has been globally validated in multiple languages, including Japanese [10], Dutch [5], Spanish [11], Chinese [9], German [6], Italian [4], Iranian [8], Hebrew [12], and Turkish [7]. In Korea, several studies have used a translated version of eHEALS, but those studies did not provide psychometric evaluation of the eHEALS in Korean [18,20,21].

    Content validity of K-eHEALS reported here was evaluated by six experts and it showed good scale CVI and individual CVI except for two items. However, previous studies on various language versions of eHEALS did not conduct content validity or report the CVI score (all six). Only one study, an Iranian version of eHEALS [8], measured face validity by four experts, but did not evaluate the CVI score. Evaluation of content validity is proposed for future psychometric studies to enhance construct validity of all instruments [26,31].

    Construct validity of the K-eHEALS was evaluated by exploratory factor analysis and hypothesis testing. The K-eHEALS showed a monofactorial unidimensional structure and explained 50.3% of the variance in the measure. This finding supports previous studies that yielded a single factor solution explaining from 52.6% (Spanish version) to 70.5% (Iranian version) of variance in the measure.

    Internal consistency of the K-eHEALS was .88, which was comparable with previous findings ranging from .78 (Turkish version) to .93 (Japanese version). Test-retest reliability of the K-eHEALS showed stability (r=.754, P<.001), with the r coefficient within range of previous studies (r=.63 for Japanese version and r=.96 for Iranian version). Therefore, the K-eHEALS is a reliable and valid tool compared to other translated versions of eHEALS.

    In South Korea, a higher percentage of Koreans use the internet to search and read health information (66.4%) compared to those who seek information from mass media (40.8%) or a health care provider (11.8%) [32]. Moreover, there is an increasing interest in the use of personal health records (PHRs). For young adults, eHealth literacy is essential for effective use of online resources and their PHR to assist in self-management and promote health conditions. Acknowledging the importance of eHealth literacy, the US Office of Disease Prevention and Health Promotion specified the need for increased heath literacy, access to the internet, and use of health information technologies to promote health of the public in Healthy People 2020, a 10-year national objective for improving public health in the United States [33]. The Health Information Technology for Economic and Clinical Health (HITECH) Act [34] also encourages health care institutions to use electronic health records (EHRs), PHRs tethered to EHRs, and related technology for meaningful use of EHRs to improve health [35]. However, there is no prominent national initiative available to promote eHealth literacy in South Korea. More research aimed at understanding of the current level of eHealth literacy in this population is needed to develop more effective eHealth interventions to promote the health of the public. The K-eHEALS, which is a reliable and valid measure, can significantly contribute to these efforts [18].

    Limitations

    A main limitation of our study is that the participants were recruited from a pool of registrants of an internet survey panel service agency who are likely active online users and this group sample may not be representative of the general young adult population. Moreover, the results cannot be generalized to older adults because this was studied in young adults. Further studies employing diverse age groups are needed to address this issue.

    Conclusion

    To promote eHealth literacy, researchers and health care providers should first understand the eHealth literacy of the individuals. The psychometric findings from this study suggest that K-eHEALS is a reliable and valid measure of eHealth literacy in young Korean adults who are active online users. We hope K-eHEALS can help Korean researchers who conduct studies in eHealth by providing a reliable and valid measure that can properly gauge participants’ eHealth literacy and develop optimal interventions.

    Acknowledgments

    This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MISP) (No: NRF-2016R1C1B2013649).

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Korean version of eHEALS (K-eHEALS).

    PDF File (Adobe PDF File), 224KB
    1. Norman CD. Social media and health promotion. Glob Health Promot 2012 Dec;19(4):3-6. [CrossRef] [Medline]
    2. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res 2006 Jun 16;8(2):e9 [FREE Full text] [CrossRef] [Medline]
    3. Norman CD, Skinner HA. eHEALS: the eHealth Literacy Scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [CrossRef] [Medline]
    4. De Caro W, Corvo E, Marucci AR, Mitello L, Lancia L, Sansoni J. eHealth Literacy Scale: a nursing analysis and Italian validation. Stud Health Technol Inform 2016;225:949. [Medline]
    5. van Der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van De Laar MA. Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011 Nov 09;13(4):e86 [FREE Full text] [CrossRef] [Medline]
    6. Soellner R, Huber S, Reder M. The concept of eHealth literacy and its measurement. J Media Psychol 2014 Jan;26(1):29-38. [CrossRef]
    7. Coskun S, Bebis H. Psychometric evaluation of a Turkish version of the e-health literacy scale (e-heals) in adolescent. Gulhane Med J 2015;57(4):378. [CrossRef]
    8. Bazm S, Mirzaei M, Fallahzadeh H, Bazm R. Validity and reliability of Iranian version of eHealth Literacy Scale. J Commun Health Res 2016;5(2):121-130.
    9. Koo M, Norman CD, Hsiao-Mei C. Psychometric Evaluation of a Chinese Version of the eHealth Literacy Scale in school age children. Int Electron J Health Educ 2012;15:29-36.
    10. Mitsutake S, Shibata A, Ishii K, Okazaki K, Oka K. [Developing Japanese version of the eHealth Literacy Scale (eHEALS)]. Nihon Koshu Eisei Zasshi 2011 May;58(5):361-371. [Medline]
    11. Paramio PG, Almagro BJ, Hernando GA, Aguaded GJI. [Validation of the eHealth Literacy Scale (eHEALS) in Spanish university students]. Rev Esp Salud Publica 2015;89(3):329-338 [FREE Full text] [CrossRef] [Medline]
    12. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan;14(1):e19 [FREE Full text] [CrossRef] [Medline]
    13. Poushter J. Pew Research Center. 2016 Feb 22. Smartphone ownership and Internet usage continues to climb in emerging economies   URL: http:/​/www.​pewglobal.org/​2016/​02/​22/​smartphone-ownership-and-Internet-usage-continues-to-climb-in-emerging-economies/​ [accessed 2018-03-26] [WebCite Cache]
    14. Akamai. 2016. State of the Internet Report 2016   URL: https:/​/www.​akamai.com/​kr/​ko/​about/​news/​press/​2016-press/​akamai-first-quarter-2016-state-of-the-internet-connectivity-report.​jsp [accessed 2018-03-26] [WebCite Cache]
    15. Kim S, Kim S, Lee S. Health literacy: development of a Korean health literacy assessment tool. Korean J Heal Educ Promot 2005;22(4):215-227.
    16. Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, et al. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med 1993 Jun;25(6):391-395. [Medline]
    17. Kim S, Oh J, Lee Y. Health literacy: an evolutionary concept analysis. J Korean Acad Soc Nurs Educ 2013;19(4):558-570 [FREE Full text]
    18. Lee B, Byoun W, Lim J. The influence of individual's e-Health literacy on doctor-patient communication. J Cybercommunication Acad Soc 2010;27(3):89-125 [FREE Full text]
    19. Cho J, Park D, Lee HE. Cognitive factors of using health apps: systematic analysis of relationships among health consciousness, health information orientation, eHealth literacy, and health app use efficacy. J Med Internet Res 2014 May 09;16(5):e125 [FREE Full text] [CrossRef] [Medline]
    20. Park H, Lee E. Self-reported eHealth literacy among undergraduate nursing students in South Korea: a pilot study. Nurse Educ Today 2015 Feb;35(2):408-413. [CrossRef] [Medline]
    21. Kim S, Son Y. Relationships between eHealth literacy and health behaviors in Korean adults. Comput Inform Nurs 2017 Feb;35(2):84-90. [CrossRef] [Medline]
    22. Anthoine E, Moret L, Regnault A, Sébille V, Hardouin J. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes 2014 Dec 09;12:176 [FREE Full text] [CrossRef] [Medline]
    23. World Health Organization. 2016. Process of translation and adaptation of instruments   URL: http://www.who.int/substance_abuse/research_tools/translation/en/ [accessed 2018-03-26] [WebCite Cache]
    24. Davis LL. Instrument review: getting the most from a panel of experts. Appl Nurs Res 1992 Nov;5(4):194-197. [CrossRef]
    25. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007 Aug;30(4):459-467. [CrossRef] [Medline]
    26. Polit D, Beck C. Nursing Research: Generating and Assessing Evidence for Nursing Practice. 10th ed. Philadelphia, PA: Wolters Kluwer; 2017.
    27. Waltz C, Strickland O, Lenz E. Measurement in Nursing and Health Research. 4th ed. New York: Springer; 2010.
    28. Tabachnick B, Fidell L. Using Multivariate Statistics. 6th ed. Harlow, UK: Pearson Education Limited; 2014.
    29. Chung S, Nahm E. Testing reliability and validity of the eHealth Literacy Scale (eHEALS) for older adults recruited online. Comput Inform Nurs 2015 Apr;33(4):150-156. [CrossRef] [Medline]
    30. Nunnally J, Bernstein I. Psychometric Theory. New York: McGraw-Hill; 1994.
    31. US Department of Health and Human Services FDA Center for Drug Evaluation and Research, US Department of Health and Human Services FDA Center for Biologics Evaluation and Research, US Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes 2006 Oct 11;4:79 [FREE Full text] [CrossRef] [Medline]
    32. Cho KW, Kam S, Chae YM. Analysis of Internet usage patterns of health consumers for Internet health information assessment criteria. Korean J Heal Educ Promot 2007;24(2):15-28.
    33. Office of Disease Prevention and Health Promotion. 2016. Healthy People 2020: health communication and health information technology   URL: https:/​/www.​healthypeople.gov/​2020/​topics-objectives/​topic/​health-communication-and-health-information-technology/​objectives [accessed 2018-03-26] [WebCite Cache]
    34. HealthIT.gov. HITECH programs & advisory committees   URL: https://www.healthit.gov/policy-researchers-implementers/hitech-programs-advisory-committees [accessed 2018-03-26] [WebCite Cache]
    35. HealthIT.gov. Meaningful use definition and objectives   URL: https://www.healthit.gov/providers-professionals/meaningful-use-definition-objectives [accessed 2018-03-26] [WebCite Cache]

    ‎
    CVI: content validity index
    eHEALS: eHealth Literacy Scale
    EHR: electronic health record
    K-eHEALS: Korean version of eHealth Literacy Scale
    PHR: personal health record
    REALM: Rapid Estimate of Adult Literacy in Medicine

    Edited by G Eysenbach; submitted 17.08.17; peer-reviewed by E Neter, J Cho; comments to author 29.11.17; revised version received 03.01.18; accepted 24.02.18; published 20.04.18

    Copyright

    ©SeonYoon Chung, Bu Kyung Park, Eun-Shim Nahm. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 20.04.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.